superresolution in fluorescence and diffraction microscopies with m ultiple i lluminations
DESCRIPTION
Superresolution in Fluorescence and Diffraction Microscopies with M ultiple I lluminations. - Jules Girard -. 2 December 2011. Introduction : Imaging with optics and resolution. Imaging device. Parameter of interest. Probing function. Detector. FT. Low-pass filter. - PowerPoint PPT PresentationTRANSCRIPT
Superresolution in Fluorescence and Diffraction Microscopies with
Multiple Illuminations
- Jules Girard -
2 December 2011
1/27
Introduction : Imaging with optics and
resolution
Probing function
Parameter of interest
× 𝑓 𝑖𝑙𝑙𝑀= 𝑓 𝑜𝑏𝑗 ∗𝑝𝑠𝑓()
Dete
ctor
Imaging device
×~𝑝𝑠𝑓
(~𝑓 𝑜𝑏𝑗 ∗~𝑓 𝑖𝑙𝑙)( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙) ~
𝑀 𝑀Low-pass filter
~𝑀=(~𝑓 𝑜𝑏𝑗∗~𝑓 𝑖𝑙𝑙)×~𝑝𝑠𝑓
FT
𝑀=( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙)∗𝑝𝑠𝑓 ~𝑀=(~𝑓 𝑜𝑏𝑗∗~𝑓 𝑖𝑙𝑙)×~𝑝𝑠𝑓
ky
kx
~𝑓 𝑜𝑏𝑗 (~𝑓 𝑜𝑏𝑗 ∗
~𝑓 𝑖𝑙𝑙)
→
Introduction : Extend resolution with
illumination
More generally :
=
~𝑓 𝑖𝑙𝑙
∗ ky
kx
ky
kx
×~𝑝𝑠𝑓
W. Lukosz and M. Marchand, Optica Acta 10, 241-255 (1963).W. Lukosz, JOSA 56, 1463 (1966).
By using multiple and inhomogeneous illuminations, we can shift high frequency parts of the object spatial spectrum into the passband defined
by the psf
2/27
Introduction : Reconstruct a super-resolution
image𝑀 𝑖= ( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙 ,𝑖 )∗𝑝𝑠𝑓 ~𝑀 𝑖=(~𝑓 𝑜𝑏𝑗∗~𝑓 𝑖𝑙𝑙 , 𝑖)×~𝑝𝑠𝑓
Inversion → numerical data processing
2 cases
is known is unknown
Non-linear inversion Find both and with the use of constraints
3/27
(𝑖=1. .𝑁 )
« Direct » inversionwith analytical approach
Presentation Outline
I. Optical Diffraction Tomography
II. Structured Illumination Fluorescence Microscopy
4/27
𝑀=( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙)∗𝑝𝑠𝑓 ~𝑀=(~𝑓 𝑜𝑏𝑗∗~𝑓 𝑖𝑙𝑙)×~𝑝𝑠𝑓
=
I. Optical Diffraction Tomography
Objective
Fourier Space
𝜽 𝒊𝒏𝒄
𝐸𝑡𝑜𝑡 ( 𝑥 , 𝑦 ,𝑧 )=𝐸𝑟𝑒𝑓 ( 𝑥 , 𝑦 ,𝑧 )+𝐸𝑑 ( 𝑥 , 𝑦 ,𝑧 )
II. Optical Diffraction Tomography
≠ illuminations → ≠ → access to ≠ parts of
~𝐸𝑑(�⃗�)=
~𝑀= (~𝑓 𝑜𝑏𝑗 ( �⃗� )∗~𝑓 𝑖𝑙𝑙 ( �⃗� ))×~𝑝𝑠𝑓 ( �⃗�)
We measure :
~Δ 𝜀(Sample dielectric
permittivity contrast)
~Etot(Total internal electric field)
= 0 for lateral
frequencies >
Reconstruct : quantitative microscopy of unstained sample
�⃗�
5/27
𝑥
𝑧
E Wolf, Optics Communications 1, 153-156 (1969).
V Lauer, Journal of Microscopy 205, 165-76 (2002).
Laser (λ=633n
m)
CC
D
Phase modulator
(G. Maire, F. Drsek, H.Giovannini)
Sample
Experiment
Calibration and normalization
→
Inversion : ) →
II. Optical Diffraction Tomography
Illumination with « plane waves » under ≠ incidences
Measure complex values of
6/27
Low : Born Approximation
is diffraction limited → Abbe limit
High : Multiple Scattering Regime
depends on object and illumination
is not diffraction limited → resolution improvement ? ( ?)
1. 2.
~Δ 𝜀
~Etot
=
𝒌𝟎𝑵𝑨 𝒌𝟎𝑵𝑨𝟐𝒌𝟎 𝑵𝑨𝒅=𝝀 /(𝟐𝑵𝑨)
II. Optical Diffraction Tomography
7/27
air
glass
50 nm
50 nm
25 nm
zx
50°
simulations
• λ = 633 nm
• Abbe limit with NA = 1.5
→ 211 nm
= 10-2
Low
|𝐸 𝑡𝑜𝑡| = 28.8
High (Ge)
<
!
|𝐸 𝑡𝑜𝑡|
II. Optical Diffraction Tomography
8/27
11
air
glass
50 nm
50 nm
25 nm
zx Germanium rods
TIRF configuration (10
angles)
NA = 1.3
→ Abbe limit : 245 nm
Experimental validation
(A. Talneau – LPN)
II. Optical Diffraction Tomography
0
Z (
µm
)
0,5
0
0,5
Z (
µm
)
9/27
II. Optical Diffraction Tomography
We achieved quantitative reconstruction of the
permittivity map of unstained sample even with a
multiple scattering regime
Multiple scattering : drawback way to improve the
resolution of ODT far beyond diffraction limit
Conclusion
10/27
II. Structured Illumination in
Fluorescence microscopy on 2D
samples
(2D)
III. Structured Illumination Microscopy in
Fluorescence
Objective Tube Lense
CC
D
𝑀=( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙)∗𝑝𝑠𝑓
(field intensity)(fluorescence density)
(2D and 1D)
−2𝑘0 𝑁𝐴+2𝑘0 𝑁𝐴 𝑘𝑥
0,5
1
0𝑘𝑥
𝑘𝑦
11/27
ky
kx
~𝑓 𝑜𝑏𝑗
∗~𝑓 𝑖𝑙𝑙
Use periodic pattern →
III. Structured Illumination Microscopy in
Fluorescence𝑀=( 𝑓 𝑜𝑏𝑗× 𝑓 𝑖𝑙𝑙)∗𝑝𝑠𝑓
𝐼𝜌
~𝑀
¿�⃗�
~𝑀
R. Heintzmann and C. Cremer, SPIE, pp. 185-196. (1998)
Mats G L Gustafsson, Journal of Microscopy 198, 82-7
(2000).
Requirements for illumination pattern :• Accurate translation → needed for discrimination of the
3 copies • High contrast → higher SNR (no dim for shifted copies
of ) 12/27
Use of non-linearities : →
(R. Heintzmann et al., JOSA A, 19, 2002 & M G L Gustafsson, PNAS, 102, 2005)
High index substrate
→ limited n and/or absorption
Nanostructured devices with plasmonics
→ field bound to the structure + difficulties to cover a large area
III. Structured Illumination Microscopy in
FluorescenceLimit : Illumination pattern is diffraction limited :
= : twice better than classical WF
How can we reach higher frequencies ?
Get below diffraction limit (surface imaging)
13/27
Glass coverslip@ 633nm
a-Si layer @ 633nm
𝒙
z=0
𝒚
𝑘 𝑦
𝑘𝑥
Grating assisted Structured Illumination Microscopy
Dielectric resonant grating ≈ 2D waveguide + 2D sub-λ grating
�⃗�𝒊𝒏𝒄
�⃗�𝒎𝒐𝒅𝒆
Hexagonal geometry : 6 equivalent orientations → near isotropic
resolution
III. Structured Illumination Microscopy in Fluorescence
𝑧
Design optimization → numerical simulations14/27
Gratings fabrication process
2. Grating patterning(e-beam + RIE)
1. aSi deposition
(PECVD)
3. Planarization(A. Cattoni)
(J. Girard, A. Talneau, A. Cattoni LPN – CNRS)
A. Cattoni, A. Talneau, A-M Haghiri-Gosnet, J. Girard, A. Sentenac (oral presentation, MNE 2011)15/2
7
III. Structured Illumination Microscopy in Fluorescence
III. Structured Illumination Microscopy in
FluorescenceExcitation modes of the grating substrate
�⃗�𝒊𝒏𝒄
�⃗�𝒎𝒐𝒅𝒆
�⃗�𝒊𝒏𝒄
�⃗�𝒎𝒐𝒅𝒆
1 beam excitation 2 beams excitation
|𝐾 −1,0|≈1.3×(2𝑘𝑂 𝑁𝐴)
|𝐾 −1,0+2 �⃗�𝑖𝑛𝑐∥|<2𝑘𝑂 𝑁𝐴
|2 [ �⃗�− 1,0+�⃗�𝑖𝑛𝑐 ∥ ]|≈1.6×2𝑘𝑂 𝑁𝐴
rightleft
17/27
III. Structured Illumination Microscopy in
Fluorescence Control of orientation, phase
and incidence angle on the substrate (65°)
Dich
roïc
Mirr
or
Obje
ctive
(O
il, N
A 1.
49)
Experimental setup
16/27
III. Structured Illumination Microscopy in
FluorescenceGrating characterization : SNOM measurements
Stretched fiber
65°
z=
1 beam excitation
�⃗�𝒊𝒏𝒄
�⃗�𝒎𝒐𝒅𝒆
�⃗�𝒊𝒏𝒄
�⃗�𝒎𝒐𝒅𝒆
(Geoffroy Scherrer, ICB, Dijon)
High Frequency Pattern from the Grating
18/27
Grid Shifting
Theoretical simulation
simulation
III. Structured Illumination Microscopy in
FluorescenceGrating characterization : Far field fluorescence measurements
2 beams excitation : Low frequency component of the intensity pattern
WF Fluorescence observation with ~homogeneous layer of fluorescent beads
19/27
III. Structured Illumination Microscopy in
Fluorescence Our manufactured gratings can produce a grid of light with
180 nm period (λ/3.5) (down to 147 nm, λ/4.3 with alternative design)
a high contrast
The possibility to shift its position
According to , a final resolution
of up to 87 nm could be reached at λ =633 nm!
However we need to know the illumination pattern for inversion procedure
=
20/27
24
“Blind” SIM Inversion
M 1= ( 𝐼1×𝜌 )∗ 𝑃𝑆𝐹M 2= ( 𝐼 2×𝜌 )∗𝑃𝑆𝐹
M n=( 𝐼𝑛×𝜌 )∗ 𝑃𝑆𝐹…
equations
unknowns :
1𝑁 ∑
𝑖=1
𝑁
𝐼𝑛=𝐼 0
+1
F ( 𝜌 , 𝐼 1 ,…, I n )=∑𝑖=1
𝑁
|𝑀 𝑖− [ ( 𝐼𝑖×𝜌 )∗ 𝑃𝑆𝐹 ]|2(Emeric Mudry & Kamal Belkebir)
Iterative optimization of estimates of and
through minimization of a cost function :
21/27
III. Structured Illumination Microscopy in Fluorescence
Observation of fluorescent beads (Ø 90nm) immersed in glycerin with
classical SIM
Experimental validation
WF image Our Result
Optimized « analytical » algorithm
Inversion by Pr. R. Heintzmann
Deconvolution of the WF image
22/27
III. Structured Illumination Microscopy in Fluorescence
|~𝐼|
𝐼
Simulation Measurement
Speckle illumination
1𝑁 ∑
𝑖=1
𝑁
𝐼𝑛 (𝑥 , 𝑦 )𝑁→∞→
𝐼 0
1. Contains every accessible frequencies
2. Known average illumination
3. Experiment far simpler than standard SIM
Speckle pattern is a perfect candidate for SIM with our ‘blind’ inversion
algorithm
23/27
III. Structured Illumination Microscopy in Fluorescence
object WF image
One measured image
N ≈ 80
Speckle illumination : simulations
Photon budget : average of 130
photon/pixel/imageReconstructed
=
=
Deconvolution
Deconvolution
=
speckles
speckles
24/27
III. Structured Illumination Microscopy in Fluorescence
Rabbit Jejunum slices (150nm thick) (Cendrine Nicoletti, ISM, Marseille)
TEM image of a similar sample WF image
Reconstructed image from 100 speckle illuminations
Deconvolution of WF image
Speckle illumination : experimental results
25/27
III. Structured Illumination Microscopy in Fluorescence
General Perspectives
I.Optical Diffraction Tomography :
Extend to 3D samples
Use other configuration (grating substrate, mirror substrate…)
II. Structured Illumination in Fluorescence Microscopy
1. Grating-assisted SIM :
Make super-resolved images of real samples : use a priori
information for inversion procedure
2. Speckle illumination :
Extend to 3D samples
27/27
SIM with unknown illumination patterns
Extension of SIM to the use of random speckle patterns
Not effective yet for grating-assisted SIM (inhomogeneous
average illumination)
26/27
III. Structured Illumination Microscopy in Fluorescence
Conclusion
Thanks…
Geoffroy Scherrer Anne Talneau Andrea Cattoni
The whole MOSAIC team for advices, seminars, discussions, equipment, facilities…
Eric Le Moal Guillaume Maire Emeric Mudry Kamal Belkebir Anne Sentenac
Thank you for your attention
33
II. Optical Diffraction Tomography
air
Si
300 nm
z
x
100 nm
110
NA = 0.7 (used up to 0.53 only for illumination)
→ Abbe limit : 500 nm (450nm for full NA)
AFM profile
Reconstructed map Reconstructed profileReconstructed profile
(linear inversion)
34
II. Optical Diffraction Tomography
Multiple scattering and resolution
= 28.8
(Germanium) (a) = 2 (b) = 7 (c) = 14
Modulation of for the object 2 :
Simulation of () =()
for a plane wave illumination (incidence 50°)
= 10-2
100nm
25nm
35
III. Structured Illumination Microscopy in Fluorescence
Grating assisted SIM : getting some images
Problem with inversion : Intensity pattern is not perfectly known
Speckle algorithm is not able to retrieve frequencies >
Add of a priori information (rough orientation and frequencies)