analysis of the folding processweb.mit.edu/2.717/www/2.717-wk13-a.pdf · 05/09/05 – wk13-b-13...

MIT 2.71705/09/05 – wk13-b-1

3D Optics

MIT 2.71705/09/05 – wk13-b-2

Outline

• Introduction• Point-spread function of 3D pupils• Phase matching (aka Bragg matching)• Defocus response of 3D pupils• Optical slicing

– scanning– rainbow– multiplex

• Computational approaches– maximum likelihood (Viterbi)– tomography (Radon transform)

MIT 2.71705/09/05 – wk13-b-3

Outline




MIT 2.71705/09/05 – wk13-b-4

Imaging a 2½D object

2f 2fx ′′ x′x

1f 1f

DepthDepthOf Of

FocusFocus

DefocusDefocus

zworsensworsens

awayawayfromfromfocalfocalplaneplane

22½½DDobjectobject

MIT 2.71705/09/05 – wk13-b-5

Volume holographic aperture filter (VH lens)

2f 2fx ′′ x′x

1f 1f

The MIT solution The MIT solution ((SinhaSinha et al, 2002et al, 2002--2004)2004)

add volume hologram in the Fourier plane

z

in focus portionin focus portiononlyonly

is visible!add volume hologram in the Fourier plane

is visible!

MIT 2.71705/09/05 – wk13-b-6

Shape recovery (“profilometry”) with VH lenses

Line scan method: 2D scanningLine scan method: 2D scanninglongitudinal + one lateral dimensionlongitudinal + one lateral dimension

raw imageson CCD camera

2½D shape(“profile”)

in digital form

slitslit--likelikefield of viewfield of view

scanning recoversscanning recoversthe full fieldthe full field

MIT 2.71705/09/05 – wk13-b-7

Outline




MIT 2.71705/09/05 – wk13-b-8

The “4F system” (telescope) : a general model for imaging

2f 2fx ′′ x′x

1f 1f

z

imageimageplane

objectobjectplane

FourierFourierplane planeplane plane

MIT 2.71705/09/05 – wk13-b-9

Fourier filtering

2f 2fx ′′ x′x

1f 1f

z

imageimageplane

objectobjectplane

FourierFourierfilter planeplane filter

PSF)(filter (in)(out) 2D∗=

MIT 2.71705/09/05 – wk13-b-10

3D aperture filtering

2f 2fx ′′ x′x

1f 1f

z

3D3D FourierFourierfilterfilter

surf3D PSF)(filter (in)(out) ∗=

MIT 2.71705/09/05 – wk13-b-11

What is a 3D pupil?

z ′′

y ′′

x ′′ Special case:•• modulation on a modulation on a spatial carrierspatial carrier ::

( ) ( )rr rK ′′=′′ ′′⋅ εε ~e ci

““volume hologramvolume hologram””•• 3D spatial heterodyning3D spatial heterodyning•• optically recorded oroptically recorded orfabricated (e.g., twofabricated (e.g., two--photon)photon)

More special case:•• modulation modulation sampledsampled in in zz direction :

( )r ′′ε

3D modulation of3D modulation ofrefractive index

direction :refractive index““multimulti--layered diffractive elementlayered diffractive element””•• 3D spatial heterodyning3D spatial heterodyning•• fabricated lithographicallyfabricated lithographically(layer(layer--byby--layer or layer or origamiorigamiTMTM))

MIT 2.71705/09/05 – wk13-b-12

How volume holographic 3D pupils are madeAlternative fabrication methods

exist, e.g. by Ondax, Inc. (proprietary)

Optical recordingOptical recording

two mutually coherenttwo mutually coherentlight beams (e.g. from thelight beams (e.g. from the

same laser) interferesame laser) interfereinside 3D holographic inside 3D holographic

materialmaterial

focal focal pointpoint

depth depth reference reference

planeplane

objectiveobjectivelenslens

signal beamsignal beamtypically a typically a ““plane waveplane wave””

(pencil of light)(pencil of light)

reference beamreference beam

has has focal pointfocal point, which, whichdetermines determines ““depth depth reference planereference plane””

MIT 2.71705/09/05 – wk13-b-13

Volume holography background

• Invented by van Heerden in 1963

• Popular in the 70s (due to improved understanding of materials such as photorefractives and photochromics) and 90s (due to advances in optoelectronics, e.g. CCD cameras, spatial light modulators, and optomechanics)

• Main applications: holographic data storage (promise of storage density much higher than optical disks due to the 3D nature of volume holograms), optical interconnects for high-speed computing

• Use of volume holograms in capacity of “unusual” lenses first reported by Barbastathis, Balberg & Brady in 1999.

MIT 2.71705/09/05 – wk13-b-14

Optical filtering using a 3D pupil?

( )yxp ,

( )yxq ′′,

( )zyx ′′′′′′ ,,ε

( )yxp ,Given input field

( )yxq ′′,seek output field

MIT 2.71705/09/05 – wk13-b-15

Optical filtering using a 2D pupil (shift-invariant)

( )yxp ,

( )yxq ′′,

( )yx ′′′′ ,ε

( )yx ′′′′ ,ε

( ) ( )vfufvuH 11 ,, λλε=

( ) ( ){ }⎟⎟⎠

⎞⎜⎜⎝

⎛ ′′ℑ=′′22

,,,fy

fxvuHyxh

λλ

Pupil function (2D)

Amplitude Transfer Function

Point Spread Function

MIT 2.71705/09/05 – wk13-b-16

Optical filtering using a 3D pupil /1

( )yxp ,

( )yxq ′′,

( )zyxP ′′′′′′ ,,

Field propagating in the vicinity of the Fourier plane:

( ) ( ) ( )⎭⎬⎫

⎩⎨⎧ ′′+

−′′+′′

⎭⎬⎫

⎩⎨⎧=′′′′′′ ∫∫ 2

1

22

1

2exp,dd2exp,,f

zyxif

yyxxiyxypxzizyxPλ

πλ

πλ

π

(equivalent to Fourier transform of p(x,y) plus defocus term)

MIT 2.71705/09/05 – wk13-b-17


( )yxp ,

( )yxq ′′,

( )zyx ′′′′′′ ,,ε

Effect of the 3D pupil: generates secondary coherent radiation of the form

( ) ( ) ( )zyxPzyxzyxg ′′′′′′×′′′′′′=′′′′′′ ,,,,,, ε

assuming 1st-order Born approximation (weak hologram) is valid:• no probe depletion• no rediffraction

MIT 2.71705/09/05 – wk13-b-18


( )yxp ,

( )yxq ′′,

( )zyxg ′′′′′′ ,,

Field at the output plane: Fourier transform of the 3D scattered field

( ) ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

′′=′′ 2

2

22

22 211,,,

fyx

fy

fxGyxq

λλλ

(but output is measured on a plane, hence the Fourier transform is sliced sliced )

MIT 2.71705/09/05 – wk13-b-19

Shape of the slice

u

v

w

MIT 2.71705/09/05 – wk13-b-20

Summary: 2D pupil vsvs 3D pupil / filtering view-point

( )yxp ,

( )yxq ′′,

( )zyx ′′′′′′ ,,ε

2D2D 3D3D( ) ( )

( )⎭⎬⎫

⎩⎨⎧ ′′+−×

×⎭⎬⎫

⎩⎨⎧ ′′+′′

⎭⎬⎫

⎩⎨⎧=′′′′′′ ∫∫

21

22

1

exp

2exp,dd2exp,,

fzyxi

fyyxxiyxypxzizyxP

λπ

λπ

λπ( ) ( )

⎭⎬⎫

⎩⎨⎧ ′′+′′

⎭⎬⎫

⎩⎨⎧=′′′′′′ ∫∫

1

2exp,dd2exp,,f

yyxxiyxypxzizyxPλ

πλ

π

( ) ( ) ( )zyxPzyxzyxg ′′′′′′×′′′′′′=′′′′′′ ,,,,,, ε( ) ( ) ( )yxPyxyxg ′′′′×′′′′=′′′′ ,,, ε

( ) ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

′′=′′ 2

2

22

22 211,,,

fyx

fy

fxGyxq

λλλ( ) ⎥

⎦

⎤⎢⎣

⎡ ′′=′′

22

,,fy

fxGyxq

λλ

MIT 2.71705/09/05 – wk13-b-21

Summary: 2D pupil vsvs 3D pupil / ATF & PSF view-point

( )yxp ,

( )yxq ′′,

( )zyx ′′′′′′ ,,ε

2D2D 3D3D

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′′

2121

1,1,;,fy

fx

fx

fxyxyxh

λλε

Point-Spread Function(shift invariant)


( ) { }

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+′−

+

⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+

ℑ=′′

22

22

21

22

2121

221

,1,1

,;,

fyx

fyx

fy

fx

fx

fx

yxyxh

λ

λλε

Point-Spread Function(shift variant)

( ) ( )vfufvuH 11 ,, λλε=

MIT 2.71705/09/05 – wk13-b-22

Summary: 2D pupil vsvs 3D pupil / ATF & PSF view-point

( )yxp ,

( )yxq ′′,

( )zyx ′′′′′′ ,,ε

2D2D 3D3D( ) ( )vfufvuH 11 ,, λλε=

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′′

2121

1,1,;,fy

fx

fx

fxyxyxh

λλε

Point-Spread Function(shift invariant)


Point-Spread Function(shift variant)

( ) ( ) zzf

yxizvfufyxvuH ′′⎟⎟⎠

⎞⎜⎜⎝

⎛′′

′+′′′=′′ ∫ dexp,,,;, 22

22

11 λπλλε

( ) { } ⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ ′+⎟⎟

⎠

⎞⎜⎜⎝

⎛ ′+ℑ=′′

2121

1,1,;,fy

fx

fx

fxyxyxh

λλε

Amplitude Transfer Function(position dependent)

MIT 2.71705/09/05 – wk13-b-23

Shift-variant imaging using 3D pupils

Bragg mismatched:Bragg mismatched:slit effectslit effect

x

x′

@Bragg match:@Bragg match:apodizerapodizer offoff--axisaxis

PlanePlane--toto--plane wave hologram, offplane wave hologram, off--axis reference onaxis reference on--axis signalaxis signal

MIT 2.71705/09/05 – wk13-b-24

Outline




MIT 2.71705/09/05 – wk13-b-25

Explanation of shift-variance: phase matching

Recording Bragg-matched readout

Diffracted beams from elemental thin slicesare all in phase in phase → strong diffraction

MIT 2.71705/09/05 – wk13-b-26


Recording Bragg-mismatched readout

Diffracted beams from elemental thin slicesare out of phase out of phase → no diffraction

MIT 2.71705/09/05 – wk13-b-27



Rθ∆

θ

Λ

L

θ

θλθsin2R LL

=Λ

=∆Angle Bragg selectivity

MIT 2.71705/09/05 – wk13-b-28

Explanation of shift-variance: k-sphere


Rk

Sk

Kλπ2

radius

=k

pk

K

RS kkK −=

MIT 2.71705/09/05 – wk13-b-29



Rk

Sk

Kλπ2

radius

=k

pk

K

dk

Kkk += RdRS kkK −=

MIT 2.71705/09/05 – wk13-b-30



Rk

Sk

Kλπ2

radius

=k

pk

K

dk

zkd∆

x̂

ẑ

( )[ ] zxxKkk ˆˆ ˆ dRd zk+⋅+=RS kkK −=k=d k

⎟⎠⎞

⎜⎝⎛ ∆= LkII zd

2matched-Braggdiffracted 2

1sinc

MIT 2.71705/09/05 – wk13-b-31

Outline




MIT 2.71705/09/05 – wk13-b-32

Defocus with 3D pupils

x′ x′x′

y′

λ640 =z λ2000 =z λ4000 =z

λλ =probe


MIT 2.71705/09/05 – wk13-b-33

Dispersion with 3D pupils

x′ x′x′

y′

λ640 =z λ2000 =z λ4000 =z

λλ 2.1probe =


MIT 2.71705/09/05 – wk13-b-34

Outline




MIT 2.71705/09/05 – wk13-b-35

How volume holographic lenses operate (line-scan)

Digital readoutDigital readout

prepre--fabricated hologramfabricated hologramin the capacity ofin the capacity of““just another lensjust another lens””in the optical trainin the optical train

line illuminationline illumination

FourierFourierlenslens

signal beamsignal beamnot presentnot present

detector planedetector plane(CCD camera)(CCD camera)

diffracteddiffracted beam isbeam is““continuationcontinuation”” of signalof signal

beam; inbeam; in--focus light onlyfocus light only

reference beamreference beamreplaced byreplaced by


inin--focus focus light onlylight only

NOTE: the imaging processNOTE: the imaging processdoes not require the recordingdoes not require the recording

of a hologramof a hologram (unlike traditional(unlike traditionalholographic imaging) holographic imaging)

MIT 2.71705/09/05 – wk13-b-36














of a hologramof a hologram (unlike traditional(unlike traditionalholographic imaging) holographic imaging)

laterallateralscanningscanning

MIT 2.71705/09/05 – wk13-b-37














of a hologramof a hologram (unlike traditional(unlike traditional““holographic imagingholographic imaging””) )

longitudinallongitudinalscanningscanning

MIT 2.71705/09/05 – wk13-b-38





in digital form

slitslit--likelikefield of viewfield of view

scanning recoversscanning recoversthe full fieldthe full field

MIT 2.71705/09/05 – wk13-b-39





in digital form

Working distance d= 50cmLongitudinal resolution ∆z FWHM< 1mm

MIT 2.71705/09/05 – wk13-b-40

Increasing the working distanceSinha et al, Appl. Opt. 43:1533, 2004

• Simple objective lens • Telephoto objective optics

L

2r

d

Objective optics

2s

2

s

2

a r

r (EFL)

θλ

θλ

LdG

LGz PRPRFWHM ==∆

a

(EFL)(EFL)L

z

d

a

2s

2

a aθλ

LdGz PRFWHM =∆

Improvement = a/rImprovement = a/r

z

VH lensVH lens

MIT 2.71705/09/05 – wk13-b-41

Increasing the working distance

−3 −2 −1 0 1 2 30.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Axial displacement (mm)

Diff

ract

ed In

tens

ity (

a.u.

)

Telephoto PR VHI, d=500 mmStand alone PR VHI, d=50 mm

Longitudinal PointLongitudinal Point--Spread Function (PSF)Spread Function (PSF)

Simple objectiveWorking distance

d=50.2 mm.

Telephoto objective optics Working distance

d=500 mm; r=2 mm.

MIT 2.71705/09/05 – wk13-b-42

Increasing the resolutionSinha et al, Optics Express 11:3202, 2003

• Normal illumination • Inclined illumination

z

d Telephotooptics

d

Φ

x∆OLDz∆

z∆NEWz∆

z

Digitalcamera

object

Fourier lens

VH lens

MIT 2.71705/09/05 – wk13-b-43

Increasing the resolution

NA=0.024; θs=12o; a=8 mm; d=460 mm; L=2 mm and Φ= 0o / 30o.

Longitudinal PSFLongitudinal PSF

MIT 2.71705/09/05 – wk13-b-44

Image of drug capsule using line scanCapsule provided by Pfizer

obtained using rotary stageobtained using rotary stage

MIT 2.71705/09/05 – wk13-b-45

Working distance and resolution (line scan method)Microturbine provided by

Chee Wei Wong, Alan Epstein, MIT Telephoto + tilted illuminationTelephoto + tilted illumination

z=0 µm z=50 µm

z=200 µm z=250 µm

originalobject Object Distance = 46 cmObject Distance = 46 cm

Resolution accomplished Resolution accomplished ≤≤ 100 100 µµmmraw imagesraw images

profileprofilereconstructionreconstruction

MIT 2.71705/09/05 – wk13-b-46

NA=0.024; θs=12o; a=8 mm; d=460 mm; L=2 mm and Φ=30o.

Image of MEMS device acquired using line scanat long working distance Nanogate provided by

James White, Alex Slocum, MIT La

tera

l dim

ensi

on (m

m)

∆z=0 µm ∆z=50 µm

∆z=100 µm ∆z=150 µm

raw imagesraw images

reconstructionreconstruction

Original object

MIT 2.71705/09/05 – wk13-b-47

Tunable grating fabricated by Wei-Chuan Shih, MIT

Image of MEMS device acquired using line scanat long working distance

object under microscopeobject under microscope

∆z=0 µm

rawrawimagesimages

∆z=24 µm

reconstructionreconstruction

Microscope Objective NA = 0.65,

Working distance (d) = 2 cm

MIT 2.71705/09/05 – wk13-b-48

Outline




MIT 2.71705/09/05 – wk13-b-49

Rainbow VH imaging

grating

iris white light source

objectplanecollimator

lens

cylindricallens

θs

camera

MIT 2.71705/09/05 – wk13-b-50

Reduce scanning: #2 / rainbow method22½½D object is illuminated by D object is illuminated by rainbowrainbow

(analyzed white light) (analyzed white light) ……… each matched depth is obtained simultaneously on the camera plaeach matched depth is obtained simultaneously on the camera planene

(no slit effect (no slit effect →→ no lateral scanning necessary)no lateral scanning necessary)while preserving the outwhile preserving the out--ofof--focus light rejection propertyfocus light rejection property

Longitudinal scanning still necessary

…

Longitudinal scanning still necessary

slice #1slice #1 slice #2slice #2

digital reconstruction of object profiledigital reconstruction of object profile

MIT 2.71705/09/05 – wk13-b-51

Imaging using the rainbow VH method

MIT 2.71705/09/05 – wk13-b-52

Outline




MIT 2.71705/09/05 – wk13-b-53

3D→2D spatial mapping in the multiplex method

C

F

I

J

K

L

Q

R

S

B

E

H

M

P

A

D

G

N

O

T

U

V

X

W

Y

Z

Θ

camera

A B C

D E F

G H I

C

F

I

CA B J

K

L

JMNO P Q

Q

R

S

VHI system

object

MIT 2.71705/09/05 – wk13-b-54

Recording the multiplex VH lens

holographicholographicmaterial

Reference beam:Reference beam:spherical wave @ spherical wave @ zz11

materialxx

zz

Signal beam:Signal beam:plane wave @ plane wave @ θθ11

MIT 2.71705/09/05 – wk13-b-55


50µm

zz

xxholographicholographic

materialmaterial



MIT 2.71705/09/05 – wk13-b-56


50µm

zz

xxholographicholographic

materialmaterial



MIT 2.71705/09/05 – wk13-b-57

Diffraction from the multiplex VH lens

50µm

zz

xxVHVHlenslens

Fourier Fourier lenslens

Probe beam: three point Probe beam: three point sourcssourcs@ @ zz11, , zz22, , zz3 3 (mutually (mutually incoherentincoherent))

Reconstruction: three plane waves Reconstruction: three plane waves @ @ θθ11, , θθ22, , θθ33

CameraCamera detects:detects:three point imagesthree point images

MIT 2.71705/09/05 – wk13-b-58

Reduce scanning: #1 / multiplex methodthree three ““slicesslices”” throughthrough

fluorescent (3D) object, stacked fluorescent (3D) object, stacked along along longitudinallongitudinal direction direction ……

…… are viewed are viewed simultaneouslysimultaneously and and sideside--byby--sidesideon the digital cameraon the digital camera

3D object:fluorescent beads

in water

implemented using three volume holographic lensesimplemented using three volume holographic lensessharing the sharing the samesame volume holographic materialvolume holographic material

(multiplexed)(multiplexed)

MIT 2.71705/09/05 – wk13-b-59

Outline




MIT 2.71705/09/05 – wk13-b-60

Inter-Symbol Interference (ISI) and the Viterbi algorithm

Contributions to central pixel on raw imageContributions to central pixel on raw imagefrom voxels in object spacefrom voxels in object space

everything else:everything else:ISIISI

WANTEDWANTED

MIT 2.71705/09/05 – wk13-b-61

Reducing complexity

2D Viterbi algorithm with ISI size of m*n typically has statesnm2 ⋅

Surface constraint: only one “1” on each row, number of states reduce to mn

1)/2(mn +Use FDVA (Feedback Decision Viterbi Algorithm), number of states reduce to

Scanning direction constraint eliminates 4/5 possible states transition paths

MIT 2.71705/09/05 – wk13-b-62

Multiplex + Viterbi reconstruction

zz11

zz22

11

22

•• 2 VHI lenses 2 VHI lenses (multiplexed inside the same crystal)(multiplexed inside the same crystal)•• 8 target reconstruction depths8 target reconstruction depths

MIT 2.71705/09/05 – wk13-b-63


Layer #3Layer #3

Layer #6Layer #6

MIT 2.71705/09/05 – wk13-b-64


Layer #3Layer #3

Layer #6Layer #6

•• Layers #3 and #6 simultaneously in focus Layers #3 and #6 simultaneously in focus (multiplexed, imaged on different parts of the camera die)(multiplexed, imaged on different parts of the camera die)

•• Remaining layers: reconstructed digitallyRemaining layers: reconstructed digitally

MIT 2.71705/09/05 – wk13-b-65

Outline




MIT 2.71705/09/05 – wk13-b-66

True 3D imaging: the Radon transform setup

MIT 2.71705/09/05 – wk13-b-67

True 3D imaging: gummy bear via the Radon transform

360 angular positions360 angular positions

MIT 2.71705/09/05 – wk13-b-68

M. C. Escher“Reptiles”

3D OpticsOutlineOutlineImaging a 2½D objectVolume holographic aperture filter (VH lens)OutlineThe “4F system” (telescope) : a general model for imagingFourier filtering3D aperture filteringWhat is a 3D pupil?How volume holographic 3D pupils are madeVolume holography backgroundOptical filtering using a 3D pupil?Optical filtering using a 2D pupil (shift-invariant)Optical filtering using a 3D pupil /1Optical filtering using a 3D pupil /2Optical filtering using a 3D pupil /3Shape of the sliceSummary: 2D pupil vs 3D pupil / filtering view-pointSummary: 2D pupil vs 3D pupil / ATF & PSF view-pointSummary: 2D pupil vs 3D pupil / ATF & PSF view-pointShift-variant imaging using 3D pupilsOutlineExplanation of shift-variance: phase matchingExplanation of shift-variance: phase matchingExplanation of shift-variance: phase matchingExplanation of shift-variance: k-sphereExplanation of shift-variance: k-sphereExplanation of shift-variance: k-sphereOutlineDefocus with 3D pupilsDispersion with 3D pupilsOutlineHow volume holographic lenses operate (line-scan)How volume holographic lenses operate (line-scan)How volume holographic lenses operate (line-scan)Increasing the working distanceIncreasing the working distanceIncreasing the resolutionIncreasing the resolutionImage of drug capsule using line scanWorking distance and resolution (line scan method)OutlineRainbow VH imagingReduce scanning: #2 / rainbow methodImaging using the rainbow VH methodOutline3D2D spatial mapping in the multiplex methodReduce scanning: #1 / multiplex methodOutlineMultiplex + Viterbi reconstructionMultiplex + Viterbi reconstructionMultiplex + Viterbi reconstructionOutlineTrue 3D imaging: the Radon transform setupTrue 3D imaging: gummy bear via the Radon transform

analysis of the folding processweb.mit.edu/2.717/www/2.717-wk13-a.pdf · 05/09/05 – wk13-b-13...

Documents