marc levoy light field = array of (virtual) cameras sub-aperture virtual camera = sub-aperture view

Marc Levoy

Light Field = Array of (virtual) Cameras

Sub-aperture

Virtual Camera = Sub-aperture View

MERL Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin

• Samples individual rays

• Predefined spectrum for lenses• Chromatic abberration

• High alignment precision

• Peripheral pixels wasted pixels

• Negligible Light Loss

• Samples coded combination of rays

• Supports any wavelength

• Reconfigurable f/#, Easier alignment

• No wastage• High resolution image for parts

of scene in focus

• 50 % Light Loss due to mask

Mask

Sensor

Microlens array

Sensor

Plenoptic Camera Heterodyne Camera

x

θ l(x,θ)

x

θ

l(x,θ)

x1

x2

θi θj

θj

x1x2

x1’ = x1 + θi*z

Shear of Light Field

θi

θj

x'1x1

Light Propagation (Defocus Blur)Light Propagation (Defocus Blur)

1-D FFT1-D FFT

2-D FFT

fx

fθL(fx,fθ)

Central Slice

x

θ l(x,θ)

x

θ l(x,θ)

Line Integral

Captured Photo

FFT of Captured

Photo

Space of LF representationsTime-frequency representationsPhase space representationsQuasi light field

WDF

TraditionTraditional light al light

fieldfield


fieldfield

Augmented LFObservable LF

RihaczekDistribution Function

incoherent

coherent

Other LF representatio

ns


ns


ns


ns

Quasi light fieldsthe utility of light fields, the versatility of Maxwell

We form coherent images by

formulating,

capturing,

and integrating

quasi light fields.

WDF


fieldfield


fieldfield


RihaczekDistributi

on Function

incoherent

coherent

Other LF representati

ons


ons


ons


ons

(i) Observable Light Field

• move aperture across plane

• look at directional spread

• continuous form of plenoptic camera

scene

apertureposition s

direction u

(ii) Augmented Light Field with LF Transformer

8

WDFWDF

Light Light FieldFieldLight Light FieldField

Augmented LF

Augmented LF

Interaction at the optical elementsInteraction at the optical elements

LF propagatio

n

(diffractive)

optical element

LF LF LF LF

LF propagatio

n

light field transformer

negative

radiance

Virtual light projector with real valued (possibly negative radiance) along a ray

9

real projector

real projector

first null (OPD = λ/2)

virtual light projector

(ii) ALF with LF Transformer

10

(iii) Rihaczek Distribution FunctionTradeoff between cross-interference terms and localization

y

u

0 m

3 m

0 m 3 m y0 m 3 m y0 m 3 m

u

y

(i) Spectrogram non-negative localization

(ii) Wigner localization cross terms

(iii) Rihaczek localization complex

Property of the Representation

Constant along rays

Non-negativity

Coherence WavelengthInterference Cross term

Traditional LF

always constant

always positive

only incoherent

zero no

Observable LF

nearly constant

always positive

any coherence

stateany yes

Augmented LF

only in the paraxial region

positive and negative

any any yes

WDFonly in the paraxial region

positive and negative

any any yes

Rihaczek DFno; linear

driftcomplex any any reduced

Benefits & Limitations of the Representation

Ability to propagate

Modeling wave optics

Simplicity of

computation

Adaptability to

current pipe line

Near Field Far Field

Traditional LF

x-shear novery

simplehigh no yes

Observable LF

not x-shear

yes modest low yes yes

Augmented LF

x-shear yes modest high no yes

WDF x-shear yes modest low yes yes

Rihaczek DF

x-shear yes

better than WDF,

not as simple as

LF

low no yes

MotivationMotivation

• What is the difference between a hologram What is the difference between a hologram and a lenticular screen?and a lenticular screen?

• How they capture ‘phase’ of a wavefront for How they capture ‘phase’ of a wavefront for telescope applications?telescope applications?

• What is ‘wavefront coding’ lens for extended What is ‘wavefront coding’ lens for extended depth of field imaging?depth of field imaging?

Application - Wavefront Coding

Dowski and Cathey 1995

same aberrant blur regardless of depth of focus

cubicphase plate

pointin scene

small changein blur shape

Can they be part of Computer Vision?Moving away from 2D images or 4D lightfields?

Wavefront coding: WLC mobile phone cameras

Holography:Reference targets

Rendering: New perspective

projection methods

Gaussian beam lasers:Modern active illumination

Rotating PSF: Depth from defocus

QuickTime™ and aBMP decompressor

are needed to see this picture.

Raskar, Camera Culture, MIT Media Lab

Computational Photography

1. Epsilon Photography– Low-level Vision: Pixels– Multiphotos by bracketing (HDR, panorama)– ‘Ultimate camera’

2. Coded Photography– Mid-Level Cues:

• Regions, Edges, Motion, Direct/global– Single/few snapshot

• Reversible encoding of data, Lightfield– Additional sensors/optics/illum– ‘Smart Camera’

3. Essence Photography– Not mimic human eye– Beyond single view/illum– ‘New artform’

http://computationalphotography.org

ResourcesResources

• WebsiteWebsite– http://scripts.mit.edu/~raskar/lightfields/http://scripts.mit.edu/~raskar/lightfields/– Or follow http://cvpr2009.org tutorial pagesOr follow http://cvpr2009.org tutorial pages

• Key new papersKey new papers• Wigner Distributions and How They Relate to the Light FieldWigner Distributions and How They Relate to the Light Field

Zhengyun Zhang Zhengyun Zhang and Marc Levoy, ICCP 2009 (best paper)and Marc Levoy, ICCP 2009 (best paper)

• Augmenting Light Field to Model Wave Optics Effects , Augmenting Light Field to Model Wave Optics Effects , Se Baek OhSe Baek Oh, Barbastathis, , Barbastathis, RaskarRaskar (in Preparation) (in Preparation)

• Quasi light fields: extending the light field to coherent radiation , Quasi light fields: extending the light field to coherent radiation , Anthony AccardiAnthony Accardi, Wornell (in Preparation), Wornell (in Preparation)

WDF


fieldfield


fieldfield


RihaczekDistribution Function

AcknowledgementsAcknowledgements

• DarthmuthDarthmuth– Marcus Testorf, Marcus Testorf,

• MITMIT– Ankit Mohan, Ahmed Kirmani, Jaewon KimAnkit Mohan, Ahmed Kirmani, Jaewon Kim– George BarbastathisGeorge Barbastathis

• StanfordStanford– Marc Levoy, Ren Ng, Andrew AdamsMarc Levoy, Ren Ng, Andrew Adams

• AdobeAdobe– Todor Georgiev, Todor Georgiev,

• MERLMERL– Ashok Veeraraghavan, Amit AgrawalAshok Veeraraghavan, Amit Agrawal

MIT Media LabMIT Media Lab

Camera CultureCamera Culture

Ramesh RaskarRamesh Raskar

MIT Media LabMIT Media Lab

http:// CameraCulture . info/http:// CameraCulture . info/

Light FieldsLight Fields______ Light FieldsLight Fields______

Light Fields in Ray and Wave Optics

Introduction to Light Fields: Ramesh Raskar

Wigner Distribution Function to explain Light Fields: Zhengyun Zhang

Augmenting LF to explain Wigner Distribution Function: Se Baek Oh

Q&A

Break

Light Fields with Coherent Light: Anthony Accardi

New Opportunities and Applications: Raskar and Oh

Q&A: All

marc levoy light field = array of (virtual) cameras sub-aperture virtual camera = sub-aperture view

Documents