spectral focal sweep (sfs): extended depth of field from chromatic aberrations iccp 2010 ollie...

Spectral Focal Sweep (SFS):

Extended Depth of Field from Chromatic Aberrations

ICCP 2010

Ollie Cossairt, Shree NayarColumbia University

Conventional Camera (F/4)

Spectral Focal Sweep: ICCP 2010

BLURRY

Conventional Camera (F/16)


NOISY

EDOF Camera (F/4)


CAPTURED

EDOF Camera (F/4)


DEBLURRED

Focus Sweep[Nagahara et al. ’08][Hausler ’72] Lens Focal PlaneSensor

Coded Aperture[Welford ’60] [Levin et al. ‘07][Veeraraghavan et al. ’07] [Zhou and Nayar ’09]

Related Work


Phase Plates[Dowski and Cathey ’95] [Chi and George ’01] [Levin et al. ’09]

Color[Guichard et al. ’09]

LensFocal Planes

RGB Sensor RedGreenBlue

Spectral Focal Sweep Idea


No moving parts or custom elements

PoorlyCorrected

Lens

+

B&WSensor

+

Focus Sweep

Lens

Focal Plane

Sensor

=

Broadband reflectance

spectra

Broadband reflectance spectra• Works exactly for black and white scenes• Many naturally occurring reflectance spectra

are also broadband [Parkkinen ‘89]

Poorly corrected lens• Simpler lens design optimization

Black & White Sensor• Can also approximate with luminance channel

from color sensor

Spectral Focal Sweep Idea


Conventional Focus Sweep PSF


v1u

u = object depth (fixed)v = sensor distance (varying)

LensEFL = f

FocalPlane

InstantaneousPSF Sensor

Scene Depth Range




LensEFL = f

FocalPlane


v2u

Scene Depth Range

d2



v3u

d3


LensEFL = f

FocalPlane


Scene Depth Range



Varying either object/sensor distance, or focal length will produce the same

PSF

Integrate over time to get final PSF

Circ/Top-hat function

Defocus disk diameter (A = aperture diameter)

Refractive Dispersion (prism)


Snell’s Law of refraction

Refraction depends

on wavelengt

h

Refractive Dispersion (lens)


uv

SensorPSFFocalPlanes

Scene Depth Range

For an uncorrected lens, Focal length depends on wavelength

R1 , R2 = lens radii of curvature

Spectral Focal Sweep PSF


Each wavelength produces a different defocus disk

A B&W Sensor integrates over wavelength, weighted by the spectral reflectance of the scene

For white objects, a SFS PSF is identical to a mechanical focal sweep PSF

SFS Lens Design


Maximum defocus diameter 100um at 400nm and 700nm

Design Criteria• Wavelength range 400-

700nm• 75mm EFL F/4• 1/3” Sensor, 10um pixel size

Optimization Criteria• Zemax Optical Design

Software• Maximize chromatic defocus

across wavelength range• Minimize spot size at 550nm• Fit to off-the-shelf lenses

Zemax SFS PSFs(white spectrum)

dep

th

field

SFS Lens Corrected lens


15x

Zemax SFS PSFs(white spectrum)

dep

th

Corrected lens


field

SFS Lens

1

15x

• Munsell database of 1250 measured reflectance spectra

• Zemax simulation of SFS PSF at 12 depth locations (1.5m-3m, 100um max defocus dia.)

• For each Munsell spectrum– Calculate average deblurring error when a

white spectrum is used for deblurring

Simulating Real Spectra


Average Deblurring Error


Input Image:

Munsell color PSFs:

White PSF at center depth:

PSF Distance Metric:

Average Deblurring Error:

[Zhou and Nayar ‘09]

Averaged over many images

Munsell Spectra PSFs


Corrected lens

dep

th

Most Munsell spectra show little PSF variation

1

15x

Average Deblurring Error(%95 of Munsell spectra)


Most Munsell PSFs introduce negligible deblurring artifacts

Doublet SFS Implementation


Edmund Optics #481841” Dia. 100mm FL Plano-convex asphere

Thorlabs #LE19291” Dia. 300mm FL positive meniscus

Zemax Design Implementation

Experimental Setup


B&W Examples(Basler 1/3” sensor)

Color Examples(Canon 450D sensor)

Corrected Lens SFS Doublet Lens

F/1.4

75

mm

EFL C

osm

icar

F/2.8

10

0m

m

EFL C

annon

F/4 S

FS 7

5m

mEFL D

ouble

tF/4

SFS

75

mm

EFL D

ouble

t

Measured SFS PSFs(white spectrum)


Measured SFS PSF approximately depth-invariant

SFS

lens

Cosm

ica

r

depth

Deblurring

Deblur with measured white PSF

B&W Images• Wiener deconvolution

Color Images• Transform from RGB to YUV space• Wiener deconvolution of Y image• Transform back to RGB space


EDOF Examples


F/4 Corrected Lens


8ms exposure

F/4 SFS Camera


8ms exposure

(Captured)

F/4 SFS Camera


8ms exposure

(Deblurred)

F/4 Corrected Lens


8ms exposure

Background Foreground

F/4 SFS Lens


8ms exposure

(Captured)

F/4 SFS Lens


(Deblurred)

8ms exposure

F/4 Corrected Lens


16ms exposure

Background

Foreground

F/4 SFS Lens


16ms exposure

(Captured)

F/4 SFS Lens


16ms exposure

(Deblurred)

F/4 Corrected Lens


16ms exposure

BackgroundForeground

F/4 SFS Lens


16ms exposure

(Captured)

F/4 SFS Lens


16ms exposure

(Deblurred)

F/4 Corrected Lens


16ms exposure

Background

Foreground

F/4 SFS Lens


16ms exposure

(Captured)

F/4 SFS Lens


16ms exposure

(Deblurred)

F/4 Corrected Lens


16ms exposure

Background

Foreground

F/4 SFS Lens


16ms exposure

(Captured)

F/4 SFS Lens


16ms exposure

(Deblurred)

Limitations


• Broadband assumption does not always hold (e.g. %5 of Munsell Spectra)– deblurring artifacts introduced

• Color deblurring method does not correct blur in chrominance channels

• Works exactly for B&W scenes• Works well for many naturally occurring

spectra• Results for color images “look” good• Easy to build EDOF camera

Conclusion


Broadbandspectra

Chromaticaberrations

B&Wsensor

Focussweep+ + ==

spectral focal sweep (sfs): extended depth of field from chromatic aberrations iccp 2010 ollie...

Documents

spectral focal sweep

sweep lens focal plane

focal length

spectral reflectance

focus sweep nagahara

wavelength slide

noisy slide

blurry slide