Spectral Imaging

Ivo IhrkeSaarland University/MPI Informatik

• spectrometer

The spectral data cube

disadvantage:• dispersion relation is nonlinearadvantage:• light efficient

Principle of Operation - Dispersion

Diffraction Grating

– At center, no diffraction

– For higher orders, diffraction is

taking place

Diffraction Order Percentage of transmitted Light

0 25%

1 20.26%

2 10.13%

3 2.25%

4 0 %

remainder 9.72%

disadvantage:• low light efficiencyadvantage:• linear relation pixel pos. <-> wavelength

– Diffraction-based example ->

– Spectrometer calibration (all types)

1. mapping

pixel <–> wavelength

2. relative intensity of wavelengths

Diffraction-Based Systems

The spectral data cube

• E.g. in satellite imaging

– Pushbroom scanning

• Spatial Scanning (2D sensor)

Spatial Scanning• Generalized Mosaics [Schechner & Nayar]

• linear filter

• each pixel column filtered differently

• rotational motion & registration to assemble image stack

• Spectral scanning

The spectral data cube

Frequency Scanning

• Michelson Interferometer with moving mirror - Fourier Transform Imaging Spectroscopy (FTIS)

Imaging Spectrometers

The quest for the instantaneous spectral data cube

(4D Imaging)

Multiplexing - Image slicers

[WiFeS – Wide Field Spectrometer]

[SPIFFI - SPectrometer for Infrared Faint Field Imaging]

Multiplexing - Image slicers

Fiber Optical cables (OKSI)

2D -> 1D reformatting

300 “pixels”

Multiplexing: Prism-Mask Based System

[Du’09]

Computational Imaging Spectrometers

The quest for the instantaneous spectral data cube

• Original method [Descour’95]

• Image a full diffraction pattern

• Perform “CT”

CTIS – Computed Tomography Imaging Spectrometry

diffraction patternspectral image

CTIS in graphics

• HDR imaging

for CTIS(not snapshot due to HDR exposure stack)

[Habel’12]

CTIS in graphics

• Spatial resolution 124x124, 54 bands

[Habel’12]

CASSI – Coded Aperture Snapshot Spectral

Imaging

CASSI – Coded Aperture Snapshot Spectral

Imaging

Implementation with prisms

• Resolution: spatial ~200 x 200 pixels

spectral ~30 bands

CASSI – Coded Aperture Snapshot Spectral Imaging

projection reconstruction (stack) spectra

• Transfer low-res spectra to high res RGB image [Rump’10,Cao’11]

Spectral Transfer

Applications

Applications

• automatic white balancingSpatially uniform illumination

[Cao11]Spatially varying illumination

raw from RGB tungsten WB `greyworld WB spectral WB spectra

Applications

• improved tracking

• real and fake skin detection[Cao11]

RGB – tracking lost

spectral – tracking OK

Applications• analyze / restore paintings

[Calit]

Applications• Satellite-Based Remote Sensing

[DigitalGlobe’10]

vegetation mapping urban land use pollution monitoring

• Multispectral at Siggraph’12

– Kim, Harvey, Kittle, Rushmeier, Dorsey, O’Prum, and Brady “3D Imaging Spectroscopy for Measuring Hyperspectral Patterns on Solid Objects”, Monday, 3:45 - 5:35 – “Appearance”

– Hosek and Wilkie, “An Analytic Model for Full Spectral Sky-Dome Radiance”, Wednesday 3:45-5:35 pm – “Physics and Mathematics

for Light”