EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 1

Computational Plenoptic ImagingGordon Wetzstein1 Ivo Ihrke2 Douglas Lanman3 Wolfgang Heidrich1

1University of British Columbia 2Saarland University 3MIT Media Lab

Eurographics 2011 – State of the Art Report

I. Introduction

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 2

The Mantis Shrimp

3 bands = trinocular visionwith each eye!

Up to 16 different photoreceptors:• 12 for color (4 UV, visible, IR)• 4 for polarization (linear & circular)

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 3

Cameras are modeled as one Human Eye

http://en.wikipedia.org/wiki/Bayer_filter

webvision.med.utah.edu

[Williams 91]

How would a camera look like if it was modeled as a mantis shrimp eye?

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 4

The Plenoptic Function

• Ray-based model for light• Includes variations in space, time,

wavelength, and directions( , , , , , , )x y zP P t V V V

[Adelson & Bergen 91]

( , , , )x yP P V V

xVyV

More common angular parameterization

• Light field is

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 5

Plenoptic Information Sampled

Allan Brooks, EE421, 2003

HVSConsumer

Video Camera

1

continuous, ~30fps(band-limited)

3, Bayer color filter array

1920x1080 (HD)

4 Plenoptic Parameters Sampled (no Vx,y,z)

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 6

Computational Photography and Plenoptic Imaging

• CPI: joint optical modulation & computational processing of plenoptic dimensions

• CP: computational optics, illumination, probes, reconstruction, image processing, sensors, etc.

[Ras

kar &

Tum

blin

]

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 7

Taxonomy and Overview

II. High Dynamic Range Imaging – 10 minutes, Wetzstein

III. Spectral Imaging – 10 minutes, Wetzstein

IV. Light Field Acquisition – 25 minutes, Lanman

V. Space & Focal Surfaces – 15 minutes, Wetzstein

VI. Temporal Multiplexing – 10 minutes, Wetzstein

VII. Further Light Properties – 10 minutes, Wetzstein

VIII. Discussion – 5 minutes, Wetzstein

EG 2011 | Computational Plenoptic Imaging STAR | I. Introduction 8

Next: High Dynamic Range Imaging