© 2010 adobe systems incorporated. all rights reserved. t. georgiev, adobe systems a. lumsdaine,...

29
© 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

Upload: juliana-mccormick

Post on 17-Dec-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University

The Multi-Focus Plenoptic Camera

Page 2: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Capturing Memorable Moments

2

Page 3: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

What’s Missing with this Picture?

3

Page 4: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

The Lippmann Sensor

Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality.

Gabriel Lippmann, 1908.

Page 5: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 5

Pixels and Cores: What do we do with that power?

Moore’s Law: Megapixels keep growing

7.2 MP = 8 by 10 at 300dpi

Available on cell phones

60MP sensors available now

Larger available soon (can a use be found?)

Use pixels to capture richer information about a scene

Computationally process captured data

GPU power also riding Moore’s Law curve

Page 6: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

How do we take advantage of these trends?

Plenoptic camera – High resolution; computationally intensive rendering

The pixels are samples of the radiance in the 4D ray space

Optical elements (lenses, space) transform the ray space

We take a picture by rendering (computationally)

We adjust the picture by transforming the ray space (computationally)

6

Page 7: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

The Full Variety: Focusing

Page 8: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Focusing

8

Page 9: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Focusing

9

Page 10: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

The Full Variety: Different Views

10

Page 11: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Different Views

11

Page 12: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Different Views

12

Page 13: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Radiance (Plenoptic Function, Lightfield)

Instead of integrating rays coming from all directions into a given point (traditional camera), capture the rays individually (capture the radiance) using the Lippmann Sensor

Record all the information about the scene

13

Page 14: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 14

Plenoptic lens array outside the camera

Page 15: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 15

Microlenses inside the camera

Page 16: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Problem - Solution

Problem: Limited depth of field

Solution: Microlenses of different focal lengths

(This is similar to HDR where we use microlenses of different apertures)

16

Page 17: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Multifocus

Lytro and Raytrix already selling plenoptic cameras

Variable focal lengths: Better focusing. Our work on that started 2009.

Raytrix actually use microlenses of different focal lengths in their camera.

Lytro use single focal length (Plenoptic 1.0)

17

Page 18: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 18

Variable Focal Lengths Microlenses

Page 19: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 19

Variable Focal Lengths Microlenses

Page 20: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 20

Variable Focal Lengths Microlenses

Page 21: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 21

Variable Focal Lengths Microlenses

Page 22: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 22

Variable Focal Lengths Microlenses

Page 23: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 23

Variable Focal Lengths Microlenses: Compare Focus Quality

Page 24: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 24

Phase Space analysis

Page 25: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 25

Phase Space analysis

Page 26: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 26

Phase Space analysis: Miocrolenses of different focal lengths

Page 27: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved. 27

To Find Out More

This work was first presented at Asilomar2010/

Page 28: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.

Live Demonstrations

28

Page 29: © 2010 Adobe Systems Incorporated. All Rights Reserved. T. Georgiev, Adobe Systems A. Lumsdaine, Indiana University The Multi-Focus Plenoptic Camera

© 2010 Adobe Systems Incorporated. All Rights Reserved.