lightfields and lumigraphs
Post on 12-Jan-2016
30 Views
Preview:
DESCRIPTION
TRANSCRIPT
Lightfields and Lumigraphs
Vision for GraphicsCSE 590SS, Winter 2001
Richard Szeliski
(with lots of slides from Michael Cohen)
3/7/2001 Vision for Graphics 2
Modeling light
How do we generate new scenes and animations from existing ones?
Classic “3D Vision + Graphics”:• take (lots of) pictures• recover camera pose• build 3D model• extract texture maps / BRDFs• synthesize new views
3/7/2001 Vision for Graphics 3
Computer Graphics
Image
Output
ModelSyntheticCamera
3/7/2001 Vision for Graphics 4
Real Scene
Computer Vision
Real Cameras
Model
Output
3/7/2001 Vision for Graphics 5
Combined
Model Real Scene
Real Cameras
Image
Output
SyntheticCamera
3/7/2001 Vision for Graphics 6
But, vision technology falls short
ModelReal Scene
Real Cameras
Image
Output
SyntheticCamera
3/7/2001 Vision for Graphics 7
… and so does graphics.
ModelReal Scene
Real Cameras
Image
Output
SyntheticCamera
3/7/2001 Vision for Graphics 8
Image Based Rendering
Real Scene
Real Cameras-or-
Expensive Image Synthesis
Images+Model
Image
Output
SyntheticCamera
3/7/2001 Vision for Graphics 9
Ray
Constant radiance• time is fixed
5D• 3D position• 2D direction
3/7/2001 Vision for Graphics 10
All Rays
Plenoptic Function:• all possible images• too much stuff!
3/7/2001 Vision for Graphics 11
Line
Infinite line
4D• 2D direction• 2D position• non-dispersive medium
3/7/2001 Vision for Graphics 12
Ray
Discretize, then interpolate
Distance between 2 rays• Which is closer together?
3/7/2001 Vision for Graphics 13
Image
What is an image?
All rays through a point• Panorama?
3/7/2001 Vision for Graphics 14
Image
Image plane
2D• position
3/7/2001 Vision for Graphics 15
Light leaving towards “eye”
2D• just dual of image
Object
3/7/2001 Vision for Graphics 16
Object
All light leaving object
3/7/2001 Vision for Graphics 17
Object
4D• 2D position• 2D direction
3/7/2001 Vision for Graphics 18
Object
All images
3/7/2001 Vision for Graphics 19
Lumigraph / Lightfield
Outside convex space
4D
StuffEmpty
3/7/2001 Vision for Graphics 20
Lumigraph
Inside convex space
4D
EmptyStuff
3/7/2001 Vision for Graphics 21
Lumigraph
How to • organize• capture• render
3/7/2001 Vision for Graphics 22
Lumigraph - Organization
2D position
2D direction
s
3/7/2001 Vision for Graphics 23
Lumigraph - Organization
2D position
2D position
2 plane parameterization
su
3/7/2001 Vision for Graphics 24
Lumigraph - Organization
2D position
2D position
2 plane parameterization
us
t s,tu,v
v
s,t
u,v
3/7/2001 Vision for Graphics 25
Lumigraph - Organization
Hold s,t constant
Let u,v vary
An image
s,t u,v
3/7/2001 Vision for Graphics 26
Lumigraph - Organization
Discretization• higher res near object
– if diffuse– captures texture
• lower res away– captures directions
s,t u,v
3/7/2001 Vision for Graphics 27
Lumigraph - Capture
Idea 1• Move camera carefully over
s,t plane• Gantry
– see Lightfield paper
s,t u,v
3/7/2001 Vision for Graphics 28
Lumigraph - Capture
Idea 2• Move camera anywhere• Rebinning
– see Lumigraph paper
s,t u,v
3/7/2001 Vision for Graphics 29
Lumigraph - Rendering
For each output pixel• determine s,t,u,v• either
– find closest discrete RGB– interpolate near values s,t u,v
3/7/2001 Vision for Graphics 30
Lumigraph - Rendering
For each output pixel• determine s,t,u,v
• either• use closest discrete RGB• interpolate near values
s u
3/7/2001 Vision for Graphics 31
Lumigraph - Rendering
Nearest• closest s• closest u• draw it
Blend 16 nearest• quadrilinear interpolation
s u
3/7/2001 Vision for Graphics 32
Lumigraph - Rendering
Depth Correction• closest s• intersection with “object”• best u• closest u
s u
3/7/2001 Vision for Graphics 33
Lumigraph - Rendering
Depth Correction • quadralinear interpolation• new “closest”• like focus
s u
3/7/2001 Vision for Graphics 34
Lumigraph - Rendering
Fast s,t,u,v finding• scanline interpolate• texture mapping• shear warp
s u
3/7/2001 Vision for Graphics 35
Lumigraph - Ray Space
3/7/2001 Vision for Graphics 36
Lumigraph - Ray Space
Image effects:• parallax• occlusion• transparency• highlights
3/7/2001 Vision for Graphics 37
Lumigraph - Demo
Lumigraph• Lion, Fruit Bowl, Visible Woman, Path Tracing
3/7/2001 Vision for Graphics 38
Lightfield - DemoDigital Michelangelo Project
– Marc Levoy, Stanford University– Lightfield (“night”) assembled by Jon Shade
3/7/2001 Vision for Graphics 39
Surface Lightfields
Turn 4D parameterization around
Leverage coherence
3/7/2001 Vision for Graphics 40
Surface Lightfields
Wood et al, SIGGRAPH 2000
3/7/2001 Vision for Graphics 41
3D Representations
Image is 2D
Lumigraph is 4D
What happened to 3D?• 3D Lumigraph subset• Concentric mosaics
3/7/2001 Vision for Graphics 42
3D Lumigraph
One row of s,t plane• i.e., hold t constant
s,t u,v
3/7/2001 Vision for Graphics 43
3D Lumigraph
One row of s,t plane• i.e., hold t constant• thus s,u,v• a “row of images”
s
u,v
3/7/2001 Vision for Graphics 44
Concentric Mosaics
Replace “row” with “circle” of images
3/7/2001 Vision for Graphics 45
Concentric Mosaics
3/7/2001 Vision for Graphics 46
Concentric Mosaics
From above
3/7/2001 Vision for Graphics 47
Concentric Mosaics
Depth correction
3/7/2001 Vision for Graphics 48
Concentric Mosaics
Panorama
3/7/2001 Vision for Graphics 49
2.5D Representations
Image is 2DLumigraph is 4D3D
• 3D Lumigraph subset• Concentric mosaics
2.5D• Layered Depth Images• View Dependent Surfaces
3/7/2001 Vision for Graphics 50
Layered Depth Image
ImageDepthLayered
2.5 D ?
3/7/2001 Vision for Graphics 51
Layered Depth Image
Rendering from LDI
• Incremental in LDI X and Y • Guaranteed to be in back-to-front order
3/7/2001 Vision for Graphics 52
Layered Depth Image
Rendering from LDI
• Incremental in LDI X and Y • Guaranteed to be in back-to-front order
3/7/2001 Vision for Graphics 53
Layered Depth Image (LDI)
3/7/2001 Vision for Graphics 58
Summary
5D: Plenoptic Function (Ray)
4D: Lumigraph / Lightfield
3D: Lumigraph Subset
3D: Concentric Mosaics
2.5D: Layered Depth Image
2.5D: Image Based Models
2D: Image
top related