112/5/2015 12:54 graphics ii 91.547 image based rendering session 11
TRANSCRIPT
![Page 1: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/1.jpg)
104/21/23 07:42
Graphics II 91.547
Image Based Rendering
Session 11
![Page 2: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/2.jpg)
204/21/23 07:42
A Rendering Taxonomy
![Page 3: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/3.jpg)
304/21/23 07:42
The Plenoptic Function
p P V V V tx y z ( , , , , , , )
“… the pencil of rays visible from any point in space, at any time, and over any range of wavelengths”
Given a set of discrete samples (complete or incomplete) from the plenoptic function, the goal of image-based rendering is to generate a continuous representation of that function.
![Page 4: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/4.jpg)
404/21/23 07:42
Movie Map(Lippman 1980)
V V Vx y z, , , , Find Nearest
Sample MovieStorage
Image
![Page 5: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/5.jpg)
504/21/23 07:42
Taxonomy of “Virtual Camera” Movement(Chen et al. 1995)
0 Camera Rotation- Camera fixed at a particular location- Three rotational degrees of freedom
=Pitch (up and down)=Yaw (about vertical axis)=Roll (about camera axis)
0 Object Rotation- Camera always pointing at center of object- Viewpoint constrained to move over surface of sphere- Three angular degrees of freedom
0 Camera movement- Viewpoint unconstrained- Viewing direction unconstrained
![Page 6: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/6.jpg)
604/21/23 07:42
Environment MapsMap Geometries
Cube Sphere Cylinder
![Page 7: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/7.jpg)
704/21/23 07:42
Quick Time VRTM
(Chen 1995)
2500 Pixels
768 Pixels
2500 x 768 = 1.9 G Pixels x 3 B/pixel = 5.8 GB10:1 compression 500 MB/panorama
![Page 8: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/8.jpg)
804/21/23 07:42
Image Distortion fromCylindrical Environment Map
Projection Plane
CylindricalEnvironmentMap
Pre-warpedProjection ontoPlane
![Page 9: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/9.jpg)
904/21/23 07:42
Quick Time VRImage Warping for Correct Perspective View
![Page 10: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/10.jpg)
1004/21/23 07:42
Quick Time VRPanoramic Display Process
Compressed Tiles
Visible Tiles
CD ROM orHard Disk
Main MemoryCompressed
TilesCache
VisibleRegion
DisplayWindow
Warp
Decompress
OffscreenBuffer
![Page 11: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/11.jpg)
1104/21/23 07:42
Quick Time VRAccomplishing (Limited) Camera Motion
![Page 12: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/12.jpg)
1204/21/23 07:42
Accomplishing Camera MotionGreene&Kass (1993 Apple Tech Doc.)
0 Regular 3-D lattice of cubic environment maps0 Each environment map is a z-buffered rendering from a
discrete viewpoint0 Image from a new viewpoint is generated by re-sampling the
environment map0 Re-sampling involves rendering the pixels in the environment
maps as 3-D polygons from the new viewpoint0 Rendering time proportional to the environment map
resolution but independent of scene complexity0 Not suitable for real-time walkthrough performance on typical
desktop computers (especially in 1993!)
![Page 13: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/13.jpg)
1304/21/23 07:42
Alternative approach:Work entirely in Image Space
0 Sequence of images from closely spaced viewpoints is highly coherent
0 Depends upon the ability to establish a pixel-by-pixel correspondence between adjacent images- Can be computed if range data and camera parameters
are known (true for rendered images)- For natural images, there are several techniques including
manual user intervention0 Pairwise correspondence between two images can be stored
as a pair of morph maps- Bi-directional maps required because of possible many to
one and one to many pixel correspondences0 Can be represented by graph data structure where nodes are
images and arcs are bi-directional morph maps
![Page 14: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/14.jpg)
1404/21/23 07:42
N-Dimensional Graph Data Structure
Image Image Image Image Image
Image Image Image Image Image
Image Image Image Image Image
Bi-directional Morph Maps
![Page 15: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/15.jpg)
1504/21/23 07:42
Simple View Interpolation
Reference Image 1 Reference Image 2
CorrespondingPixels
Morph maps
![Page 16: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/16.jpg)
1604/21/23 07:42
Image Overlap orImage Folding
P1
P2
ReferenceView
InterpolatedView
![Page 17: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/17.jpg)
1704/21/23 07:42
Image Holes orImage Stretching
InterpolatedView
ReferenceView
P1P2
![Page 18: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/18.jpg)
1804/21/23 07:42
Example of Hole Region
Viewpoint 1 Viewpoint 2
![Page 19: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/19.jpg)
1904/21/23 07:42
Example of Hole RegionMinimizing by Closely Spaced Viewpoints
Viewpoint 1 Viewpoint 2
![Page 20: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/20.jpg)
2004/21/23 07:42
Source Image Viewed from Camera Moved to the Right
Ref. View 1
Ref. View 2
![Page 21: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/21.jpg)
2104/21/23 07:42
Offset Vectors for Camera Motion Morph Map
![Page 22: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/22.jpg)
2204/21/23 07:42
Locus of Morph Map for MotionParallel to Image Plane and Floor
![Page 23: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/23.jpg)
2304/21/23 07:42
Distortion of Intermediate Images withLinear Warp
Linear path of one feature
![Page 24: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/24.jpg)
2404/21/23 07:42
Morphing Parallel Views
Reference image
Reference image
Interpolated image
V0
V1
Ci
V V V
p p p
p V
i 0 1
i 0 1
i i
( )
( )
1
1
s s
s s
P
![Page 25: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/25.jpg)
2504/21/23 07:42
View Interpolation:The Algorithm
I0 I1
I0 ' I1 'Is '
Is
1 1
2 2
3
![Page 26: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/26.jpg)
2604/21/23 07:42
Example 1 of calculated intermediate images
Reference Image 1 Reference Image 2
Intermediate Views
![Page 27: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/27.jpg)
2704/21/23 07:42
Example 2 of calculated intermediate images
Reference Image 1 Reference Image 2Interpolated Image
![Page 28: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/28.jpg)
2804/21/23 07:42
Multiple-Center-of-Projection Images(Rademacher&Bishop 1998)
0 Information from a set of viewpoints stored in a single image0 Features
- Greater connectivity information compared with collections of standard images
- Greater flexibility in the acquisition of image-based datasets, e.g. sampling different portions of the scene at different resolutions
![Page 29: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/29.jpg)
2904/21/23 07:42
Multiple-Center-of-Projection ImagesDefinition
0 A multiple-center-of-projection image consists of a two-dimensional image and a parameterized set of cameras meeting the following conditions:- The cameras must lie on either a continuous curve or a
continuous surface- Each pixel is acquired by a single camera- Viewing rays vary continuously across neighboring pixels- Two neighboring pixels must either correspond to the
same camera or to neighboring cameras- Each pixel contains range information
![Page 30: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/30.jpg)
3004/21/23 07:42
MCOP Image
![Page 31: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/31.jpg)
3104/21/23 07:42
Strip Camera used for Capture of Real MCOP Images
![Page 32: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/32.jpg)
3204/21/23 07:42
Camera Path in Capturing MCOP Image of Castle
![Page 33: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/33.jpg)
3304/21/23 07:42
Image Plane for Camera Motion
![Page 34: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/34.jpg)
3404/21/23 07:42
Resulting 1000 x 500 MCOP Image
![Page 35: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/35.jpg)
3504/21/23 07:42
Reprojection
Camera model, stored per column:
Center of projection
Vector from to image plane origin
Horizontal axis of viewing plane
Vertical axis of viewing plane
Disparity = distance from to the image planedivided by distance from to the pixel’s world space point
Ci
CiOi
U i
Vi
ijCi
Ci
x
y
z
i
j
C
C
Cij
ix ix ix
iy iy iy
iz iz iz
ix
iy
iz
1
1
U V O
U V O
U V O
ReprojectionFormula:
![Page 36: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/36.jpg)
3604/21/23 07:42
View of Castle Reconstructed from MCOP Image
![Page 37: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/37.jpg)
3704/21/23 07:42
AnotherView of Castle Reconstructed from MCOP Image
![Page 38: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/38.jpg)
3804/21/23 07:42
Lumigraphs
0 Lumigraph = a representation of the light resulting from a scene
0 Limited data representation of the plenoptic function0 Generated from multiple images and camera “poses”0 Rendering: Image = Lumigraph + Camera Model0 Special case of 4D Light Field (Levoy, Hanrahan)
![Page 39: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/39.jpg)
3904/21/23 07:42
What is a Lumigraph?
For all points on the surrounding surface,For all directions,
The color intensity of the ray.
Assumption: We are outside a convex hull containingthe objects
![Page 40: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/40.jpg)
4004/21/23 07:42
Parameterization of the Lumigraph
Images from Steven Gortler, SIGGRAPH 1999
![Page 41: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/41.jpg)
4104/21/23 07:42
Building the Lumigraph
![Page 42: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/42.jpg)
4204/21/23 07:42
Approximating the LumigraphWith Discrete Samples
![Page 43: 112/5/2015 12:54 Graphics II 91.547 Image Based Rendering Session 11](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649f565503460f94c7b2e8/html5/thumbnails/43.jpg)
4304/21/23 07:42
Views of a Light Field (Lumigraph)
Levoy & Hanrahan, Light Field Rendering, Computer Graphics