[ppt]signal-theoretic representations of appearancecs283/fa10/lectures/283... · web viewadvanced...
TRANSCRIPT
![Page 1: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/1.jpg)
Advanced Computer Graphics Advanced Computer Graphics (Fall 2010)(Fall 2010)
CS 283, Lecture 16:
Image-Based Rendering and Light Fields
Ravi Ramamoorthi
http://inst.eecs.berkeley.edu/~cs283/fa10
![Page 2: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/2.jpg)
Next few slides courtesy Paul Debevec; SIGGRAPH 99 course notes
![Page 3: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/3.jpg)
![Page 4: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/4.jpg)
![Page 5: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/5.jpg)
![Page 6: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/6.jpg)
IBR: Pros and ConsIBR: Pros and Cons Advantages
Easy to capture images: photorealistic by definition Simple, universal representation Often bypass geometry estimation? Independent of scene complexity?
Disadvantages WYSIWYG but also WYSIAYG Explosion of data as flexibility increased Often discards intrinsic structure of model?
Today, IBR-type methods also often used in synthetic rendering (e.g. real-time rendering PRT) General concept of data-driven graphics, appearance Also, data-driven geometry, animation, simulation
![Page 7: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/7.jpg)
![Page 8: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/8.jpg)
IBR: A brief historyIBR: A brief history Texture maps, bump maps, environment maps [70s]
Poggio MIT 90s: Faces, image-based analysis/synthesis
Mid-Late 90s Chen and Williams 93, View Interpolation [Images+depth] Chen 95 Quicktime VR [Images from many viewpoints] McMillan and Bishop 95 Plenoptic Modeling [Images w disparity] Gortler et al, Levoy and Hanrahan 96 Light Fields [4D] Shade et al. 98 Layered Depth Images [2.5D] Debevec et al. 00 Reflectance Field [4D] Inverse rendering (Marschner,Sato,Yu,Boivin,…)
Today: IBR hasn’t replaced conventional rendering, but has brought sampled and data-driven representations to graphics
![Page 9: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/9.jpg)
![Page 10: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/10.jpg)
![Page 11: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/11.jpg)
OutlineOutline Overview of IBR
Basic approaches Image Warping
[2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
![Page 12: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/12.jpg)
Warping slides courtesy Leonard McMillan, SIGGRAPH 99 course notes
![Page 13: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/13.jpg)
![Page 14: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/14.jpg)
![Page 15: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/15.jpg)
![Page 16: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/16.jpg)
![Page 17: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/17.jpg)
![Page 18: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/18.jpg)
![Page 19: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/19.jpg)
![Page 20: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/20.jpg)
![Page 21: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/21.jpg)
![Page 22: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/22.jpg)
![Page 23: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/23.jpg)
OutlineOutline Overview of IBR
Basic approaches Image Warping
[2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
![Page 24: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/24.jpg)
![Page 25: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/25.jpg)
![Page 26: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/26.jpg)
![Page 27: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/27.jpg)
![Page 28: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/28.jpg)
![Page 29: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/29.jpg)
![Page 30: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/30.jpg)
![Page 31: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/31.jpg)
![Page 32: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/32.jpg)
![Page 33: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/33.jpg)
![Page 34: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/34.jpg)
OutlineOutline Overview of IBR
Basic approaches Image Warping
[2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
![Page 35: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/35.jpg)
Layered Depth Images [Shade 98]Layered Depth Images [Shade 98]
Geometry
Camera
Slide from Agrawala, Ramamoorthi, Heirich, Moll, SIGGRAPH 2000
![Page 36: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/36.jpg)
Layered Depth Images [Shade 98]Layered Depth Images [Shade 98]
LDI
![Page 37: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/37.jpg)
Layered Depth Images [Shade 98]Layered Depth Images [Shade 98]
LDI
(Depth, Color)
![Page 38: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/38.jpg)
![Page 39: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/39.jpg)
Miller 98, Nishino 99, Wood 00
Reflected light field (lumisphere) on surface
Explicit geometry as against light fields. Easier compress
Surface Light FieldsSurface Light Fields
![Page 40: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/40.jpg)
Acquiring Reflectance Field of Human Acquiring Reflectance Field of Human Face [Debevec et al. SIGGRAPH 00]Face [Debevec et al. SIGGRAPH 00]
Illuminate subject from many incident directions
![Page 41: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/41.jpg)
Example ImagesExample Images
![Page 42: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/42.jpg)
OutlineOutline Overview of IBR
Basic approaches Image Warping
[2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
Sampled data representations
![Page 43: [PPT]Signal-Theoretic Representations of Appearancecs283/fa10/lectures/283... · Web viewAdvanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light](https://reader035.vdocuments.us/reader035/viewer/2022070607/5aa502b37f8b9a2f048cea54/html5/thumbnails/43.jpg)
Conclusion (my views)Conclusion (my views) IBR initially spurred great excitement: revolutionize pipeline
But, IBR in pure form not really practical WYSIAYG Explosion as increase dimensions (8D transfer function) Good compression, flexibility needs at least implicit geometry/BRDF
Real future is sampled representations, data-driven method Acquire (synthetic or real) data Good representations for interpolation, fast rendering Much of visual appearance, graphics moving in this direction
Understand from Signal-Processing Viewpoint Sampling rates, reconstruction filters Factored representations, Fourier analysis
Light Fields fundamental in many ways, including imaging