leandro
DESCRIPTION
TRANSCRIPT
![Page 1: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/1.jpg)
A Theory of Shape by Space Carving
Kiriakos N. Kutulakos Department of Computer Science and Department of Dermatology, University of
Rochester, Rochester, NY
Steven M. Seitz The Robotics Institute, Carnegie Mellon University, Pittsburgh
Published at theSpecial Issue on Genomic Signal Processing, 2000.
![Page 2: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/2.jpg)
Problem Statement
• Assuming:– Arbitrary (known) camera placement.– Arbitrary scene shape.– Arbitrary scene texture, color and background.– Lambertian.
• Input: N photographs of a scene
• Output: 3D structure of the scene
![Page 3: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/3.jpg)
Context
• Traditional Approach to 3D Reconstruction– Correspondence based
“Many scenes can account for the input images”
![Page 4: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/4.jpg)
Shape by Space Carving
• Scene or volume based
• No correspondences needed!
![Page 5: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/5.jpg)
Picture-Shape Constraint
![Page 6: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/6.jpg)
Photo Hull
• Definition: Maximal volume, V, reproducing all possible silhouettes of an object, S, from viewing region R.
R
S V
![Page 7: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/7.jpg)
Photo Hull in 3D
![Page 8: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/8.jpg)
Shape by Space Carving
• Given a volume containing the object, iteratively carves it out until it converges to the Photo Hull.
![Page 9: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/9.jpg)
Shape by Space Carving Analogy
![Page 10: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/10.jpg)
Shape by Space Carving
True Scene Reconstruction
Least-Commitment Recovery
![Page 11: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/11.jpg)
Carving outHow to decide if a certain voxel belongs to the Photo Hull?
C o n s i s t e
n c y
![Page 12: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/12.jpg)
ConsistencyRendering colored voxels from input viewpoints reproduces original images.
Point photo-consistent: A point (voxel) that can be assigned a color consistent with its projection in all visible cameras.
shape
![Page 13: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/13.jpg)
Consistency (cont.)
Shape Photo-consistent: A shape for which all surface points are photo-consistent.
Photo-Hull: Union of all photo-consistent subsets of initial volume.
VPhoto-hull of V
Photo-consistent subsets of V
![Page 14: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/14.jpg)
Proposed Algorithm
1. Initialize the volume containing the shape;
2. Analyze each visible voxel for point consistency (plane sweep strategy);
3. Repeat step 2 until no voxel is found inconsistent;
4. Merge photo consistent subsets.
![Page 15: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/15.jpg)
Plane Sweep Strategy
Sequence of plane sweeps along 6 principle directions.
Only cameras behind the sweep plane are used for consistency checking.
![Page 16: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/16.jpg)
Experimental Results
![Page 17: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/17.jpg)
Experimental Results
![Page 18: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/18.jpg)
Experimental Results
![Page 19: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/19.jpg)
Experimental Results
![Page 20: Leandro](https://reader035.vdocuments.us/reader035/viewer/2022062615/548ba1eab4795988248b47f9/html5/thumbnails/20.jpg)
Space Carving Discussion
• Least-commitment Reconstruction– No a-priori constraints– Flexible acquisition– Insensitivity to occlusion– Efficient– Scalable
• Setup– Camera calibration– Sensitivity to specularities and transparencies