Download - Example Based 3D Shape Completion
![Page 1: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/1.jpg)
Example Based 3D Shape Completion
Mark Pauly1,2, Niloy J. Mitra1, Joachim Giesen2, Markus Gross2, Leonidas J. Guibas1
1 Stanford University 2 ETH, Zurich
![Page 2: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/2.jpg)
Example Based 3D Shape Completion
Shape Completion
• Incomplete raw scans
• Imperfect range scanned data• Complex objects with occluded
regions
• Misalignment of multiple-views depth image scans
• Specular highlights
Ill-posed problem
Use prior knowledge !!
![Page 3: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/3.jpg)
Example Based 3D Shape Completion
Diffusion (Davis, Marschner, Garr, Levoy, 2002)
![Page 4: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/4.jpg)
Example Based 3D Shape Completion
Context Based Solution (Sharf, Alexa, Cohen-or, 2004)
![Page 5: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/5.jpg)
Example Based 3D Shape Completion
Template based Solution (Allen, Curless, Popovic, 2003; Kraevoy and Sheffer, 2005)
![Page 6: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/6.jpg)
Example Based 3D Shape Completion
Our Solution
• Use 3D model database to provide geometric priors for shape completion
• Apply non-rigid transforms on the models• More deformation less likely completion
• Consistently combine geometric information from multiple context models
• Final result comes with confidence values
![Page 7: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/7.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 8: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/8.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 9: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/9.jpg)
Example Based 3D Shape Completion
Data Classification
Local analysis
• quality of fit
• uniformity of sample distribution
Scored Point Cloud
• confidence value assigned to each point
High
Low
![Page 10: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/10.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 11: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/11.jpg)
Example Based 3D Shape Completion
Database Retrieval
1.93 1.71 1.46 1.27 1.0
![Page 12: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/12.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 13: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/13.jpg)
Example Based 3D Shape Completion
Non-rigid Alignment
Similar to the approaches proposed by:• Allen, Curless and Popovic, 2003.
• Sumner and Popovic, 2004.
![Page 14: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/14.jpg)
Example Based 3D Shape Completion
Non-rigid Alignment
Deformation Model
• Piecewise linear.Each vertex of the mesh assigned an independent displacement vector.
Optimize for smallest Shape Matching Penalty
• Distortion Measure
• Geometric Error
Feature Correspondence
Derived in the continuous setting to allow consistent comparison between different context models.
![Page 15: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/15.jpg)
Example Based 3D Shape Completion
Warped Models
High
Low
Context Model Warped Model Matching Penalty
![Page 16: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/16.jpg)
Example Based 3D Shape Completion
Non-rigid Transform
![Page 17: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/17.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 18: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/18.jpg)
Example Based 3D Shape Completion
Initial Segmentation
Input Data Warped Context Model
![Page 19: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/19.jpg)
Example Based 3D Shape Completion
Patch Growing
Initial Segmentation Final Segmentation
![Page 20: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/20.jpg)
Example Based 3D Shape Completion
Video
![Page 21: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/21.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 22: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/22.jpg)
Example Based 3D Shape Completion
Blending Final Segments
![Page 23: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/23.jpg)
Example Based 3D Shape Completion
Result
![Page 24: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/24.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 25: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/25.jpg)
Example Based 3D Shape Completion
Shape Completion Pipeline
![Page 26: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/26.jpg)
Example Based 3D Shape Completion
Segmentation
Giraffe Example
Context Models
Deformed Models
![Page 27: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/27.jpg)
Example Based 3D Shape Completion
Final Model
Giraffe Example
Context Models
Deformed Models
![Page 28: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/28.jpg)
Example Based 3D Shape Completion
Evaluation
Input Data Context Model Final Model Evaluation
![Page 29: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/29.jpg)
Example Based 3D Shape Completion
Enriching the Database
![Page 30: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/30.jpg)
Example Based 3D Shape Completion
Additional Constraints
Physical Model Acquired Data Context Model
No Constraints Symmetry Constraints
![Page 31: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/31.jpg)
Example Based 3D Shape Completion
Galleria dell’ Accademia
![Page 32: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/32.jpg)
Example Based 3D Shape Completion
Galleria dell’ Accademia
![Page 33: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/33.jpg)
Example Based 3D Shape Completion
Galleria dell’ Accademia
![Page 34: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/34.jpg)
Example Based 3D Shape Completion
Galleria dell’ Accademia
![Page 35: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/35.jpg)
Example Based 3D Shape Completion
Future Directions
• Improve the retrieval stage.
• Automatic feature point detection.
• Use of more semantic information.
• Apply learning techniques to shape completion.
• Completion of additional attributes like surface texture, color.
![Page 36: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/36.jpg)
Example Based 3D Shape Completion
Acknowledgements
• NSF grants CARGO-0138456, ITR-0205671, FRG-0454543, ARO grant DAAD19-03-1-033.
• Stanford Graduate fellowship.
• Mario Botsch, David Koller, Doo Young Kwon, Marc Levoy, Filip Sadlo, Vin da Silva, and Bob Sumner.
![Page 37: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/37.jpg)
Example Based 3D Shape Completion
Questions
![Page 38: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/38.jpg)
Example Based 3D Shape Completion
![Page 39: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/39.jpg)
Example Based 3D Shape Completion
Correspondence
High
Input Data Warped Model Low
Invalid
Valid
![Page 40: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/40.jpg)
Example Based 3D Shape Completion
Video
![Page 41: Example Based 3D Shape Completion](https://reader036.vdocuments.us/reader036/viewer/2022062314/56813b8c550346895da4bd4f/html5/thumbnails/41.jpg)
Example Based 3D Shape Completion
• Example-based image completion[Drori et al. 2003; Jia and tang 2003; Sun et al. 2003]
• Texture synthesis[Efros and Leung 1999; Efros and Freeman 2001;
Wei and Levoy 2000; Ying et al. 2001]
• Smooth surface completion
[Curless and Levoy 1996; Davis et al. 2002;
Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003]
• Model-based surface reconstruction [Savchenko et al. 2002]
• Curve analogies [Hertzmann et al. 2002]
Drori et al. 03
Liepa 03
Wei and Levoy 00
Hertzmann et al. 02
Related Works