example based 3d shape completion mark pauly 1,2, niloy j. mitra 1, joachim giesen 2, markus gross...
Post on 15-Jan-2016
221 views
TRANSCRIPT
Example Based 3D Shape Completion
Mark Pauly1,2, Niloy J. Mitra1, Joachim Giesen2, Markus Gross2, Leonidas J. Guibas1
1 Stanford University 2 ETH, Zurich
Example Based 3D Shape Completion
Shape Completion
• Incomplete raw scans
• Imperfect range scanned data• Complex objects with occluded
regions
• Misalignment of multiple-views depth image scans
• Specular highlights
Ill-posed problem
Use prior knowledge !!
Example Based 3D Shape Completion
Diffusion (Davis, Marschner, Garr, Levoy, 2002)
Example Based 3D Shape Completion
Context Based Solution (Sharf, Alexa, Cohen-or, 2004)
Example Based 3D Shape Completion
Template based Solution (Allen, Curless, Popovic, 2003; Kraevoy and Sheffer, 2005)
Example Based 3D Shape Completion
Our Solution
• Use 3D model database to provide geometric priors for shape completion
• Apply non-rigid transforms on the models• More deformation less likely completion
• Consistently combine geometric information from multiple context models
• Final result comes with confidence values
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Data Classification
Local analysis
• quality of fit
• uniformity of sample distribution
Scored Point Cloud
• confidence value assigned to each point
High
Low
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Database Retrieval
1.93 1.71 1.46 1.27 1.0
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Non-rigid Alignment
Similar to the approaches proposed by:• Allen, Curless and Popovic, 2003.
• Sumner and Popovic, 2004.
Example Based 3D Shape Completion
Non-rigid Alignment
Deformation Model
• Piecewise linear.Each vertex of the mesh assigned an independent displacement vector.
Optimize for smallest Shape Matching Penalty
• Distortion Measure
• Geometric Error
Feature Correspondence
Derived in the continuous setting to allow consistent comparison between different context models.
Example Based 3D Shape Completion
Warped Models
High
Low
Context Model Warped Model Matching Penalty
Example Based 3D Shape Completion
Non-rigid Transform
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Initial Segmentation
Input Data Warped Context Model
Example Based 3D Shape Completion
Patch Growing
Initial Segmentation Final Segmentation
Example Based 3D Shape Completion
Video
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Blending Final Segments
Example Based 3D Shape Completion
Result
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Shape Completion Pipeline
Example Based 3D Shape Completion
Segmentation
Giraffe Example
Context Models
Deformed Models
Example Based 3D Shape Completion
Final Model
Giraffe Example
Context Models
Deformed Models
Example Based 3D Shape Completion
Evaluation
Input Data Context Model Final Model Evaluation
Example Based 3D Shape Completion
Enriching the Database
Example Based 3D Shape Completion
Additional Constraints
Physical Model Acquired Data Context Model
No Constraints Symmetry Constraints
Example Based 3D Shape Completion
Galleria dell’ Accademia
Example Based 3D Shape Completion
Galleria dell’ Accademia
Example Based 3D Shape Completion
Galleria dell’ Accademia
Example Based 3D Shape Completion
Galleria dell’ Accademia
Example Based 3D Shape Completion
Future Directions
• Improve the retrieval stage.
• Automatic feature point detection.
• Use of more semantic information.
• Apply learning techniques to shape completion.
• Completion of additional attributes like surface texture, color.
Example Based 3D Shape Completion
Acknowledgements
• NSF grants CARGO-0138456, ITR-0205671, FRG-0454543, ARO grant DAAD19-03-1-033.
• Stanford Graduate fellowship.
• Mario Botsch, David Koller, Doo Young Kwon, Marc Levoy, Filip Sadlo, Vin da Silva, and Bob Sumner.
Example Based 3D Shape Completion
Questions
Example Based 3D Shape Completion
Example Based 3D Shape Completion
Correspondence
High
Input Data Warped Model Low
Invalid
Valid
Example Based 3D Shape Completion
Video
Example Based 3D Shape Completion
• Example-based image completion[Drori et al. 2003; Jia and tang 2003; Sun et al. 2003]
• Texture synthesis[Efros and Leung 1999; Efros and Freeman 2001;
Wei and Levoy 2000; Ying et al. 2001]
• Smooth surface completion
[Curless and Levoy 1996; Davis et al. 2002;
Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003]
• Model-based surface reconstruction [Savchenko et al. 2002]
• Curve analogies [Hertzmann et al. 2002]
Drori et al. 03
Liepa 03
Wei and Levoy 00
Hertzmann et al. 02
Related Works