Randomized Algorithms for Comparing and Understanding 3D Geometry
Niloy J. MitraDepartment of Electrical Engineering
Advisor: Leonidas J. Guibas Associate Advisor: Marc Levoy
Comparing and Understanding 3D Geometry
Need for Digital 3D Models
games
movies
architectural design
medicine
Google earth
manufacturing
Comparing and Understanding 3D Geometry
Creating Geometry: 3D Modelers
Comparing and Understanding 3D Geometry
Capturing Geometry: 3D Scanners
laser scanner 3D geometry entertainment
Comparing and Understanding 3D Geometry
Shape Acquisition
Comparing and Understanding 3D Geometry
Shape Acquisition
Comparing and Understanding 3D Geometry
Shape Acquisition
• Partial similarity between shapes
Comparing and Understanding 3D Geometry
Shape Acquisition
• Partial similarity between shapes
Comparing and Understanding 3D Geometry
Shape Acquisition
• Partial similarity between shapes• Efficient shape retrieval for partial queries
Comparing and Understanding 3D Geometry
Geometry Processing
[Funkhouser et al. `05]
[Katz and Tal `04]
[Gelfand et al. `05]
[Sharf et al.`04]
Comparing and Understanding 3D Geometry
Model Organization and Retrieval
[Kazdhan et al. `04]
Comparing and Understanding 3D Geometry
Partial Shape Similarity
Self-similarity of an object symmetry
partial similarity
Comparing and Understanding 3D Geometry
Total vs Partial Matching• Total matching is easy
PCA (Principal Component Analysis) Axes
Comparing and Understanding 3D Geometry
Total vs Partial Matching• Partial matching is difficultWhich region matches which other region(s)?Space of rigid transforms rotation + translation
Brute force approach not feasible
Instead of exhaustive searching, use local geometry to guide where to search
Easy to verify a transform
Comparing and Understanding 3D Geometry
ContributionsAlgorithms to:• Identify and extract similar (symmetric) patches of different
size• Estimate partial shape similarity between models without
explicitly aligning them
Properties:• Scalable and parallel• Theoretical error bounds• Output sensitive depends on complexity of solution and
not on the complexity of model(s)
Comparing and Understanding 3D Geometry
Outline• Introduction• Related Work• Symmetry Detection• Probabilistic Fingerprints• Conclusion and Future Work
Comparing and Understanding 3D Geometry
Related Work: Global Alignment
[Huber and Hebert`01] [Li and Guskov `05] [Gelfand et al. `05]
• Feature based alignment• Combinatorial search, need multiple objects together
Comparing and Understanding 3D Geometry
Related Work: Desc. Based Align.
[Kazhan et al. `03][Osada et al. `02]
• Feature based alignment• Combinatorial search, need multiple objects together
• Descriptor based alignment • Fails for partial similarity
Comparing and Understanding 3D Geometry
Related Work: Geometric Hashing
[Gal and Cohen-Or`05][Wolfson and Rigoutsos`97]
• Feature based alignment• Combinatorial search, need multiple objects together
• Descriptor based alignment • Fails for partial similarity
• Geometric hashing• Tradeoff memory for time
Comparing and Understanding 3D Geometry
Related Work: Symmetry Detection
[Podolak et al. `06]
brute force : O(n6)
[Loy and Eklundh `06]
Hough transform on feature points
[Thrun and Wegbreit `05]
Shape from symmetry
Comparing and Understanding 3D Geometry
Outline• Introduction• Related Work• Symmetry Detection• Probabilistic Fingerprints• Conclusion and Future Work
Comparing and Understanding 3D Geometry
Symmetry in Nature“Symmetry is a complexity-reducing concept [...]; seek it everywhere.”
- Alan J. Perlis
"Females of several species, including […] humans, prefer symmetrical males." - Chris Evan
Comparing and Understanding 3D Geometry
Partial Symmetry Detection (SIGGRAPH 2006)
GivenObject/shape (represented as point cloud, mesh, ... )
Identify and extract similar (symmetric) patches of different size across different resolutions
Goal
Comparing and Understanding 3D Geometry
Partial Symmetry
Transform Types:
• Reflection
• Rotation + Translation
• Uniform Scaling
Comparing and Understanding 3D Geometry
Reflective Symmetry
Comparing and Understanding 3D Geometry
Reflective Symmetry : A Pair Votes
Comparing and Understanding 3D Geometry
Reflective Symmetry : Voting Continues
Comparing and Understanding 3D Geometry
Reflective Symmetry : Voting Continues
Comparing and Understanding 3D Geometry
Reflective Symmetry : Largest Cluster
• Height of cluster ! size of patch
• Spread of cluster ! approximation level
Comparing and Understanding 3D Geometry
Pipeline
Comparing and Understanding 3D Geometry
Pruning: Local Signatures• Local signature invariant under transforms• Signatures disagree points don’t correspond
Use (1, 2) for curvature based pruning
Comparing and Understanding 3D Geometry
Reflection: Normal-based Pruning
Comparing and Understanding 3D Geometry
Point Pair Pruning
all pairs curvature based curvature + normal based
Comparing and Understanding 3D Geometry
Transformations• Reflection point-pairs• Rigid transform more information
Robust estimation of principal curvature frames [Cohen-Steiner et al. `03]
Comparing and Understanding 3D Geometry
Mean-Shift ClusteringKernel:• Type radially symmetric hat• Radius
Comparing and Understanding 3D Geometry
Verification• Clustering gives a good guess• Verify build symmetric patches• Locally refine solution using ICP algorithm [Besl and McKay `92]
Comparing and Understanding 3D Geometry
Random Sampling• Height of clusters related to symmetric region size• Random samples !
larger regions likely to be detected earlier• Output sensitive
Comparing and Understanding 3D Geometry
AnalysisAssumptions:• Smooth surface -sampled• No noise
Relates number(n’) of random samples to: • Size of symmetric patch (p)• Confidence (1-)• Sampling spacing, kernel radius, continuity of signature
Tools for Analysis:• Signature continuity• Chernoff bound
Comparing and Understanding 3D Geometry
Compression: Chambord
Comparing and Understanding 3D Geometry
Compression: Chambord
Comparing and Understanding 3D Geometry
Compression: Chambord
Comparing and Understanding 3D Geometry
Approximate Symmetry: Dragon
correction fieldUNITS: fraction of bounding box diagonal
detected symmetries
Comparing and Understanding 3D Geometry
Limitations
• Cannot differentiate between small sized symmetries and comparable noise
[Castro et al. `06]
Comparing and Understanding 3D Geometry
Articulated Motion: Horses
registration symmetry detection between two objects
Comparing and Understanding 3D Geometry
Outline• Introduction• Related Work• Symmetry Detection• Probabilistic Fingerprints• Conclusion and Future Work
Comparing and Understanding 3D Geometry
Partial Shape Similarity (SGP 2006)
Are two shapes similar in parts?
Efficient tests require compact signatures• database query • online setting
Comparing and Understanding 3D Geometry
partial similarity
Probabilistic Fingerprints
probabilisticfingerprint
probabilisticfingerprint
comparecompact
independent
Comparing and Understanding 3D Geometry
Insight Partial matching ! difficult problemTotal matching ! easy problem
Reduce partial matching ! many small total matching problems
Results in few false positives ! quick to verify and discard
Comparing and Understanding 3D Geometry
Input Shapes
Comparing and Understanding 3D Geometry
Sample Points
Comparing and Understanding 3D Geometry
Shingles: Overlapping Patches
Comparing and Understanding 3D Geometry
Shingles: Overlapping Patches
Comparing and Understanding 3D Geometry
Bag of Patches: Ordering Discarded
Comparing and Understanding 3D Geometry
Pipeline
Comparing and Understanding 3D Geometry
Pipeline: Uniform Sampling• Uniform spacing use [Turk`92]• Sample spacing
Comparing and Understanding 3D Geometry
Pipeline: Shingle Generation• Shingles: overlapping unordered patches• Shingle radius: •
Comparing and Understanding 3D Geometry
Pipeline: Signatures• Stable signatures• Invariant to rigid transforms
• Spin-images• Shape
unordered high-dimensional point set with rigid transform factored out
Comparing and Understanding 3D Geometry
• Similarity/resemblance• Defined wrt. signatures
• Compare two bag of points in high-dim space• No alignment required• Brute force evaluation impractical
Pipeline: Resemblance
Comparing and Understanding 3D Geometry
How to Compare Point Sets
• Compare two point sets no need to align• Don’t have red and blue points together
Comparing and Understanding 3D Geometry
Reduce Sample Size
• Randomly sample red points• Randomly sample blue points• still need to solve for correspondence
independently
Comparing and Understanding 3D Geometry
Min-hashing [Broder`97]
Each of m random ‘experts’• Has an ordering of space-boxes• Selects the point that lies in lowest ordered box
2 3
Comparing and Understanding 3D Geometry
Min-hashing [Broder`97]
Each of m random ‘experts’• Has an ordering of space-box• Selects the point that lies in lowest ordered box
2 3
1 1
Comparing and Understanding 3D Geometry
Pipeline: Min-hashing [Broder`97]
Feature selection by random experts• ‘Features’ only useful for correspondence
• Need not have any visual importance• Reduces set comparison to element-wise comparison
Comparing and Understanding 3D Geometry
Applications: Adaptive Features
Comparing and Understanding 3D Geometry
Applications: Adaptive Features
merged scan
Comparing and Understanding 3D Geometry
Applications: Shape Space
• Partial similarity• Articulated motion
Comparing and Understanding 3D Geometry
Applications: Database Retrieval
Comparing and Understanding 3D Geometry
• Pre-processing time in seconds:
• Query time → 15 msec/model • Fingerprint size → 10kb
model # vertices uniform sampling
spin image Rabin hash min-hash
skull 54k 0.8 7.5 0.05 4.5
Caesar 65k 1.4 7.3 0.08 10.3
bunny 121k 1.8 13.8 0.04 2.9
horse 8k 0.7 5.7 0.05 7.3
Statistics
Comparing and Understanding 3D Geometry
Limitations• Fails if resemblance is small• How to handle uniform scaling?• Stability of spin-images
Comparing and Understanding 3D Geometry
Outline• Introduction• Related Work• Symmetry Detection• Probabilistic Fingerprints• Conclusion and Future Work
Comparing and Understanding 3D Geometry
Conclusion• Simple probabilistic framework
• Local evidence → global reasoning• Geometric information for guidance• Complexity of problem, not complexity of model
• Symmetry information → High level model understanding
• Possible to compare two shapes using very compact fingerprints without aligning the models
• Local reasoning → possible false positives → verification
Comparing and Understanding 3D Geometry
Future Works• Continuous scanning, assembly, hole filing• Extension to deformable, time varying models• Understanding of high dimensional data• Online transmission, authentication, and security
Data courtesy: Prof. B. Chen
Comparing and Understanding 3D Geometry
CollaboratorsNatasha Gelfand, Stanford
Joachim Giesen, MPII Saarbrücken
Markus Gross, ETH Zürich
Leonidas Guibas, Stanford
An Nguyen, Stanford
Mark Pauly, ETH Zürich
Helmut Pottmann, Vienna University of Technology
Comparing and Understanding 3D Geometry
AcknowledgementsGunnar CarlsonLeonidas GuibasJean-Claude LatombeMarc LevoyMark Pauly
Mike Cammarano, Billy Chen, Milton Chen , Kayvon Fatahalian, Gaurav Garg, Eran Guendelman, Daniel Horn, Mike Houston, Jeff Klingner, David Koller, Manu Kumar, Ren Ng, John Owens, Doantam Phan, Marie Ringel, Pradeep Sen, Eino-Ville Talvala, Vaibhav Vaish, Ron Yeh
Emilio Antunez, Qing Fang, Natasha Gelfand, Olaf A. Hall-Holt, Kyle Heath, Rachel Kolodny, Nikola Milosavljevic, An Nguyen, Steve Oudot, Maksim Ovsjanikov, Daniel Russel, Aneesh Sharma, Jaewon Shin, Primoz Skraba, Michael Wand, Yusu Wang, Danny Yang, Afra Zomorodian
Pierre Alliez, Mario Botsch, Pat Hanrahan, Michael Hoffer, Rajiv Motwani, Richard Keiser, Doo Young Kwon, Bob Sumner, Martin Wicke
Manuela Cavegn, Heather Gentner, John Gerth, Ada Glucksman, Hoa Nguyen
Joseph W. and Hon Mai Goodman Stanford Graduate FellowshipCargo, Darpa, ITR, NIH, and NSF funding
Comparing and Understanding 3D Geometry
Acknowledgements
Stanford Cricket Club
Stanford Outing Club
Stanford Climbing Wall
Stanford Alpine Club
Kamran Ahsan, Abhishek Bapna, Akanksha Bapna, Indrahit Bhattacharya, Gaurav Chandra, Anirban Dasgupta, Anupam Datta, Amal Ekbal, Gaurav Garg, Mahesh Hardikar, Sara Kalantari, Uma Kelkar, Neha Kumar, Subhasish Mitra, Shoubhik Mukhopadhyay, Subha Nabar, Anindya Pathak, Inam Ur-Rehman, Mitul, Saha, Debasis Sahoo, Sriram Sankaranarayanan, Arjun Singh, Padma Sundaram, Vaibhav Vaish, gsb-SIE fellows, climbing buddies, cricket club folks, … many I missed
Comparing and Understanding 3D Geometry
Acknowledgements
ParentsBrother
Devasree
Comparing and Understanding 3D Geometry
Acknowledgements
Comparing and Understanding 3D Geometry
thank you!