data-driven shape analysis --- joint shape matching...
TRANSCRIPT
![Page 1: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/1.jpg)
Data-Driven Shape Analysis--- Joint Shape Matching II
1
Qi-xing HuangStanford University
![Page 2: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/2.jpg)
Last lecture
X º 0
(Positive) semidefiniteness
Xij = XTj1Xi1 X =
264 Im...XTn1
375 h Im ¢ ¢ ¢ Xn1i
![Page 3: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/3.jpg)
Last lecture
Xii = Im; 1 · i · nsubject to
minimizeP
(i;j)2EkXinputij ¡Xijk1
Xij1 = 1;XTij1 = 1; 1 · i < j · n
X º 0X ¸ 0
ADMM [Boyd et al.11]
![Page 4: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/4.jpg)
Last lecture
Xii = Im; 1 · i · nsubject to
minimizeP
(i;j)2EkXinputij ¡Xijk1
Xij1 = 1;XTij1 = 1; 1 · i < j · n
X º 0X ¸ 0
ADMM [Boyd et al.11]
![Page 5: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/5.jpg)
Deterministic guarantee• Exact recovery condition:
#incorrect corres. per point< algebraic-connectivity(G)/4
![Page 6: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/6.jpg)
Constrained optimization framework
minimize
Subject to
Constraints on X
Symmetricmatrices
minimize
Subject to Constraints on X
Asymmetricmatrices
![Page 7: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/7.jpg)
Rotation
minimize
subject to
[Wang and Singer’13]
![Page 8: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/8.jpg)
• Aligning large-shape collections
• Consistent functional maps
• SLAM
Outline
![Page 9: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/9.jpg)
Aligning shapes
![Page 10: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/10.jpg)
Sequential approach
Optimized orientations-- in the xy plane
Un-oriented shapes Optimized scaling andTranslation along (x,y,z)
![Page 11: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/11.jpg)
Consistent orientations
States of each shape
maxxi
P(i;j)2G
Áij(xi; xj)
Áij(k; l) = 1 Áij(k; l) = 0
Iterative coordinate ascent[Leordeanu et al 06]
MRF Formulation :
![Page 12: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/12.jpg)
Shape matching – local phase
P(i;i0)2G
d2(Fi(Si);Fi0(Si0))
FFD of each shape
Intractable
![Page 13: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/13.jpg)
Shape matching --- local phase
Pair-wise matching
![Page 14: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/14.jpg)
Shape matching --- local phase
Optimize the FFD for eachshape independently
Optimize the FFD for eachshape independentlymii0k = (Fi(pii0k) + Fi0(qii0k))=2
![Page 15: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/15.jpg)
Matching quality
Benchmark datasets [Kim et al 13]
![Page 16: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/16.jpg)
• Aligning large-shape collections
• Consistent functional maps
• SLAM
Outline
![Page 17: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/17.jpg)
Starting from a Regular Map
lion → cat
![Page 18: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/18.jpg)
Attribute Transfer via Pull-Back
cat → lion
![Page 19: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/19.jpg)
The Operator View of Maps
Functions on cat are transferred to lion using F
F is a linear operator (matrix)
from cat to lion
![Page 20: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/20.jpg)
Functional Correspondences
20
“probe functions”“F2F” correspondences are just as naturalas “P2P” correspondences
From
To
![Page 21: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/21.jpg)
Functional representation
Part-based correspondences
![Page 22: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/22.jpg)
Representation of Image Relationship
Point features Segments Descriptors
Indicator functionsDelta functions SIFT flow[Liu et al. 09]
2 RNN: #pixels
![Page 23: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/23.jpg)
Functional correspondence/map• Descriptors correspondence
• Segmentation correspondence
![Page 24: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/24.jpg)
Reduced representation
• Basis of functional space– First M eigenfunctions of the graph Laplacian
• Reconstruct any function with small error (M=30)
Binary indicator function Reconstructed function Thresholdedreconstructed function
…Example basis functions
![Page 25: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/25.jpg)
Approach I
![Page 26: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/26.jpg)
Estimation of Functional Maps• Recover functional maps by aligning image features:
![Page 27: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/27.jpg)
Estimation of Functional Maps• Regularization term:
– Correspond bases of similar spectra– Enforce sparsity of map
Map with regularization Map without regularization
![Page 28: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/28.jpg)
Problems of Pair-wise Maps• Transport a function along a cycle
![Page 29: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/29.jpg)
Problems of Pair-wise Maps• Transport a function along a cycle
![Page 30: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/30.jpg)
Problems of Pair-wise Maps• Transport a function along a cycle
– typically does not go back
![Page 31: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/31.jpg)
Problems of Pair-wise Maps• Transport a function along a cycle
– typically does not go back
• Need to rectify pair-wise maps
![Page 32: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/32.jpg)
Consistency of the Network• Cycle-consistency
– A function transported along any loop should be identicalto the original function
ijX
jkX
kiX
![Page 33: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/33.jpg)
Consistency of the Network• Cycle-consistency
– A function transported along any loop should be identicalto the original function
ijX
jkX
kiX
if
kf jf
ki jk ij i iX X X f f
![Page 34: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/34.jpg)
Enforcing Cycle-Consistency
– Consistency term:
![Page 35: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/35.jpg)
Joint Estimation of Functional Maps• Overall optimization
minX
(i;j)2Gwij(kXijDi ¡Djk1
+ ¹X
1·s;s0·M
³j¸si ¡ ¸s
0j jXij(s; s0)
´2+ ¸kXijYi ¡ Yjk2F)
s:t: Y TY = Im
![Page 36: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/36.jpg)
Joint Estimation of Functional Maps• Overall optimization
minX
(i;j)2Gwij(kXijDi ¡Djk1
+ ¹X
1·s;s0·M
³j¸si ¡ ¸s
0j jXij(s; s0)
´2+ ¸kXijYi ¡ Yjk2F)
s:t: Y TY = Im
When Y is fixed: solving independent pair-wise functional maps
![Page 37: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/37.jpg)
Joint Estimation of Functional Maps• Overall optimization
minX
(i;j)2Gwij(kXijDi ¡Djk1
+ ¹X
1·s;s0·M
³j¸si ¡ ¸s
0j jXij(s; s0)
´2+ ¸kXijYi ¡ Yjk2F)
s:t: Y TY = Im
When X is fixed: exact eigen-decomposition problem
![Page 38: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/38.jpg)
• Indicator function transportation– Better correspondence between “cows” with map
consistency
Consistent Functional Maps
Source
Target
Withoutconsistency
Withconsistency
![Page 39: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/39.jpg)
Approach II
![Page 40: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/40.jpg)
• There exists latent basis on each object
• Definition: For each basis vector for each loop
• Equivalent definition:
Consistency
or
Expanded
![Page 41: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/41.jpg)
Map collection matrix is low-rank
Low-rank factorization
![Page 42: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/42.jpg)
• Pair-wise probe functions
• Formulation
Matrix recovery
Local optimization to recover Y
![Page 43: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/43.jpg)
Illustraction
![Page 44: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/44.jpg)
Consistent functions
![Page 45: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/45.jpg)
Multi-level
= = =
Super-objects
![Page 46: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/46.jpg)
• Aligning large-shape collections
• Consistent functional maps
• SLAM
Outline
![Page 47: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/47.jpg)
Geometry reconstruction
RegistrationReconstruction
Scanning
![Page 48: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/48.jpg)
Joint registration and reconstruction
Initial FinalHuang, Adams, Wand.Bayesian Surface Reconstruction via Iterative Scan Alignment to an Optimized Prototype,SGP’07.
![Page 49: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/49.jpg)
Fragment reconstruction
Ephesus, turkey
![Page 50: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/50.jpg)
SLAM
22K scans, 180 points per scan
Huang and Anguelov.High Quality Pose Estimation by Aligning Multiple Scans to a Latent Map. ICRA'10.
![Page 51: Data-Driven Shape Analysis --- Joint Shape Matching IIgraphics.stanford.edu/courses/cs468-14-spring/slides... · 2014-04-24 · Initial Final Huang, Adams, Wand. Bayesian Surface](https://reader035.vdocuments.us/reader035/viewer/2022070911/5fb3821b853948172e00614b/html5/thumbnails/51.jpg)
SLAM
Computer history museum Exploratorium museum