biological shape
DESCRIPTION
Matching Shapes Serge Belongie * , Jitendra Malik and Jan Puzicha U.C. Berkeley * Present address: U.C. San Diego. Biological Shape. D’Arcy Thompson: On Growth and Form , 1917 studied transformations between shapes of organisms. Deformable Templates: Related Work. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/1.jpg)
Computer Vision GroupUniversity of California Berkeley
Matching Shapes
Serge Belongie*, Jitendra Malikand Jan Puzicha
U.C. Berkeley
*Present address: U.C. San Diego
![Page 2: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/2.jpg)
Computer Vision GroupUniversity of California Berkeley
Biological Shape
• D’Arcy Thompson: On Growth and Form, 1917– studied transformations between shapes of organisms
![Page 3: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/3.jpg)
Computer Vision GroupUniversity of California Berkeley
Deformable Templates: Related Work
• Fischler & Elschlager (1973)
• Grenander et al. (1991)
• Yuille (1991)
• von der Malsburg (1993)
![Page 4: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/4.jpg)
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
![Page 5: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/5.jpg)
Computer Vision GroupUniversity of California Berkeley
Comparing Pointsets
![Page 6: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/6.jpg)
Computer Vision GroupUniversity of California Berkeley
Shape ContextCount the number of points inside each bin, e.g.:
Count = 4
Count = 10
...
Compact representation of distribution of points relative to each point
![Page 7: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/7.jpg)
Computer Vision GroupUniversity of California Berkeley
Shape Context
![Page 8: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/8.jpg)
Computer Vision GroupUniversity of California Berkeley
Comparing Shape Contexts
Compute matching costs using Chi Squared distance:
Recover correspondences by solving linear assignment problem with costs Cij
[Jonker & Volgenant 1987]
![Page 9: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/9.jpg)
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
![Page 10: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/10.jpg)
Computer Vision GroupUniversity of California Berkeley
• 2D counterpart to cubic spline:
• Minimizes bending energy:
• Solve by inverting linear system
• Can be regularized when data is inexact
Thin Plate Spline Model
Duchon (1977), Meinguet (1979), Wahba (1991)
![Page 11: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/11.jpg)
Computer Vision GroupUniversity of California Berkeley
MatchingExample
model target
![Page 12: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/12.jpg)
Computer Vision GroupUniversity of California Berkeley
Outlier Test Example
![Page 13: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/13.jpg)
Computer Vision GroupUniversity of California Berkeley
Synthetic Test Results
Fish - deformation + noise Fish - deformation + outliers
ICP Shape Context Chui & Rangarajan
![Page 14: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/14.jpg)
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
![Page 15: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/15.jpg)
Computer Vision GroupUniversity of California Berkeley
Terms in Similarity Score• Shape Context difference
• Local Image appearance difference– orientation– gray-level correlation in Gaussian window– … (many more possible)
• Bending energy
We used the weighted sumShape context + 1.6*image appearance + 0.3*bending
![Page 16: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/16.jpg)
Computer Vision GroupUniversity of California Berkeley
Object Recognition Experiments
• COIL 3D Objects
• Handwritten Digits
• Trademarks
Exactly the same algorithm used on all experiments
![Page 17: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/17.jpg)
Computer Vision GroupUniversity of California Berkeley
COIL Object Database
![Page 18: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/18.jpg)
Computer Vision GroupUniversity of California Berkeley
Error vs. Number of Views
![Page 19: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/19.jpg)
Computer Vision GroupUniversity of California Berkeley
Prototypes Selected for 2 Categories
Details in Belongie, Malik & Puzicha (NIPS2000)
![Page 20: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/20.jpg)
Computer Vision GroupUniversity of California Berkeley
Error vs. Number of Views
![Page 21: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/21.jpg)
Computer Vision GroupUniversity of California Berkeley
Handwritten Digit Recognition
• MNIST 60 000: – linear: 12.0%
– 40 PCA+ quad: 3.3%
– 1000 RBF +linear: 3.6%
– K-NN: 5%
– K-NN (deskewed): 2.4%
– K-NN (tangent dist.): 1.1%
– SVM: 1.1%
– LeNet 5: 0.95%
• MNIST 600 000 (distortions): – LeNet 5: 0.8%– SVM: 0.8%– Boosted LeNet 4: 0.7%
• MNIST 20 000: – K-NN, Shape Context
matching: 0.63%
![Page 22: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/22.jpg)
Computer Vision GroupUniversity of California Berkeley
![Page 23: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/23.jpg)
Computer Vision GroupUniversity of California Berkeley
Trademark Similarity
![Page 24: Biological Shape](https://reader036.vdocuments.us/reader036/viewer/2022062407/56812ee0550346895d947ee5/html5/thumbnails/24.jpg)
Computer Vision GroupUniversity of California Berkeley
Conclusion
• Introduced new matching algorithm matching based on shape contexts and TPS
• Robust to outliers & noise
• Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm