rotation invariant spherical harmonic representation of 3d shape descriptors
DESCRIPTION
Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors. Michael Kazhdan Th o mas Funkhouser Szymon Rusinkiewicz Princeton University. Motivation. Large databases of 3D models. Computer Graphics (Princeton 3D Search Engine). Mechanical CAD - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/1.jpg)
Rotation Invariant Spherical Harmonic Representation of
3D Shape DescriptorsMichael KazhdanMichael Kazhdan
ThThoomas Funkhousermas FunkhouserSzymon RusinkiewiczSzymon RusinkiewiczPrinceton UniversityPrinceton University
![Page 2: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/2.jpg)
Motivation
Large databases of 3D modelsLarge databases of 3D models
Mechanical CAD(National Design Repository)
Molecular Biology(Audrey Sanderson)
Computer Graphics(Princeton 3D Search Engine)
![Page 3: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/3.jpg)
Retrieval Approach
3D Model ShapeDescriptor
Model Database
Nearest Neighbor
![Page 4: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/4.jpg)
Shape Unchanged by Rotation
=
![Page 5: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/5.jpg)
Problem
Many shape descriptors are functions that Many shape descriptors are functions that rotate with the shaperotate with the shape
Extended Gaussian Image[Horn ’84]
Spherical Attribute Image[Ikeuchi ’95]
Shape Histogram [Ankerst ’99]
Spherical Extent Function[Vranic ’00]
Reflective Symmetry Descriptor [Kazhdan ’02]
Gaussian EDT[Funkhouser ’03]
![Page 6: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/6.jpg)
Goal
Compute similarity of shape descriptors Compute similarity of shape descriptors independent of rotationindependent of rotation
- = ?
![Page 7: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/7.jpg)
Brute Force Approach
-
-
-
-
min (rotation)- =
Impractical for databasesImpractical for databases
![Page 8: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/8.jpg)
Normalization
Use PCA to place models into a canonical Use PCA to place models into a canonical coordinate framecoordinate frame
Covariance MatrixComputation
Principal Axis Alignment
![Page 9: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/9.jpg)
Normalization
Doesn’t always workDoesn’t always work• Only second order informationOnly second order information
![Page 10: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/10.jpg)
Our Approach
Eliminate rotation dependence in spherical Eliminate rotation dependence in spherical and 3D descriptorsand 3D descriptors
Shape Descriptor
EGI [Horn ’84]SAI [Ikeuchi ’95]EXT [Vranic ’00]RSD [Kazhdan ’02]EDT [Funkhouser ’03]etc.
Shape Descriptor
![Page 11: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/11.jpg)
Our Approach
Eliminate rotation dependence in spherical Eliminate rotation dependence in spherical and 3D descriptorsand 3D descriptors
Rotation Invariant
Representation
Shape Descriptor
![Page 12: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/12.jpg)
Outline
IntroductionIntroductionBackgroundBackgroundHarmonic RepresentationHarmonic RepresentationPropertiesPropertiesExperimental ResultsExperimental ResultsConclusion and Future WorkConclusion and Future Work
![Page 13: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/13.jpg)
Key Idea
Obtain rotation invariant representation by Obtain rotation invariant representation by storing amplitude and eliminating phasestoring amplitude and eliminating phase
+ + + +…=
[Lo 1989]
[Burel 1995]
![Page 14: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/14.jpg)
Fourier Descriptors
CircularFunction
![Page 15: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/15.jpg)
Fourier Descriptors
+ + += + …
Cosine/Sine Decomposition
CircularFunction
![Page 16: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/16.jpg)
Fourier Descriptors
=
+ + +
Constant
= + …
Frequency Decomposition
CircularFunction
![Page 17: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/17.jpg)
Fourier Descriptors
=
+ + +
+
Constant 1st Order
= + …+
Frequency Decomposition
CircularFunction
![Page 18: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/18.jpg)
Fourier Descriptors
=
+ + +
+ +
Constant 1st Order 2nd Order
= + …+
Frequency Decomposition
CircularFunction
![Page 19: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/19.jpg)
Fourier Descriptors
=
+ + +
+ + +
Constant 1st Order 2nd Order 3rd Order
= + …
+ …
+
Frequency Decomposition
CircularFunction
![Page 20: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/20.jpg)
+ + + + …+
Fourier Descriptors
= + + +
Constant 1st Order 2nd Order 3rd Order
+ …
Frequency Decomposition
=Amplitudes invariant
to rotationCircularFunction
![Page 21: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/21.jpg)
Harmonic Representation
SphericalFunction
![Page 22: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/22.jpg)
Harmonic Representation
=
SphericalFunction
+ + + + …
Harmonic Decomposition
![Page 23: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/23.jpg)
Harmonic Representation
=
SphericalFunction
+ + + + …
+ + + +…=
Constant 1st Order 2nd Order 3rd Order
![Page 24: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/24.jpg)
Harmonic Representation
+ + + +…=
Norms Invariantto Rotation
Store “how much” Store “how much” (L(L22-norm) of the -norm) of the shape resides in shape resides in each frequencyeach frequency
![Page 25: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/25.jpg)
3D Function (Voxel Grid)
Restrict to concentric spheresRestrict to concentric spheres
![Page 26: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/26.jpg)
3D Function (Voxel Grid)
=
=
=
+
+
+ +
+
+ +
+
+
+
+
+
Compute harmonic representation of each Compute harmonic representation of each sphere independentlysphere independently
![Page 27: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/27.jpg)
3D Function (Voxel Grid)
Combine harmonic representationsCombine harmonic representations
Radius
Frequency
![Page 28: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/28.jpg)
Matching
LL22-difference of harmonic representations…-difference of harmonic representations…
Harmonic Representation Harmonic Representation
-2
![Page 29: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/29.jpg)
Matching
min (rotations)
--2 2
… … bounds proximity of descriptors over all bounds proximity of descriptors over all rotationsrotations
![Page 30: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/30.jpg)
Outline
IntroductionIntroductionBackgroundBackgroundHarmonic RepresentationHarmonic RepresentationPropertiesPropertiesExperimental ResultsExperimental ResultsConclusion and Future WorkConclusion and Future Work
![Page 31: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/31.jpg)
Advantages
The harmonic representations is:The harmonic representations is:•Rotation invariantRotation invariant•Multi-resolutionMulti-resolution•CompactCompact•DiscriminatingDiscriminating
![Page 32: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/32.jpg)
Compact
…
……
![Page 33: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/33.jpg)
Compact
……
……
![Page 34: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/34.jpg)
Compact
……
……
![Page 35: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/35.jpg)
Compact
……
……
![Page 36: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/36.jpg)
Compact
……
……
![Page 37: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/37.jpg)
Information Loss
•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss•Cross-radial information lossCross-radial information loss
![Page 38: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/38.jpg)
Information Loss (Spherical Descriptor)
•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss
![Page 39: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/39.jpg)
Information Loss (Spherical Descriptor)
•Intra-frequency information lossIntra-frequency information loss•Cross-frequency information lossCross-frequency information loss
+
+
22.5o90o
=
=
![Page 40: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/40.jpg)
Information Loss (3D Descriptor)
•Cross-radial information lossCross-radial information loss
![Page 41: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/41.jpg)
Outline
IntroductionIntroductionBackgroundBackgroundHarmonic RepresentationHarmonic RepresentationPropertiesPropertiesExperimental ResultsExperimental ResultsConclusion and Future WorkConclusion and Future Work
![Page 42: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/42.jpg)
Shape DescriptorsExtended Gaussian Image
Horn 1984
Spherical Extent Function Vranic 2000
Shape Histogram Ankerst 1999
Gaussian EDT Funkhouser 2003
![Page 43: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/43.jpg)
Experimental Database
Viewpoint “household” databaseViewpoint “household” database1,890 models, 85 classes1,890 models, 85 classes
153 dining chairs 25 livingroom chairs 16 beds 12 dining tables
8 chests 28 bottles 39 vases 36 end tables
![Page 44: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/44.jpg)
Gaussian EDT ResultsPCA-Normalized Results
Harmonic Representation Results
Query
11 22 33 44 55
66 77 88 99 1010
11 22 33 44 55
66 77 88 99 1010
![Page 45: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/45.jpg)
Gaussian EDT Results
Precision vs. RecallPrecision vs. Recall
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
HarmonicsPCA
![Page 46: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/46.jpg)
Retrieval ResultsHarmonicsPCA
HarmonicsPCA
HarmonicsPCA
HarmonicsPCA
EGI
EDTEXT
SECT
50% 100%
100%
0%
50%
RecallPr
ecis
ion
0%
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
•EGI: Extended Gaussian Image
•SECT: Shape Histogram (Sectors)
•EXT: Spherical Extent Function
•EDT: Gaussian Euclidean Distance Transform
![Page 47: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/47.jpg)
Retrieval ResultsHarmonicsPCA
EGIHarmonicsPCA
SECT
50% 100%
100%
0%
50%
RecallPr
ecis
ion
0%
HarmonicsPCA
EDT
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
HarmonicsPCA
EXT
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
50% 100%
100%
0%
50%
Recall
Prec
isio
n
0%
•EGI: Extended Gaussian Image
•SECT: Shape Histogram (Sectors)
•EXT: Spherical Extent Function
•EDT: Gaussian Euclidean Distance Transform
![Page 48: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/48.jpg)
Exhaustive Gaussian EDT Results
HarmonicPCA
min L2
100%
50%
0%0% 50% 100%
Recall
Prec
isio
n
Gaussian EDT -
-
-
-
min (rotation)
![Page 49: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/49.jpg)
Summary and Conclusion
Provide a rotation invariant representation of Provide a rotation invariant representation of shape descriptors that:shape descriptors that:• Eliminates PCA dependenceEliminates PCA dependence• Gives better matching performanceGives better matching performance• Is more compactIs more compact• Is a multi-resolution representationIs a multi-resolution representation
![Page 50: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/50.jpg)
Future Work
Managing Information LossManaging Information Loss• Obtain cross radial information for 3D descriptorsObtain cross radial information for 3D descriptors• Obtain cross frequency informationObtain cross frequency information• Get finer resolution of rotation invariance within Get finer resolution of rotation invariance within
frequenciesfrequencies
More GenerallyMore Generally• Consider new shape descriptorsConsider new shape descriptors
![Page 51: Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors](https://reader035.vdocuments.us/reader035/viewer/2022070419/56815ba9550346895dc9ab00/html5/thumbnails/51.jpg)
Thank YouFundingFunding
National Science FoundationNational Science Foundation
Sloan FoundationSloan Foundation
Spherical HarmonicsSpherical HarmonicsDan Rockmore and Peter KostelecDan Rockmore and Peter Kostelec
SpharmonicKit:SpharmonicKit:
http://www.cs.dartmouth.edu/~geelong/spherehttp://www.cs.dartmouth.edu/~geelong/sphere3D Shape Matching3D Shape Matching
Patrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce ChenPatrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce Chen
Princeton 3D Model Search Engine:Princeton 3D Model Search Engine:
http://shape.cs.princeton.eduhttp://shape.cs.princeton.edu