bag-of-feature-graphs: a new paradigm for non-rigid shape retrieval
DESCRIPTION
Bag-of-Feature-Graphs: A New Paradigm for Non-rigid Shape Retrieval. Tingbo HOU, Xiaohua HOU, Ming ZHONG and H ong QIN Department of Computer Science Stony Brook University (SUNY SB). Nonrigid Shape Retrieval. Shape Query. S hape D atabase. R etrieved S hapes. …. …. - PowerPoint PPT PresentationTRANSCRIPT
ICPR 2012Department of Computer Science Center for Visual Computing
Bag-of-Feature-Graphs: A New Paradigm for Non-rigid Shape Retrieval
Tingbo HOU, Xiaohua HOU, Ming ZHONG and Hong QIN
Department of Computer ScienceStony Brook University (SUNY SB)
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Nonrigid Shape Retrieval
……
Shape Query
Shape Database
Retrieved Shapes
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Overview of BoFG Inspired by the ideas from Bag-of-Words (BoW) and Spatial-
Sensitive Bag-of-Words (SS-BoW)
Feature-driven
Concise and fast to compute
Spatially informative
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Previous Works Relevant to This Project
Bag-of-Words1. Y. Liu, H. Zha, and H. Qin. CVPR, 2006.2. H. Tabia, M. Daoudi, J. P. Vandeborre, and O. Colot. 3DOR, 2010.3. R. Toldo, U. Castellani, and A. Fusiello. VC, 2010. 4. G. Lavoué. 3DOR, 2011.
Shape Google (Spatially-Sensitive Bag-of-Words)1. M. Ovsjanikov, A. M. Bronstein, L. J. Guibas and M. M. Bronstein. NORDIA,
2009.2. (SI-HKS) M. M. Bronstein and I. Kokkinos. CVPR, 2010.3. A. M. Bronstein, M. M. Bronstein, L. J. Guibas, and M. Ovsjanikov. ACM
TOG, 2011.
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Background (1) Heat Kernel on surface
Amount of heat transferred from a point to in time
: -th eigenvalue and eigenfunction of the Laplace-Beltrami operator Heat Kernel Signature (HKS):
HKS descriptor A vector of HKS probed at different values of
Properties of Heat Kernel Intrinsic (Invariant to rigid and isometric deformation) Informative (locally and globally shape aware) Stable
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Background (2) Geometric words
A representative HKS vector Clustered in the HKS descriptor space by the k-means algorithm
Vocabulary
Similarity of point and word
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Shape-Google Revisit (1)
Bag-of-Words Word distribution of each point
BoW descriptor: vector
Measure the frequencies of words appearing on the shape
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Shape-Google Revisit (2)
Spatially-Sensitive Bag-of-Words SS-BOW descriptor: matrix
Measure the frequencies of word pairs
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
New Paradigm: Bag-of-Feature Graphs (1)
Motivation: Reduce computation complexity
Considering all points on shape -> only considering feature points
Vector/matrix of word frequencies -> feature graphs associated with words
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Formulation (1) Feature set:
Feature graph associated with the -th geometric word represented as matrix
: Heat Kernel
Bag-of-Feature-Graphs representation of shape
…𝐺1 𝐺2 𝐺3
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Formulation (2) BoFG descriptor
Multi-dimensional scaling (MDS): Choosing the 6 largest eigenvalues of each graph matrix denoted by
vector
Shape distance
Retrieval by approximate nearest neighbor (ANN) search
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Nonrigid Shapes and Their BoFG Descriptors
ICPR 2012Department of Computer Science Center for Visual Computing
Time Complexity of BoW, SS-BOW and BoFG
: Number of vertices
: Time complexity for computing HKS descriptor of a vertex
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Experiments Test dataset: TOSCA1
12 classes of 148 non-rigid shapes Each shape has 3K 30K vertices
Evaluated methods: BoW, FSS-BoW, SI-HKS,
Vocabulary 48 words for BoW and SS-BoW (clustered from all shape points) 4 words for BoFG (clustered only from feature points)
Feature numbers in BoFG: for each shape
1http://toca.cs.technion.ac.il/book/shrec.html
ICPR 2012Department of Computer Science Center for Visual Computing
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Experiments
Time performance (in seconds) of three descriptors on two shapes with 3K and 30k vertices
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Experiments
Precision-recall curves of evaluated methods, with categories of (1) null, (2) scale changes and (3) holes.
(1) (2) (3)
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Partial shape retrieval Query shape is only a part of a complete model
Online feature alignment is required to extract corresponding sub-graphs
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Summary Bag-of-Feature-Graphs (BoFG) is a new paradigm for shape
representation
This representation is feature-driven, concise, and spatially-aware
The key idea is to construct graphs of features associated with geometric words
BoFG has much improved time-performance and competitive retrieval results in comparison with other state-of-the-art methods
Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing
Future Work Investigate graph comparison with heavy outliers
Improve the performance on partial shape retrieval
Acknowledgements: Research Grants from National Science Foundation
ICPR 2012Department of Computer Science Center for Visual Computing
Thank You!
Questions?