probabilistic fingerprints for shapes niloy j. mitraleonidas guibas joachim giesenmark pauly...

Probabilistic Fingerprints for Shapes

Niloy J. Mitra Leonidas Guibas Joachim Giesen Mark Pauly

Stanford University MPII Saarbrücken ETH Zurich

Introduction

• Shape Analysis and Comparison• shape retrieval, shape clustering, feature selection,

correspondence, compression, re-use, etc

•Question: Are two shapes similar?

≈ ?

Introduction

• More general: Are two shapes similar in parts?• relative size of overlap region partially matching under

rigid motion

• scan alignment

• context-based editing

• shape recognition, etc.

• Efficient tests require compact signatures• database query

• network setting

• fast pre-filtering, etc.

Background

• Methods for global registration• Gelfand, Mitra, Guibas and Pottmann, Robust Global Registration, SGP 2005

• Li and Guskov, Multi-scale Features for Approximate Alignment of Point-based Surfaces, SGP 2005

• Huber and Hebert, Fully Automatic Registration of Multiple 3D Data Sets, CVBVS 2001

• Global shape descriptors• Kazhdan, Funkhouser and Rusinkiewicz, Rotation Invariant Spherical

Harmonic Representation of 3D Shape Descriptors, SGP 2003

• Osada, Funkhouser, Chazelle and Dobkin, Shape Distributions, ACM TOG 2002

• Reuter and Wolter, Laplace-Spectra as fingerprints for shape matching, SPM 2005

Background

• Geometric Hashing• Wolfson, Rigoutsos. Geometric Hashing: An Overview, IEEE Computational

Science and Engineering, 4(4), 1997

• Gal and Cohen-Or, Salient geometric features for partial shape matching and similarity, ACM TOG 2006

• File matching• Broder, Glassman, Manasse, Zweig. Syntactic Clustering of the Web, World

Wide Web Conference, 1997

• Broder, On the Resemblance and Containment of Documents, Sequences 1997

• Schleimer, Wilkerson and Alex Aiken, Winnowing: local algorithms for document fingerprinting, Sigmod, ’03

Probabilistic Fingerprints

• Function such that• Given two shapes S1 and S2, with high probability

• if f(S1) ≠ f(S2) then S1 and S2 are dissimilar

• if f(S1) = f(S2) then S1 and S2 are similar

• f is efficiently computable

• compact, i.e.,

• output sensitive

• localized (partial matching)

• robust to sampling and articulated motion

Pre-Processing

Input

Sample

Shingles

SignaturesDescriptorsFingerprint

Pre-Processing

Input Sample

• Uniform random sample • guarantee δ-coverage

• avoid arbitrarily dense sampling [Turk 92]

such that

Sample

Pre-Processing

Shingles

• Local surface patches• intersection with ρ-balls

• create sufficient overlap for robust signature estimation, i.e.,

Pre-Processing

Shingles Signatures

• Local signatures should be invariant to• rigid transforms

• sampling & local perturbations

• Examples: Spin images, shape histograms, integral descriptors, etc.

Pre-Processing

DescriptorsSignatures

• Optional: Compressed descriptors• e.g., Rabin’s hashing

• Signature set • multi-set of points in high-dimensional space

• spatial relation of shingles not preserved

Resemblance

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

Resemblance

Probabilistic Fingerprint

• Let be random permutations

• Estimate of resemblanceindicator function

Probabilistic Fingerprint

• Estimate of resemblance

• Example: m = 3

8 0 32 54 76 91 1011 1213 14

8 03 25 4769 1 101112 1314

8 03 25 47 6 91 1011 12 1314

Probabilistic Fingerprint

• Estimate of resemblance

8 0 32 54 76 91 1011 1213 14

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

Probabilistic Fingerprint

• Estimate of resemblance

8 0 32 54 76 91 1011 1213 14

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

Probabilistic Fingerprint

• Estimate of resemblance

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

8 03 25 4769 1 101112 1314

Probabilistic Fingerprint

• Estimate of resemblance

8 03 25 4769 1 101112 1314

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

Probabilistic Fingerprint

• Estimate of resemblance

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

8 03 25 47 6 91 1011 12 1314

Probabilistic Fingerprint

• Estimate of resemblance

80 32 54 76 91 10 11 12 13 14

80 32 54 76 91 10 11 12 13 14

8 03 25 47 6 91 1011 12 1314

Pre-Processing

FingerprintDescriptors

• Probabilistic Fingerprint• reduce using min-hashing

• based on random permutations of universe

• set of ‘random experts’ consistent for all models

Min-Hashing

• Feature selection by random experts• reduces set comparison to element-wise

comparison

• estimate resemblance using m permutations = perform m coin tosses to estimate bias of coin

• Analysis• probabilistic bounds using Markov inequality &

strong Chernoff bound

• relates size of the fingerprint to confidence in estimated resemblance

Data Reduction

Shingles Signatures Descriptors Fingerprint

quantization min hashing

set size remains constant

100k 100k 100k 1k

set reduction

Applications

• Resemblance between partial scans

Applications

• Adaptive feature selection

Applications

• Alignment using adaptive feature selection

scan A scan B final alignment

Applications

• Multiple scans• greedy alignment

using priority queue

• fingerprint matching determines score

• advanced alignment method for verification

• merging fingerprints requires no re-computation

Applications

• Shape distributions

Applications

• Database retrieval

Statistics

• Pre-processing time in seconds:

• Query time: ~ 15 msec on average

• Fingerprint size ~10kb

model #vts. uniform sampl.

spin image

Rabin hash

min-hash

skull 54k 0.8 7.5 0.05 4.5

Caesar 65k 1.4 7.3 0.08 10.3

bunny 121k 1.8 13.8 0.04 2.9

horse 8k 0.7 5.7 0.05 7.3

Remarks & Insights

• Resemblance defined as set operation on signature sets → quantization is crucial

• Random experts effectively extract consistent set of features → requires no explicit correspondence

• Fingerprints do not preserve spatial relation of shingles → false positives are possible

• Few parameters that are easy to tune

Remarks & Insights

• Accumulate local evidence for global inference

• Spatial structure vs. unordered signature set?

• Semantic features vs. random experts?

Thank You!