a diffusion wavelet approach for 3 d model matching
TRANSCRIPT
A DIFFUSION WAVELET APPROACH FOR 3-D MODEL
MATCHINGAuthors: K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh
Presented by: Raphael Steinberg
2
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
3
INTRODUCTION
Currently - A larger than ever number of 3D Models in CAD, computer games, multimedia, molecular biology, computer vision and more
There is a need for 3D Retrieval
4
INTRODUCTION (2)
Tagging are not always available or sufficient to describe the model we require
Combine topological information with multi-scale properties
5
Model Reusability (CAD/Animation)
Model MatchingVideo Retrieval (2.5D/Virtual
environments)EcommerceCorrecting defectsEfficient RepresentationMany other uses…
MOTIVATION FOR 3D RETRIEVAL
6
OBSTACLES IN RETRIEVAL
Partial retrieval - Non-transitive
Functional description
How to match text tags with vertices and texture?
Orthonormal coordinate system
7
8
3D MODEL MATCHING – PRIOR ART
Feature vectors using wavelets to mesh vertices – localized in both space & frequency – Paquet et. al. 2000
Random sampling for comparison – Osada et. al. 2001
9
SPHERICAL HARMONICS (SH)
Global method in Euclidean space
lacks multi-scale analysis
Legendre polynomials solve the Laplace equation in Spherical coordinatesVranic et. al. 2001
10
SPHERICAL WAVELETS (SW)
Multi-scale in Euclidean space
Lacks connectivity on the manifold
Tannenbaum et. al. 2007
11
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
12
DIFFUSION MAPS INTRODUCTION
Originally suggested by Stephan Lafon and R.R. Coifman from Yale Math, circa 2005
Many other manifold learning techniques exist
Data analysis based on geometric properties of the data set
13
MANIFOLD LEARNING ALGORITHMS
MANI - Manifold learning Matlab tool
14
DIFFUSION MAPS
X
Y
Z
• vi is a feature vector
• Contains descriptive information about the 3D model
Kiv
,Div D K
Coifman - 2005
15
DIFFUSION MAPS
Assumptions• Points are sampled uniformly on
the manifold• Smooth manifold (no fractals in
our case)• Fixed boundary conditions• Enough points = feature vectors
(N→∞)
16
1) Use RBF Gaussian Kernel to choose ε
2) Normalize W to create a Stochastic Matrix
DIFFUSION MAPS ALGORITHM
2/2
, , e i jx x
i j i jw x x
1M D W
,1
N
i jj
D w
Lu et. al. 2009
17
DIFFUSION MAPS ALGORITHM (2)
3) Diffuse by taking higher powers of t “The diffusion distance is equal to the Euclidean distance in the diffusion map space” , Nadler et. al. 2005
4) Cut manifold according to dominant eigenvalues
18
DIFFUSION MAPS CODE EXAMPLEfunction checker();
close all;
tetha=2*pi*rand(1,500);
z=[cos(tetha);sin(tetha)];
figure(1);scatter(z(1,:),z(2,:),'b*');hold on;
N=size(z,2);
epsilon=linspace(0.01,.3,10);
%epsilon=.3;
W=nan(N);
summer=nan(1,length(epsilon));
for k=1:length(epsilon)
for i=1:N
parfor j=1:N
W(i,j)=exp(-sum((z(:,j)-z(:,i)).^2)/2/epsilon(k));
end
end
summer(k)=sum(sum(W));
end
figure;scatter(log(epsilon),log(summer));title('Epsilon - linear region')
p=polyfit(log(epsilon),log(summer),1);
d=2*p(1);%manifold dimension
M=W*diag(1./sum(W,2));
[U V]=svds(M);
sync=max(U(:,2));
figure(1);scatter(U(:,2)./sync,U(:,3)./sync,'rd')
title('Original manifold as stars and reconstructed manifold as diamonds')
end
19
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
20
PROBLEMS WITH MESH SIMPLIFICATION
21
WAVELETS
22
NOVELTY – DIFFUSION WAVELETS
Combination of Diffusion Maps and Wavelets
Used for non-linear dimensionality reduction
Extension of wavelets to the unit circle (just as diffusion maps extends the Fourier transform)
23
DIFFUSION WAVELETS INTUITION
fine c o a rs e
o rigina l m e s h
glo b a l
lo c a l
glo b a l
lo c a l
24
EXAMPLE OF DIFFUSION WAVELETS
Wavelet basis
ψ(2,2,3)
Scaling basis φ(1,1,1)
Wavelet basis
ψ(4,2,5)
Waveletbasis
ψ(3,2,3)
25
DIFFUSION WAVELETS
Use an optimization scheme to construct the scaling functions
Each scaling function should deal with a single dimension and be orthogonal to the other scaling functions
Extension of wavelets to the sphere (or to any other manifold)
26
DIFFUSION WAVELETS (2)
Better than LOD (Level of Detail - simplifies meshes)
Involved algorithm – very few implementations exist
0 1 1j
1 2 1j
0M 1M 1jM jM
0G 1G 1jG jG2
1A 2 j
jA0A
... ...
...
27
WAVELET DECOMPOSITION EXAMPLE
f ine
c o ars e
s c ale j= 0
s c ale j= 1
s c ale j= 4
x 0,0
x 1,1 x 1,2
x 2,1 x 2,2
x 4,1 x 4,2
28
WAVELET COEFFICIENTS
0 500 1000 1500-0.1
0
0.1
0.2
0.3
0 500 1000 1500-0.05
0
0.05
0.1
0.15
0.2
0 500 1000 1500-0.4
-0.2
0
0.2
0.4
0 500 1000 1500-0.4
-0.2
0
0.2
0.4
Scale 3 Scale 4
Scale 1 Scale 2
29
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
30
FINDING SHAPE FEATURE VECTORS (X)
, , , , , 0, 4j j j j jX j a c j
2 3 4 5 6 7 8 9 10
0.65
0.7
0.75
0.8
0.85
0.9
Number of decomposition level
Cla
ssifi
catio
n ra
te
• Take 1,450 coefficients (wavelet + scaling) at each decomposition level• Increase wavelet coefficient number from 0 to 450 and decrease scaling coefficient number from 1,450 to 1,000
31
FISHER’S DISCRIMINANT RATIO
b – between classesw – within class (after wavelet decomposition) j – scale
.
.b
w
Interclass distR
Intraclass distSS
-10 -5 0 50
0.2
0.4
0.6
0.8
1
1.2
1
2
Sw 1
Sw 2
Sb
32
FISHER’S DISCRIMINANT RATIO
'
4
, , , 4,3, ,0j
i jFDR X FDR X j
'2
1j j
w w i
j j
iw w
XFDR i
RX
C C
33
IRPR CURVE
Measure performance – use Princeton University 3D database
IRPR – Information Retrieval Precision-Recall
34
IRPR CURVE
m = relevant matches r = # of retrieved models
1) Precision =
2) Recall =
mr
1i
mn
i in Class size
1 2, , ,i
i i ii n
35
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
36
3D MODEL RETRIEVAL PROCEDURE
Compute the diffusion wavelet for each 3D model
Obtain the model representing vector X
Compute the 2nd order statistics of X for each scale
37
1) Start with a coarsest scale comparison
2)Advance up to the finest scale
3) Stop on threshold or when finest scale reached
* Use a threshold to determine if a model is from a certain class
MODEL MATCHING PROCEDURE
38
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
39
EXPERIMENTAL RESULTS
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
Recall (%)
Pre
cis
ion(%
)
refine to scale 0
refine to scale 4
refine to scale 8
0 20 40 60 80 1000
20
40
60
80
100
Recall (%)
Pre
cisi
on(%
)
DW
SW
SH
DW gives better results than SH and SW
Differences in scaling levels
40
VISUAL RESULTS
41
SCHEDULE
IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions
42
AUTHORS’ CONCLUSIONS
Surfaces with sharp peaks, grooves or holes contain high-frequency information which is not addressed by the wavelet multi-resolution (use diffusion wavelet packets instead?)
Possible to extend to partial matching
DW presents better results than SH and SW
43
MY CONCLUSIONS
Paper presents a novel solutionDiffusion Wavelets was never used before for 3D Retrieval
Less novel solutions:IRPR is a common measure in
database retrievalFischer Discriminant Ratio is a
common statistical measure
44
MY CONCLUSIONS (2)
Technically sound, feasibleTaking available code it seems
possible to reconstruct the resultsSeems like a reasonable solution to
the problem of 3D object retrieval
45
MY CONCLUSIONS (3)
The diffusion wavelet part could be explained in more detail
Not clear in which way the wavelets are constructed
How are the wavelet functions affected when a new model is inserted?
46
MY CONCLUSIONS (4)
Not self-containing but reference papers are exceptionally goodMissing explanations about
diffusion wavelets
47
“WOULD LIKE TO HAVE” (TECHNICAL/1)
Non-rigid extensionsHow would retrieval change if we
know the 3D model is non-rigid?Can we have an extension of
Diffusion Wavelets for non-rigid manifolds?
48
“WOULD LIKE TO HAVE” (TECHNICAL/2)
How to automatically choose the level of decomposition
49
“WOULD LIKE TO HAVE” (TECHNICAL/3)
An intuitive explanation - why prefer Diffusion Wavelets over Diffusion Wavelet Packets?
Wavelet Packets seem to give more information especially in high frequencies…
50
“WOULD LIKE TO HAVE” (TECHNICAL/4)
Numerical problems of overflow of the FDR - use logarithm instead of inverse?
51
“WOULD LIKE TO HAVE” (PRESENTATION/1)
Block diagram of the algorithm
52
“WOULD LIKE TO HAVE” (PRESENTATION/2)
Web-based Graphical User Interface
53
“WOULD LIKE TO HAVE” (PRESENTATION/3)
Error analysis
54
“WOULD LIKE TO HAVE” (PRESENTATION/4)
More explanations on Diffusion Wavelets
55
CONCLUSIONS
Shape retrieval requires multi-scale analysis
3D models, like most real-life objects, are embedded in a low dimension manifold
Results are robust to noise and to mesh simplifications
56
CONCLUSIONS (2)
Diffusion Wavelets give good retrieval results for 3D objects
Possible to extend the proposed method to include texture, sound, smell, elasticity and any other possibly given attribute of the 3D model
57
THE END
58
REFERENCES[1] K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh. , Department of Mechanical Engineering, National University of
Singapore “A diffusion wavelet approach for 3-D model matching” Computer Aided Design, Elsevier, Nov. 2008
[2] Presentation by R.R. Coifman et. al.
[3] J. Lu et. al. “Dominant Texture and Diffusion Distance Manifolds“, Eurographics, Volume 28 ,
Issue 2, Pages 667 - 676, Mar. 2009
[4] Diffusion wavelets Matlab code:
http://www.math.duke.edu/~mauro/diffusionwavelets.html#Code|outline
[5] The Princeton Shape Benchmark:
http://shape.cs.princeton.edu/benchmark/
[6] Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I. “Diffusion maps, spectral clustering and eigenfunctions of Fokker-Planck operators”.
[7] Ulrike von Luxburg, “A tutorial on spectral clustering”. Statistical Journal 2007
[8] Personal communications with K.P. Zhu
[9] MANI - Manifold learning Matlab tool
http://www.math.umn.edu/~wittman/mani/
[10] Vranic D, Saupe D, Richter J. Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics. In: Proc. IEEE workshop on multimedia signal processing; 2001. p. 29398.
59
REFERENCES[11] Osada R, Funkhouser T, Chazelle B, Dobkin D. Matching 3D models with shape
distributions, In: Proc. shape modeling international. 2001. p. 15466.
[12] Laga H, Nakajima M. Statistical spherical wavelet moments for content-based 3D model Retrieval. In: Computer graphics international 2007, CGI. 2007; 2007. p.1-8.
[13] Nain D, Haker S, Bobick A, Tannenbaum A. Multiscale 3-D shape representation and segmentation using spherical wavelets. IEEE Transactions on Medical Imaging 2007;26(4), pages 598-618.