team 5: size and shape comparison - ima.umn.edu€¦ · team 5: size and shape comparison author:...
TRANSCRIPT
Problem FormulationCoordinates-based Methods
Invariant Methods
Team 5: Size and Shape Comparison
Mentor: Mark A. Stuff
Team Members:Suman Balasubramanian1
Ying Wai (Daniel) Fan2
Yuen Yick Kwan3
Josef Aaron Sifuentes4
Jinglong Ye1
Weifeng Zhi5
1Mississippi State U., 2Emory U., 3Purdue U., 4Rice U., 5U. Kentucky
Team 5: Size and Shape Comparison
Problem FormulationCoordinates-based Methods
Invariant Methods
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
2 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Problem Motivation
I The availability, usage and mobility of sensor components inrecording objects of interest enable generation of many formsof three dimensional data.
I In order to find the correspondence between the threedimensional data sets when no prior knowledge is available,new methods of estimating the transformation from a set ofunlabeled landmarks are required.
3 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Optimization Problem
Given two sets M and N with |M| < |N|, find S , Π, R and T suchthat
‖M − SΠNR − T‖F
is minimized, where
S = selection matrix,
Π = permutation matrix,
R = rotation matrix,
T = translation.
4 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
5 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
6 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Rigid Procrustes Analysis
Suppose |M| = |N| = n and no permutation, then we have tominimize
minR,T
‖N −MR − T‖F . (1)
The result from Procrustes analysis gives
R = UV T , (2)
where N̂T M̂ = VΛUT with M̂ and N̂ are centered M and N,U,V ∈ SO(3) and Λ = diag(λ1, λ2, . . . , λn) withλ1 ≥ λ2 ≥ · · ·λn−1 ≥ |λn|;
T = cN − cMR, (3)
where cM and cN are the centroids of points in M and Nrespectively.
7 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
8 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Iterative Closest Point (ICP)
I Iteratively estimates the rigid motion between M and N.I Steps:
0 Find initial guesses of rigid motion:I Find a large tetrahedron in M and some tetrahedrons in N
that look like this tetrahedron.I The initial guesses are the rigid motions between the
tetrahedrons.
1 Associate points in M to points in N using closest neighbor.2 Recompute the rigid motion (e.g. Procrustes).3 Iterate (go back to 1) until convergence.
Source: Rusinkiewicz and Levoy “Efficient Variants of the ICP Algorithm”, Third International Conference on 3-D
Digital Imaging and Modeling, 2001
9 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Rigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
ICP Results
Left: Original cow.Middle: Perturbed cow (blue) and the recovered cow (red) withσ = 0.01.Right: Perturbed cow (blue) and the recovered cow (red) withσ = 0.01.
10 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
11 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
12 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Examples of geometric invariants
I Sets of distances from the centroid to each vertex: n
invariants
I Set of all pairwise distances between the points,(n2
)invariants.
I M. Boutin, G. Kemper,On reconstructing n-pointconfigurations from the distribution of distances or areas,Advances in Applied Mathematics,2(2004)709− 735.
I Set of areas of all triangles:(n3
)invariants.
I · · ·
13 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
14 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Inner Products Invariants
Compile the two sets of n points {mi}ni=1 and {ni}n
i=1 intomatrices
M = [m1 m2 · · · mn]T ∈ Rn×3
N = [n1 n2 · · · nn]T ∈ Rn×3
Then if we build the matrices of inner product relations
FM = MMT
FN = NNT
The matrices are the same under permutation.Note also that these matrices are rank 3.
15 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Spectral Comparisons of Inner Product Matrices
Thus if Π is the permutation matrix that corresponds like pointsunder the rigid motion
ΠFMΠT = FN ,
which indicates that FM and FN are similar. Thus
σ(FM) = σ(FN)
We also obtain that σ(MTM) = σ(NTN)Then we construct a score function SI for similarity
SI =3∑
i=1
|λNi − λM
i |2
This gives a way to compare likeness of two sets of points under arigid motion and permutation without knowing what the rigidmotion or permutation is.
16 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Comparisons of Distance Matrices
Similarly if we build the matrix of distance relations
[DM ]ij = ‖mi −mj‖[DN ]ij = ‖ni − nj‖
As in the case of the inner product matrices, the two distancematrices are the same under permutation. Therefore
σ(DM) = σ(DN)
We construct another score function SD for similarity
SD =1
n
n∑i=1
|λNi − λM
i |2
Thus we have another way of comparing likeness of two sets ofpoints under a rigid motion and permutation without knowingwhat the rigid motion or permutation is.
17 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Subset selection by eigenvalue comparison.
I Method requires searching(nm
)possible subsets.
I Matching eigenvalues does not guarantee isometric subsets.
18 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Lyapunov Equations
Thus we can solve for the permutation by solving
ΠDM −DNΠ = 0
These are known as the Lyapunov Equations (also called theSylvester Equations). We can solve these by vectorizing the aboveequations
vec(ΠDM)− vec(DNΠ) = 0
⇐⇒ [(DM ⊗ I)− (I⊗DN)] ~Π = 0
where ~Π is the vectorized permutation matrix.
19 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Solving Lyapunov Equations
This implies that
~Π ∈ N ( (DM ⊗ I)− (I⊗DN) )
I Let (λMi , vM
i )ni=1 be the eigenpairs of DM
I and (λNi , vN
i )ni=1 be the eigenpairs of DN
I then ( (λMi − λN
j ), (vMi ⊗ vN
j ) )ni ,j=1 are eigenpairs of( (DM ⊗ I)− (I⊗DN) ).
Then if M and N are the same under a rigid motion
~Π ∈ span( (vM1 ⊗ vN
1 ), (vM2 ⊗ vN
2 ), . . . , (vMn ⊗ vN
n ) ).
20 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Solving Lyapunov Equations
Let V be the matrix whose vectors are the above basis of our nullspace, then there exists α ∈ Rn s.t.
Vα = ~Π
We add the constraints that the row sum and column sum of Π isone by enforcing the constraints
(eT ⊗ I)~Π = e
(I⊗ eT )~Π = e
where e is the vector of all ones in Rn.
21 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Solving Lyapunov Equations-two strategies
One strategy is [(eT ⊗ I)~Π
(I⊗ eT )~Π
]Vα =
[ee
]I Cheaper in terms of computer memoryI More susceptible to noise in the data.
Our second strategy is DM ⊗ I− I⊗DN
ω(eT ⊗ I)~Π
ω(I⊗ eT )~Π
~Π = ω
0ee
where ω is a weight we use to enforce the column and row sumconditions more strongly than the null space condition.
I More expensive in terms of computer memoryI More robust to noisy data.
22 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Examples
Figure: Given two sets of cows in R3, where the blue cow is a rigidmotion and permutation of the red cow.
23 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Examples
Figure: By solving for the permutation, we can use Procrustes to obtainthe optimal rigid motion to line up the cows.
24 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Examples
Figure: By using the same algorithm we get robust results for noise withstandard deviation of σ = 0.01
25 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Examples
Figure: By using the same algorithm we get robust results for noise withstandard deviation of σ = 0.1
26 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
27 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
I Objective: to find the points in N that are also in MI without computing the rigid motion and permutationI without trying all n!
m!(n−m)!) m-subsets of N.
I We denote the distance between ni and nj by di ,j .
I Suppose DN and DM are the set of pairwise distances ofpoints in N and M respectively.
I Fix some ordering in DN , say
DN ={
d1, d2, . . . , dl , . . . , d n(n−1)2
}= {d1,2, d1,3, . . . , di ,j , . . . , dn−1,n} , 1 ≤ i < j ≤ n.
28 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
I Define the edge selection vector s as
sl =
{1, if dl ∈ DM
0, otherwise(4)
I Let f1, . . . , fp be some linearly independent functions. Formthe function value matrix F with entries
Fk,l = fk(dl). (5)
I Then if b = F s, then
bk =
n(n−1)2∑
l=1
Fk,lsl =
n(n−1)2∑
l=1
fk(dl)sl =∑
dl∈DM
fk(dl) (6)
29 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
I Therefore, given N and M, we can compute the DN and DM
and from which we can compute F and b by (5) and (6) andthen solve for s in
F s = b (7)
to know the membership of dl ’s in DM and thus also thewhich ni ’s is in M.
30 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Our choice of fk ’s
I In our experiments, we use the family of cosine functions asfk ’s:
fk(d) = cos(kπd).
Entries of F are bounded between −1 and 1.
I The number of rows, p, of F has to be at least n(n−1)2 . To
make our linear system better conditioned, we may set p to belarger than n(n−1)
2 and instead solve the least square problem:
mins‖b− F s‖2
F (8)
31 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
32 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1
I Construct the edge matrix E :
El ,j =
{1, if nj is an endpoint of the edge dl
0, otherwise
I The point selection vector x be defined as
xj =
{1, if nj ∈ M
−1, otherwise,
I Then given M and N, we can compute F , E and b and thensolve for x in
FEx = F (2s) = 2F s = 2b. (9)
to know which point in N is in M.
33 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
I Note that FE is of size p by n while F is of size p by n(n−1)2 ,
so (9) is easier to solve than (7) and it saves the step offinding points connecting to edges selected by s.
Figure: F matrix Figure: FE matrix
34 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Outline
Problem Formulation
Coordinates-based MethodsRigid Procrustes AnalysisIterative Closest Point (ICP) Algorithm
Invariant MethodsComparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m = n − 1When m < n − 1
35 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
When m < n − 1
I When M is missing more than one point of N, (9) is only anapproximation, i.e.
FEx ≈ 2b, (10)
I We do not obtain from solving (9) a x with entries -1 and 1,but some perturbed version of it.
I However, from our experiments, when we choose the numberof rows, p, in F and b large enough, say p = n2, theperturbation in x is not large enough to change the signs.
I So after taking the sign operation to the computed x, we stillget a vector of entries -1 and 1.
36 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Subset Algorithm Pictures
Figure: We are given two clouds of random points, where the blue cloudis a subset of the red cloud under a permutation and rigid motion. Aboveare the respective distance matrices.
37 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Subset Algorithm Pictures
Figure: By using our subset selection algorithm, we can pick which redpoints correspond to the blue points under a rigid motion andpermutation. By taking out the appropriate rows and columns of ourdistance matrix associated to the red points, we get two equal matricesunder permutation. 38 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Subset Algorithm Pictures
Figure: By using our permutation selection algorithm, we can determinethe point correspondence in order to apply Procrustes. Notice that nowthe distance matrices are the same, as they are invariant under a rigidmotion.
39 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Subset Algorithm Pictures
Figure: Applying Procrustes now gives us the correct rigid motion tocorrespond our two clouds of points.
40 / 41
Problem FormulationCoordinates-based Methods
Invariant Methods
Comparing Lists of Geometric InvariantsComparing Matrices of Geometric InvariantsSubset Selection Function Estimation
Thank You for Your Attention and Hospitality.Any Questions?
41 / 41