an optimal probabilistic graphical model for point set matching

Outline

An optimal probabilistic graphical model for pointset matching

Tiberio Caetano1,2, Terry Caelli1 and Dante Barone2

1Department of Computing ScienceUniversity of Alberta, Canada

2Instituto de InformaticaUFRGS, Brazil

SSPR 2004

Tiberio Caetano, Terry Caelli and Dante Barone An optimal probabilistic graphical model for point set matching

Outline

1 The ProblemProblem DefinitionProperties of the FormulationComputational Complexity

2 The SolutionGraphical ModelsThe Problem as Inference in a Graphical ModelGraphical Model construction and inference

3 ExperimentsExperimental SetupInexact matching

The ProblemThe SolutionExperiments

Summary

Problem DefinitionProperties of the FormulationComputational Complexity

Outline

Summary

The Problem

f: D −> C

Template

Summary

The Problem

f: T −> S

Template

Summary

The Problem

f: T −> S

Template

Summary

The Problem as Weighted Graph Matching

The Euclidean Distance Matrix (EDM) of a point set uniquelydetermines the rigid conformation

As a result...

Conformations can be compared by comparing the EDMs

Summary

As a result...

Summary

As a result...

Summary

Problem Definition

Problem

Given two point sets T = {di , i = 1, . . . ,T} andS = {cj , j = 1, . . . ,S} in Rn (n ∈ N+), find the function f : T → Sthat maximizes

P(f ) =∑i ,j∈T

S(||di − dj ||, ||cf (di ) − cf (dj )||),

where S(·, ·) is a similarity function and || · || is the L2 metric

Summary

Properties

Handles only invariance to isometries (rotations, translations,reflexions)

f can be any function

OK NOOK

Summary

Computational Complexity

Complexity

There are ST possible mapping functions f

Brute force solution: test each and compute score P(f )

Exponential complexity

Question

How to solve it?

Summary

Complexity

There are ST possible mapping functions f

Brute force solution: test each and compute score P(f )

Exponential complexity

Question

How to solve it?

Summary

Graphical ModelsThe Problem as Inference in a Graphical ModelGraphical Model construction and inference

Outline

Summary

Graphical Models

Graphical Models are stochastic processes defined over graphs

Nodes of the graph are random variables

Edges of the graph are probabilistic dependencies betweenrandom variables

Summary

Types of Graphical Models

Undirected Graphical Models (= Markov Random Fields)

Symmetric probabilistic relations (undirected edges)

Summary

Types of Graphical Models

Directed Graphical Models (= Bayesian Networks)

Possibly Non-Symmetric probabilistic relations (directededges)

Summary

Features of a Graphical Model

Features

The probabilistic dependencies between neighbor nodes(“potential functions”)

The connectivity of the graph

Summary

Undirected Graphical Models

Undirected

We consider exclusively undirected Graphical Models

Interest

The Interest is to compute the MAP estimate for the model

Summary

Undirected Graphical Models

Undirected

We consider exclusively undirected Graphical Models

Interest

The Interest is to compute the MAP estimate for the model

Summary

Hammersley-Clifford Theorem

HC Theorem

States that the joint distribution of a Graphical Model is factorizedover products of functions over maximal cliques of the graph:

p(x) =∏C∈C

ψC (xC )

Whose maximization is equivalent to minimize

U(x) = −∑C∈C

logψC (xC )

If we choose only pairwise cliques, then a suitable definition of theψC s leads to the original formulation of the problem as a weightedgraph matching one

Summary

Our Formulation

Template points are random variables

Scene points are realizations

Modeling domain X j= xlX i= xk

Problem domain

Thus...

The best mapping becomes the MAP solution of the model.

Summary

Our Formulation

Template points are random variables

Scene points are realizations

Modeling domain X j= xlX i= xk

Problem domain

Thus...

The best mapping becomes the MAP solution of the model.

Summary

Constructing the model

Recall that to construct a Graphical Model one needs

A set of pairwise potential functions

A connectivity pattern

Summary

Potential Functions

We use pairwise potential functions

A potential function measures how good is a pairwise map

HIGH Potential

LOW Potential

Summary

Similarity Functions

Possible Similarity Functions

A potential function ψ is build from a similarity function Slike the following ones:

GaussianHyperbolic Tanget

G(dij , dkl) = exp

(− 1

2σ2|dij − dkl |2

)H(dij , dkl) = 1−tanh

[|dij − dkl |

]Tiberio Caetano, Terry Caelli and Dante Barone An optimal probabilistic graphical model for point set matching

Summary

Potential Functions

Each connected pair i , j can map to S2 possible pairs:

ψij(Xi ,Xj) =1

S(Xi = x1,Xj = x1) . . . S(Xi = x1,Xj = xS)...

. . ....

S(Xi = xS ,Xj = x1) . . . S(Xi = xS ,Xj = xS)

i X jX

Summary

Connectivity

What is an appropriate connectivity for the graphical model?

We can’t use the fully connected graph because exactinference has exponential complexity

It is possible to prove that a particular sparse graph whereexact inference is doable in polynomial time is equivalent tothe fully connected graph in the limit of exact matching

Summary

Connectivity

Summary

Connectivity

Summary

Some Geometry

Summary

Some Geometry

Summary

Some Geometry

Summary

Some Geometry

Summary

Some Geometry

Summary

Some Geometry

Determined

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Summary

Connectivity

Global Rigidity

The resulting graph is said to be globally rigid.

Summary

There is a Lemma:

This can be generalized to any dimension, n ∈ N+

Summary

Connectivity

k-tree

The resulting graph is technically a 3-tree, in the case of R2,and a k-tree in the case of Rk−1

A k-tree has a maximal clique size fixed in k+1

4−clique3−tree,

Summary

Connectivity

There is a Theorem:

It is possible to prove that a Graphical Model whose topology isgiven by a k-tree is equivalent to the fully connected model in thelimit of exact matching

Intuition:

The potential functions depend only on the relative distances.Since a k-tree is sufficient to encode all distances, it is alsosufficient to encode all potential functions

Summary

Connectivity

There is a Theorem:

It is possible to prove that a Graphical Model whose topology isgiven by a k-tree is equivalent to the fully connected model in thelimit of exact matching

Intuition:

The potential functions depend only on the relative distances.Since a k-tree is sufficient to encode all distances, it is alsosufficient to encode all potential functions

Summary

The Model

The final model looks like... (for R2)

4 5 TX

Optimality

It is optimal in the limit of exact matching because it isequivalent to the full model

Summary

The Model

The final model looks like... (for R2)

4 5 TX

Optimality

It is optimal in the limit of exact matching because it isequivalent to the full model

Summary

Junction Tree Famework

Junction Tree Framework

The Junction Tree Framework provides algorithms for exact MAPcomputation in graphical models, whose complexity is exponentialonly on the size of the maximal clique

Summary

Junction Tree Famework

Junction Tree Framework

The Junction Tree Framework provides algorithms for exact MAPcomputation in graphical models, whose complexity is exponentialonly on the size of the maximal clique

Summary

Optimization

Optimize using JT algorithm (Hugin)

1 Construct hypergraph (a “Junction Tree”) where nodes aremaximal cliques of original graph

2 “Pass messages” in a systematic way in this hypergraph, whatmeans updating the clique potentials according to specificrules

3 After messages have been passed, the nodes contain the exactMAP estimate for the model

Summary

Optimization

Optimize using JT algorithm (Hugin)

1 Construct hypergraph (a “Junction Tree”) where nodes aremaximal cliques of original graph

2 “Pass messages” in a systematic way in this hypergraph, whatmeans updating the clique potentials according to specificrules

3 After messages have been passed, the nodes contain the exactMAP estimate for the model

Summary

Optimization

3X 4X 3X1 X2 X 5X 3X1 X2

X 3X1 X2

X TX3X1 X2

X3X1 X2

XXX1 X2 T−1

Summary

Optimization

3X 4X 3X1 X2 X 5X 3X1 X2

X 3X1 X2

X TX3X1 X2

X3X1 X2

XXX1 X2 T−1

Summary

Optimization

3X 4X 3X1 X2 X 5X 3X1 X2

X 3X1 X2

X TX3X1 X2

X3X1 X2

XXX1 X2 T−1

Summary

Optimization

Messages

V → W:φ∗S = max

V \SψV

ψ∗W =φ∗SφSψW

Summary

Complexity

The overall complexity for matching tasks in Rk−1 is

O(TSk+1)

T → size of the template patternS → size of the scene pattern

k+1 → size of the maximal clique

Summary

Experimental SetupInexact matching

Outline

Summary

Experimental Setup

We compare this approach (JT) with standard ProbabilisticRelaxation Labeling (PRL)

Matching point sets in R2

Experiments with inexact matching (for exact matching italways yields perfect results)

Summary

Inexact matching: varying problem size

15 20 25 30 35 40 45 500.4

Number of points in the codomain pattern (S)

eT = 10, std = 2 pixels

Summary

Inexact matching: varying noise

1 2 3 4 5 6 7 8 90.2

1T = 10, S = 30

Position jitter (std, in pixels)

esJTPRL

Summary

PRL x JT

Locally optimal

Iterative

Globally optimal

Non-iterative (two-pass)

Summary

PRL x JT

Locally optimal

Iterative

Globally optimal

Summary

PRL x JT

Locally optimal

Iterative

Globally optimal

Summary

Point set matching is formulated as inference in a Graphical Model

Summary

Optimal inference is performed in polynomial time in a sparsemodel which is equivalent to the full model

Summary

The experimental results are perfect for exact matching andsignificantly better than PRL for inexact matching

Summary

Main Lesson

Redundancies

What makes search in this problem NP-hard is redundantinformation

It can be solved optimally in polynomial time if we properlytake advantage of this redundancy

Summary

Main Lesson

Redundancies

What makes search in this problem NP-hard is redundantinformation

It can be solved optimally in polynomial time if we properlytake advantage of this redundancy

Summary

To do...

Comparison

Compare with other techniques (spectral methods, continuousoptimization, etc.)

Invariance

Extend to more complex invariances

Bounds for error in inexact matching

Summary

To do...

Comparison

Invariance

Summary

To do...

Comparison

Invariance

Summary

To do...

Comparison

Invariance

Summary

Related Results

CVPR 2004

Attributed Graph Matching (n-sized cliques)

ICPR 2004

Different types of similarity functions

Tiberio’s Thesis, 2004

The theoretical development, proofs and numerous experimentalresults are available

Summary

Related Results

CVPR 2004

ICPR 2004

Summary

Related Results

CVPR 2004

ICPR 2004

Summary

Related Results

CVPR 2004

ICPR 2004

Summary

More Info

More information (papers and Thesis)

www.cs.ualberta.ca/˜tcaetano

an optimal probabilistic graphical model for point set matching

Documents