1Numerical geometry of non-rigid shapes Partial similarity

Partial similarity

© Alexander & Michael Bronstein, 2006-2009© Michael Bronstein, 2010tosca.cs.technion.ac.il/book

048921 Advanced topics in visionProcessing and Analysis of Geometric Shapes

EE Technion, Spring 2010

2Numerical geometry of non-rigid shapes Partial similarity

Greek mythology

3Numerical geometry of non-rigid shapes Partial similarity

I am a centaur.Am I human? Am I equine?Yes, I’m partially human.

Yes, I’m partially equine.

Partial similarity

Partial similarity is anon-metric relation

4Numerical geometry of non-rigid shapes Partial similarity

Human vision example

?

5Numerical geometry of non-rigid shapes Partial similarity

Visual agnosia

Oliver Sacks

The man who mistook his wife for a hat

6Numerical geometry of non-rigid shapes Partial similarity

Recognition by parts

Divide the shapes into parts

Compare each part separately using a full similarity criterion

Merge the part similarities

7Numerical geometry of non-rigid shapes Partial similarity

Partial similarity

X and Y are partially similar = X and Y havesignificantsimilar parts

Illustration: Herluf Bidstrup

8Numerical geometry of non-rigid shapes Partial similarity

Ingredients

Similar Dissimilar

Part = subset of the shape

Similarity = full similarity applied to parts

9Numerical geometry of non-rigid shapes Partial similarity

Significance

Significance = measure of the part

Area: size of the part

Significant Insignificant

10Numerical geometry of non-rigid shapes Partial similarity

Significance

Problem: how to select significant parts?

Significance of a part is a semantic definition

Different shapes may have different definition of significance

11Numerical geometry of non-rigid shapes Partial similarity

Significance

Do all parts have the same importance?

It really depends on the data

High Gaussian curvature

High variation of curvature

“features”

12Numerical geometry of non-rigid shapes Partial similarity

Statistical significance

A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009

syzygy in astronomy means alignment of three bodies of the solar system along a straight or nearly straight line. a planet is in syzygy with the earth and sun when it is in opposition or conjunction. the moon is in syzygy with the earth and sun when it is new or full.

syzygy in astronomy means alignment of three bodies of the solar system along a straight or nearly straight line. a planet is in syzygy with the earth and sun when it is in opposition or conjunction. the moon is in syzygy with the earth and sun when it is new or full.

in is

theor

Query q Database D

Frequent = important

Frequent = non-discriminative

syzygy

Rare = discriminativeSignificance of a term t

Term frequency Inverse documentfrequency

13Numerical geometry of non-rigid shapes Partial similarity

Numerical example

A. Bronstein, M. Bronstein, Y. Carmon, R. Kimmel, CVA 2009

14Numerical geometry of non-rigid shapes Partial similarity

Multicriterion optimization

Simultaneously minimize dissimilarity and insignificance over all the

possible pairs of parts

This type of problems is called multicriterion optimization

Vector objective function

How to solve a multicriterion optimization problem?

15Numerical geometry of non-rigid shapes Partial similarity

Pareto optimality

Traditional optimization Multicriterion optimization

A solution is said to be a global

optimum of an optimization problem

if there is no other such that

A solution is said to be a Pareto

optimum of a multicriterion

optimization problem

if there is no other such that

Optimum is a solution such that there is no other better solution

In multicriterion case, better = all the criteria are better

16Numerical geometry of non-rigid shapes Partial similarity

Pareto frontier

Vilfredo Pareto(1848-1923)

INSIGNIFICANCE

DIS

SIM

ILA

RIT

Y

UTOPIA

Attainable set

Pareto frontier

17Numerical geometry of non-rigid shapes Partial similarity

Set-valued partial similarity

The entire Pareto frontier is a set-valued distance

The dissimilarity at coincides with full similarity

18Numerical geometry of non-rigid shapes Partial similarity

Scalar- vs. set-valued similarity

INSIGNIFICANCE

DIS

SIM

ILA

RIT

Y

Use intrinsic

similarity as

19Numerical geometry of non-rigid shapes Partial similarity

Is a crocodile green or ?

What is better: (1,1) or (0.5,0.5)?

What is better: (1,0.5) or (0.5,1)?

Order relations

20Numerical geometry of non-rigid shapes Partial similarity

Order relations

There is no total order relation between vectors

As a result, two Pareto frontiers can be non-commeasurable

INSIGNIFICANCE

DIS

SIM

ILA

RIT

Y

BLUE < RED

GREEN ? RED

21Numerical geometry of non-rigid shapes Partial similarity

Scalar-valued partial similarity

In order to compare set-valued partial similarities, they should be scalarized

Fix a value of insignificance

INSIGNIFICANCE

DIS

SIM

ILA

RIT

YCan be written as

using Lagrange multiplier

Define scalar-valued partial similarity

22Numerical geometry of non-rigid shapes Partial similarity

Scalar-valued partial similarity

INSIGNIFICANCE

DIS

SIM

ILA

RIT

Y

Fix a value of dissimilarity

Distance from utopia point

Area under the curve

23Numerical geometry of non-rigid shapes Partial similarity

Characteristic functions

Problem: optimization over all

possible parts

A part is a subset of the shape

Can be represented using a

characteristic function

Still a problem: optimization over binary-valued

variables (combinatorial optimization)

24Numerical geometry of non-rigid shapes Partial similarity

Fuzzy sets

Relax the values of the

characteristic function to the

continuous range

The value of is the

membership of the point in the

subset

is called a fuzzy set

25Numerical geometry of non-rigid shapes Partial similarity

Fuzzy partial similarity

Compute partial similarity using fuzzy parts

Fuzzy insignificance

Fuzzy dissimilarity

where is the weighted distortion

26Numerical geometry of non-rigid shapes Partial similarity

Numerical optimization

GMDS problem:

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

27Numerical geometry of non-rigid shapes Partial similarity

Numerical optimization

Freeze parts and solve for correspondence

Freeze correspondence and solve for parts

1

2

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

28Numerical geometry of non-rigid shapes Partial similarity

Example of intrinsic partial similarity

Partial similarityFull similarity Scalar

partial similarity

Human

Human-like

Horse-like

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

29Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

How to make ICP compare partially similar shapes?

Introduce weights into the shape-to-shape distance to reject points with bad

correspondence

Possible selection of weights:

where is a threshold on normals

is a threshold on distance

and are the normals to shapes and

30Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

Without rejection With rejection

31Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

Parts obtained by matching of man and centaur using ICP with rejection for different thresholds

Problem: there is no explicit influence of the rejection thresholds and the

size of the resulting parts

Pareto framework allows to control the size of the selected parts!

32Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

Correspondence stage (fixed and )

Closest point correspondence

Weighted rigid alignment

Correspondence

33Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

Part selection stage (fixed and )

34Numerical geometry of non-rigid shapes Partial similarity

Extrinsic partial similarity

Controllable part size by changing the value of

35Numerical geometry of non-rigid shapes Partial similarity

Not only size matters

What is better?...

Many small parts……or one large part?

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

36Numerical geometry of non-rigid shapes Partial similarity

Regularity

Irregular = long boundary Regular = short boundary

Shape factor (circularity)

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

37Numerical geometry of non-rigid shapes Partial similarity

Regularity

Irregular = four connected components

Regular = one connected component

Equal shape factor!

Topological regularity

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

38Numerical geometry of non-rigid shapes Partial similarity

Mumford-Shah functional

Salvation comes from image segmentation

[Mumford&Shah]: given image , find the segmented region

replace by a membership function

The two problems are equivalent

39Numerical geometry of non-rigid shapes Partial similarity

Mumford-Shah functional

In our problem, we need only the integral along the boundary

which becomes in the fuzzy setting

40Numerical geometry of non-rigid shapes Partial similarity

Numerical example

INSIGNIFICANCE

DISSIMILARITY

IRREGULARITY

A. Bronstein, M. Bronstein, A. Bruckstein, R. Kimmel, IJCV 2008; A. Bronstein, M. Bronstein, NORDIA 2008

41Numerical geometry of non-rigid shapes Partial similarity

ICP exampleSignificance

Re

gul

arit

y