01/18 Lab meeting

UCLA Vision Lab

Department of Computer Science

University of California at Los Angeles

Fabio Cuzzolin

Los Angeles, January 18 2005

… past and present PhD student, University of Padova, Department of Computer

Science (NAVLAB laboratory) with Ruggero Frezza

Visiting student, ESSRL, Washington University in St. Louis

Visiting student, UCLA, Los Angeles (VisionLab)

Post-doc in Padova, Control and Systems Theory group

Young researcher, Image and Sound Processing Group,

Politecnico di Milano

Post-doc, UCLA Vision Lab

3

… the research

research

Computer vision object and body tracking

data association

gesture and action recognition

Discrete mathematics

linear independence on lattices

Belief functions and imprecise probabilities

geometric approach

algebraic analysis

combinatorial analysis

1Upper and lower probabilities

5

Past work

Geometric approach to belief functions (ISIPTA’01, SMC-C-05)

Algebra of families of frames (RSS’00, ISIPTA’01, AMAI’03)

Geometry of Dempster’s rule (FSKD’02, SMC-B-04)

Geometry of upper probabilities (ISIPTA’03, SMC-B-05)

Simplicial complexes of fuzzy sets (IPMU’04)

The theory of belief functions

7

Uncertainty descriptions

A number of theories have been proposed to extend or replace classical probability: possibilities, fuzzy sets, random sets, monotone capacities, etc.

theory of evidence (A. Dempster, G. Shafer)

belief functions Dempster’s rule families of frames

8

Motivations

9

Axioms and superadditivity

probabilities additivity: if then

1)(.2 p0)(.1 p

BA )()()( BpApBAp

belief functions 3. superadditivity

0)(.1 s 1)(.2 s

i ji

nn

jiin AAsAAsAsAAs )...()1(...)()()...( 11

1

10

Example of b.f.

11

belief functions s: 2Θ ->[0,1]

AB

BmAs )( A

Belief functions

B2

B1

1)( B

Bm

..where m is a mass function on 2Θ s.t.

12

Dempster’s rule

b.f. are combined through Dempster’s rule'', ssss

ABBmABel)()(

Ai

Bj

AiBj=A

intersection of focal elements

ji

ji

BAji

ABAji

BmAm

BmAm

Am)()(1

)()(

)(21

21

13

Example of combination

s1: m({a1})=0.7, m({a1 ,a2})=0.3

a1

a2

a3

a4

s2: m()=0.1, m({a2 ,a3 ,a4})=0.9

s1 s2 : m({a1})=0.19, m({a2})=0.73

m({a1 ,a2})=0.08

14

Bayes vs Dempster

Belief functions generalize the Bayesian formalism as:

1- discrete probabilities are a special class of belief functions

2 - Bayes’ rule is a special case of Dempster’s rule

3 - a multi-domain representation of the evidence is contemplated

15

My research

Theory of evidence

algebraic analysis geometric

analysis

categorial?

probabilistic analysis

combinatorial analysis

Algebra of frames

17

Family of frames

example: a function y [0,1] is quantized in three different ways

1

.0 .1

.00 .01 .10 .11

.0 .1 .2 .3 .4

0.00 0.09

0.90 0.99

0.49

0.25 0.750.50

refining

Common refinement

18

Lattice structure

minimal refinement

1F

maximal coarsening

F is a locally Birkhoff (semimodular with finite length) lattice bounded

below

order relation: existence of a refining

Geometric approach to upper and lower probabilisties

20

it has the shape of a simplex

),( APClS A

Belief space

the space of all the belief functions on a given frame

each subset A A-th coordinate s(A) in an Euclidean space

21

Geometry of Dempster’s rule

constant mass loci

foci of conditional subspaces

Dempster’s rule can be studied in the geometric setup too

Px

Py

P

F (s)x

F (s)y

y

x

y

x

22

),,( AACl A

the space of plausibilities is also a simplex

*sP

),(~*

yxs ppP

s

yyP

xxP

P

x

y

p

p,1

0,

x

y

p

p

S

Geometry of upper probs

23

PPs s *

Belief and probabilities

s

*sP

P)(sP

1xP 2xP

nxP

Ps

)(sP

}{ AX

study of the geometric interplay of belief and probability

24

}{

)()(

xA A

AmsP

Consistent probabilities

Each belief function is associated with a set of consistent probabilities, forming a simplex in the probabilistic subspace

the vertices of the simplex are the probabilities assigning the mass of each focal element of s to one of its points

the center of mass of P(s) coincides with Smets’ pignistic function

25

Possibilities in a geometric setup

possibility measures are a class of belief functions

they have the geometry of a simplicial complex

Combinatorial analysis

27

Total belief theorem

a-priori constraint

conditional constraint

generalization of the total probability theorem

28

Existence

candidate solution: linear system nn where the columns of A are the focal

elements of stot

bAx

problem: choosing n columns among m s.t. x has positive components

method: replacing columns through

i j

ji eeeee

29

Solution graphs

all the candidate solutions form a graph

Edges = linear transformations

30

New goals...

algebraic analysis

combinatorial analysis

Theory of evidence

geometric analysis

?

probabilistic analysis

31

Approximations

compositional criterion the approximation behaves like s when

combined through Dempster

tpCs

C dttstss minarg

problem: finding an approximation of s

probabilistic and fuzzy approximations

32

Indipendence and conflict

s1,…, sn are not always combinable

1,…, n are indipendent if

iinn AAA ,)(...)( 11

any s1,…, sn are combinable are defined on independent frames

33

Pseudo Gram-Schmidt

Vector spaces and frames are both semimodular lattices -> admit independence

mn

mn

ssss

FF

',...,',...,

',...,',...,

11

11

pseudo Gram-Schmidt

new set of b.f. surely combinable

34

Canonical decomposition unique decomposition of s into simple b.f.

nees ...1

convex geometry can be used to find it

xx PPs

yy PPs

s

xe

ye

35

Tracking of rigid bodies

m-1m

past and present target association

rigid motion constraints can be written as conditional belief functions total belief needed

Am-1

past targets - model

associations

m-1m

Am-1

= Am-1 m-1

m

Am-1 ()

old estimates

rigid motion constraints

Kalman filters

Am

current targets – model association

Am

new estimates

data association of points belonging to a rigid body

36

Total belief problem and combinatorics

relation with positive linear systems

homology of solution graphs

matroidal interpretation

general proof, number of solutions, symmetries of the graph

2Computer vision

38

Vision problems

HMM and size functions for gesture recognition (BMVC’97)

object tracking and pose estimation (MTNS’98,SPIE’99, MTNS’00, PAMI’04)

composition of HMMs (ASILOMAR’02)

data association with shape info (CDC’02, CDC’04, PAMI’05)

volumetric action recognition (ICIP’04,MMSP’04)

Size functions for gesture recognition

40

Size functions for gesture recognition Combination of HMMs (for dynamics) and

size functions (for pose representation)

41

Size functions “Topological” representation of contours

42

Measuring functions

Functions defined on the contour of the shape of interest

real image

measuring function

family of lines

43

Feature vectors a family of measuring functions is chosen

… the szfc are computed, and their means form the feature vector

44

Hidden Markov models

Finite-state model of gestures as sequences of a small number of poses

45

Four-state HMM

Gesture dynamics -> transition matrix A

Object poses -> state-output matrix C

46

EM algorithm

feature matrices: collection of feature vectors along time

EM A,C

learning the model’s parameters through EM

two instances of the same gesture

Compositional behavior of Hidden Markov models

48

Composition of HMMs Compositional behavior of HMMS: the

model of the action of interest is embedded in the overall model

Example: “fly” gesture in clutter

49

State clustering Effect of clustering on HMM topology

“Cluttered” model for the two overlapping motions

Reduced model for the “fly” gesture extracted through clustering

50

Kullback-Leibler comparison

We used the K-L distance to measure the similarity between models extracted from clutter and in absence of clutter

Model-free object pose estimation

52

Model-free pose estimation Pose estimation: inferring the configuration of a moving object from one or

more image sequences

Most approaches in the literature are model-based: they assume some knowledge about the nature of the body (articulated, deformable, etc) and some sort of model Qtq k ˆT=0 t=T

53

Model-free scenario Scenario: the configuration of an uknown object

is desired, given no a-priori information about the nature itself of the body

The only info available is carried by the images

We need to build a map from image measurements to body poses

This can be done by using a learning technique, based on training data

54

“Evidential” model

The evidential model is built during the training stage, when the feature-pose maps are learned

1

23

approximate feature spaces

training set of sample poses

feature-pose maps (refinings)

55

32

1

Feature extraction

From the blurred image the region with color similar to the region of interest is selected, and the bounding box is detected.

1

2

3

56

Estimates from the combined model

Ground truth versus estimates...

... for two components of the pose

JPDA with shape information for data association

58

JPDA with shape info

YX

Z

XY

Z

robustness: clutter does not meet shape constraints

occlusions: occluded targets can be estimated

JPDA model: independent targets

Shape model: rigid links

Dempster’s fusion

59

Triangle simulation

the clutter affects only the standard JPDA estimates

60

Body tracking

Application: tracking of feature points on a moving human body

Volumetric action recognition

62

Volumetric action recognition

problem: recognizing the action performed by a person viewed by a number of cameras

step 1: modeling the dynamics of the motion

step 2: extracting image features

2D approaches: features are extracted from single views -> viewpoint dependence

volumetric approach: features are extracted from a volumetric reconstruction of the moving body

63

Multiple sequences

synchronized views from different cameras, chromakeying

64

Volumetric intersection

silhouette extraction of the moving object from all views

3D object shape reconstruction through intersection of occlusion cones

more views -> more details

65

3D feature extraction

locations of torso, arms, and legs of the moving person

k-means clustering to

separate bodyparts

66

Feature matrices

two instances of the action “walking”

TORSO COORDINATES

ABDOMEN COORDINATES

RIGHT LEG COORDINATES

LEFT LEG COORDINATES

XY

Z

67

Modeling and recognition

• model of the “walking” action

…

classification: each new feature matrix is fed to all the learnt models, generating a set of likelihoods

HMM 1

HMM 2

HMM n

3Combinatorics

69

Independence on lattices

three distinct independence relations

modularity equivalent formulations

LI3

LI2=LI3

LI1

semimodular lattice modular lattice

LI2

LI1

70

Scheme of the proof

4Conclusions

72

from real to abstract

OBJECT TRACKING DATA ASSOCIATION

MEASUREMENTCONFLICT

POINTWISE ESTIMATE

CONDITIONALCONSTRAINTS

ALGEBRAICANALYSIS

GEOMETRICAPPROACH TOTAL BELIEF

the solution of real problems stimulates new theoretical issues

73

…concluding

the ToE comes from a strong critics to the Bayesian framework

useful for sensor fusion problems under incomplete information

real problem solutions stimulate the extension of the formalism

complex objects mathematically rich Young theory need completion

74

In the near future..

search for a metric on the space of dynamical systems – stochastic models

sistematic description of the geometric approach to non-additive measures

understand the intricate relations between probability and combinatorics