1Challenge the future

Chaotic Invariants for Human Action Recognition

Ali, Basharat, & Shah, ICCV 2007

2Challenge the future

Premise: Moving reference joints carry information about human

actions

3Challenge the future

Assumption: Human actions are generated by a nonlinear dynamical

system• Dynamical: the system’s behaviour changes over time

• Nonlinear: the rule(s) describing this change cannot be written as a linear functionHow to capture the nonlinear physics of

human actions?

4Challenge the future

Assumption: Human actions are generated by a nonlinear dynamical

system• Movement trajectories of reference joints only provide a low-dimensional observation of the human action system

• But, they still carry information about the entire (nonlinear) system

5Challenge the future

Approach: Chaotic Invariants

1. Reconstruct the dynamical behaviour of the human action system based on movement trajectories of reference joints• delay-embedding theorem (Takens, 1981)

2. Characterize this reconstructed dynamical behaviour with chaotic invariants

3. Action recognition based on chaotic invariants

6Challenge the future

Example of application of delay-embedding theorem:

• Lorenz system:

7Challenge the future

Example of application of delay-embedding theorem:

• Lorenz system:

• plotting x, x - delay, x -2*delay

strange attractor:

8Challenge the future

Appl. of delay-embedding theorem to movements of reference joints:

9Challenge the future

Characterisation of strange attractor with chaotic invariants

• Maximum lyapunov exponent:Quantifies the divergence of the strange attractor

• Correlation integral: Quantifies the density of points in the phase space (using a threshold for nearby points)

• Correlation dimension: Quantifies the sensitivity of the correlation integral for the applied threshold

10Challenge the future

9 Different Actions

11Challenge the future

3 time series (x, y, and z) for 5 reference joints

12Challenge the future

Results of activity classification using chaotic invariants: