real time motion capture using a single time-of-flight camera varun ganapathi, christian plagemann,...

17
Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123 邱邱邱

Upload: jordan-houston

Post on 17-Dec-2015

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Real Time Motion CaptureUsing a Single Time-Of-Flight Camera

Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun

CVPR 2010

Q36981123 邱碁森

Page 2: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Outline

• Introduction• Probabilistic Model• Inference• Experiments• Conclusions

Page 3: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Introduction

• Motion capture is used to human-machine interaction, smart surveillance and so on.

• Time-of-flight sensors offers rich sensory information, not sensitive to changes in lighting, shadows, and some other problems.

• This paper propose an efficient filtering algorithm for tracking human pose for fast operation at video frame.

Page 4: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

What is Probabilistic Model?

• A tree-shaped kinematic chain (skeleton)– Human body is modeled as 15 body parts– The transformations of the body Xt at time t is a

set: Xt= {Xi}, i = 1~15

– X1: the root of tree → the pelvis part• root(pelvis): could freely rotate and translate • other parts: connected to the their parent,

allow to rotate (not to translate)

Page 5: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

What is Probabilistic Model? (cont.)

• The absolute orientation of a body part i: Wi(X)– multiplying the transformations of its ancestors in

the kinematic chain– Wi(X) = X1 X∙ 2 · ...· Xparent(i) X∙ i

Page 6: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Why need the Probabilistic Model?

• Determine the most likely state at at time t– the pose set Xt

– the first discrete-time derivative set Vt (velocities)

– zt: the recorded range measurements

• The system is modeled as a dynamic Bayesian network (DBN)

Page 7: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Probabilistic Model

• The measured range scan is denoted by z = {zk} k=1

M • where zk gives the measured depth of the

pixel at coordinate k.

Page 8: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Probabilistic Model

• Assumption: the accelerations in our system are drawn from a Gaussian distribution with zero mean

Page 9: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Inference

• How to perform efficient inference at each frame?– Model Based Hill Climbing Search (HC)• A component locally optimizes the likelihood function

– Evidence Propagation (EP)• An inference procedure generate likely states which are

used to initialize the HC

Page 10: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Model Based Hill Climbing Search

• coarse-to-fine– The procedure can then potentially be applied to a smaller

interval about the value chosen at the coarser level

• hill-climbing– Start from the base of kinematic chain which includes the

largest body parts, and proceed toward the limbs

1

23

optimize the X axis

0.50.450.4...-0.35-0.4-0.45-0.5

sample:

then chose the best one

Page 11: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Evidence Propagation

• Problem: – fast motion cause motion blur– occlusion cause the estimate of the state of

hidden parts to drift– the likelihood function has ridges (difficult to

navigate)• This procedure that identifies promising

locations for body parts to find likely poses

Page 12: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Evidence Propagation

• Steps in this procedure:1. Body Part Detection: identify possible body part

locations from the current range image2. Probabilistic Inverse Kinematics: update the body

configuration X given possible correspondences between mesh vertices and part detections

3. Data Association and Inference: determine the best subset of such correspondences

Page 13: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Body Part Detection

• Five body parts: head, left hand, right hand, left foot and right foot are found from the current range image.

• Interest Point(AGEX) Detection – start on the geodesic centroid of the mesh: AGEX1(M)

– recursively find the vertex AGEXk(M) which has max geodesic distance to AGEXk-1(M)

• Identification of Parts– points are classified as body part by training these data using a

marker-based motion capture system( LED mark)

Page 14: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Evidence Propagation

Page 15: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Experiments

• Using a Swissranger SR4000 Time-of-Flight camera

• Tracking results on real-world test sequences, sorted from most complex (left) to least complex (right).

Page 16: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Experiments

• A Tennis sequence

Only use Model-Based search

Our combined tracker

Page 17: Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123

Conclusions

• A novel algorithm for combining part detections with local hill-climbing for marker less tracking of human pose.

• With the hybrid, GPU-accelerated filtering approach