Chi Li, Le Lu, Gregory D. Hager, Hanzi Wang

2013/1/13 Johns Hopkins University

Robust Object Tracking in Crowd Dynamic Scenes using Explicit Stereo Depth

The Challenges of Object Trackingin Dynamic Scenes

Drifting Limitation of the appearance model

Sharp & Irregular Model Change Motion Field Appearance Feature Space

Partial and Complete Occlusion Occlusion Detection (Hard!) Object Reacquisition

2013/1/13 Johns Hopkins University

What if depth as the main cue?

Intuition: Powerful for background subtraction

Stable under sharp or irregular model change

Reliable indicator of occlusion detection

2013/1/13 Johns Hopkins University

Previous Work using Depth

Depth-assisted Tracking Ground plane & Odometry Depth-assisted detection

Multi-view Tracking Cannot be applied in dynamic scene

Kinect-based Human Tracking Limited Depth-of-Field Hard to be extended to arbitrary object

2013/1/13 Johns Hopkins University

Depth-driven Tracking Framework

2013/1/13 Johns Hopkins University

Depth Estimation

Color Clustering

Superpixel Classification Dominant

Depth Group

SVM Shape Filter+

Space-Color Histogram

Foreground

Occlusion Handling

Under Occlusion？Occlusion Detection

Reacquisition OB Tracking

no

yesReset Tracker

Next Fram

e

no

noyesyes

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

2013/1/13 Johns Hopkins University

Depth pixel clustering in (X Y D) space

Meanshift Clustering

Dominant Depth Group

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

Dominant Depth Blob Segmentation

Superpixel Classification: If more than 60% of pixels in on superpixel belong

to the dominant depth group, we classify this superpixel into the dominant depth blob.

2013/1/13 Johns Hopkins University

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

2013/1/13 Johns Hopkins University

Dominant depth blob splitting based on vertical shape distribution

……

Projection on X axis

Object Hypotheses

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

2013/1/13 Johns Hopkins University

SVM vertical shape filter

Object Hypotheses

……

Blobs After filtering

……

Vertical Feature Extraction

SVM Vertical Shape Filter

Single Object Blobs

GaussianFiltering

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

2013/1/13 Johns Hopkins University

SVM shape filter training Positive Sample: Manually labeled foreground mask Negative Sample: Manually labeled + noisy mask

from depth segmentation including 2, 3 and background + object.

After morphological filtering, we project pixels only above the centroid of the depth blob.

Apply interpolation and median filter to adjust the dimension of vertical feature to the same length

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

2013/1/13 Johns Hopkins University

Choose candidate based on color distribution

Space-color Histogram

Similarity Score

1 N

. . . . . .

Concatenation of the histograms of different regions

Depth Pixel Clustering

Depth BlobSegmentation

Object Hypotheses Generation

SVM Vertical Shape Filter

Space-Color Histogram

Occlusion Handling

Occlusion Detection Object Reacquisition

2013/1/13 Johns Hopkins University

Experiment1——Comparison

2013/1/13 Johns Hopkins University

Tracking Error

Comparison Example

2013/1/13 Johns Hopkins University

Tracking Error CurvatureExperiment1——Comparison

Experiment2——Depth Vs. Appearance

2013/1/13 Johns Hopkins University

Experiment3——Figure Recovery

2013/1/13 Johns Hopkins University

Experiment3——Figure Recovery

2013/1/13 Johns Hopkins University

Experiment3——Figure Recovery

2013/1/13 Johns Hopkins University

2013/1/13 Johns Hopkins University

Experiment4—Successive Occlusion Handling

253# 258#

266#

271#277#

2013/1/13 Johns Hopkins University

Experiment5——Failure Case1

2013/1/13 Johns Hopkins University

Experiment6——Failure Case2

Any Questions?

2013/1/13 Johns Hopkins University

Thank you for listening!

2013/1/13 Johns Hopkins University