bramble: the bayesian multiple-blob tracker by michael isard and john maccormick presented by...
TRANSCRIPT
![Page 1: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/1.jpg)
BraMBLe: The Bayesian Multiple-BLob Tracker
By Michael Isard and John MacCormick
Presented by Kristin Branson
CSE 252C, Fall 2003
![Page 2: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/2.jpg)
Problem The goal is to track an unknown number of
blobs from static camera video.
![Page 3: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/3.jpg)
Solution The Bayesian Multiple-BLob (BraMBLe)
tracker is a Bayesian solution. It estimates
),...,,|p( 11 ZZZX ttt
State at frame t Image Sequence
Number,Positions,Shapes,Velocities,…
![Page 4: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/4.jpg)
Bayes Rule
p(p( )) p)| (| Z XX XZ
Posterior State Distribution Observation
Likelihood
Prior
![Page 5: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/5.jpg)
Sequential Bayes
Posterior State Distribution Observation
Likelihood
Prior
Instead of modeling directly, BraMBLe models and .
)|p( :1 tt IX)|p( tt XI )|p( 1tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
![Page 6: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/6.jpg)
)|p( 1:11 tt ZX
Update Algorithm
)|p( :1 tt ZX
)|p( :1 tt ZX
)|p( 1:1 tt ZX
)|p( 1tt xx
)|p( tt XZ
)|p( :11 tt ZX
)|p( 11 tt XZ)|p( 1 tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
![Page 7: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/7.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 8: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/8.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 9: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/9.jpg)
Image Observations We want to choose our observations so
that we can compute quickly:
Z
)|p( XZ
g gz )|p()|p( XXZ
Individual observations are conditionally independent.
![Page 10: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/10.jpg)
Image Observations We want . A bank of filters is applied at each grid point.
Z
g gz )|p()|p( XXZ
Y
Cr
Cb
Filter plots from http://www.cs.jhu.edu/~wolff/course600.461/week3.2/sld012.htm
Mexican Hat
Gaussian
![Page 11: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/11.jpg)
Image Observations We want to choose our model so that we
can compute quickly:
Z
)|p( Xgz
g ggg g lzz )|p()|p()|p( XXZ
Observation depends on membership of grid point.
0gl
2gl1gl
3gl
![Page 12: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/12.jpg)
Image Observations We want to choose our model so that we
can compute quickly:
We can precompute and quickly evaluate any state x.
Z
)|p( Xgz
g ggg g lzz )|p()|p()|p( XXZ
ggg llz )|p(
![Page 13: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/13.jpg)
Appearance Models The appearance models are learned from
training data.
forek
foref
forefpfpf G
Klz ),(
1)0|p(
Training Data
backk backbackgk
backgkgg G
Klz ),(
1)0|p(
![Page 14: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/14.jpg)
Observation Likelihood
)0|p(
)0|p(log
gg
gg
lz
lz
![Page 15: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/15.jpg)
Observation Likelihood Review We defined our image observations
so that We defined our observation model
so that
We can precompute and quickly evaluate for many choices of x.
Z.)|p()|p(
g gz XXZ
)|p( Xgz
.)|p()|p()|p( g ggg g lzz XXZ
ggg lglz ,)|p( )|p( XZ
![Page 16: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/16.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 17: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/17.jpg)
Object Model The blob configuration is
mm xxX ...,,, 1Number of objects
ObjectState
1x
2x
3x
3m
X
![Page 18: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/18.jpg)
Object Model The blob configuration is
The object state is
svlx ,,,i
mm xxX ...,,, 1
IdentityLocation
VelocityShape
X
![Page 19: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/19.jpg)
Person Model
Generalized-Cylinder Model
CalibratedCamera
![Page 20: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/20.jpg)
Prediction Model The number of objects can change:
Each object has a constant probability of remaining in the scene.
There is a constant probability that a new object will enter the scene.
)|p( 1tt XX
r
i
In this formulation, hypotheses with different numbers of objects can be compared directly.
![Page 21: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/21.jpg)
Prediction Model Damped constant location velocity:
)|p( 1tt XX
11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx
1tx 1tx
1 10.8t tx x
![Page 22: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/22.jpg)
Prediction Model Damped constant location velocity:
Auto-regressive shape:
)|p( 1tt XX
11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx
12
1 1 10.2(p( | ) ( ,) )t t tt t sss sG s s
1ts
1 1 10.2( )t t ts s s
1ts
![Page 23: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/23.jpg)
Model Review
The observation likelihood is fast to compute for different hypotheses .
The prediction model allows generation of from
Estimating requires an efficient way ofRepresenting .Computing the multiplications and integrations.
tX
tX 1.t X
)|p( tt XZ
)|p( 1tt XX
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
)|p( :1 tt ZX
)|p( :1 tt ZX
![Page 24: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/24.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 25: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/25.jpg)
Efficient Representation is represented by a set of particles,
:
Sampling from the set using the weights approximates sampling from
1tX 2
tX 1Nt
X NtX...
N Points:
N Weights: 1t 2
t 1Nt N
t
)|p( :1 tt ZX
1:p( | ).t tX Z
{( , )}i it tX
![Page 26: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/26.jpg)
Efficient Representation is represented by a set of particles,
:
Sampling from the set using the weights approximates sampling from
)|p( :1 tt ZX
1:p( | ).t tX Z
{( , )}i it tX
![Page 27: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/27.jpg)
Efficient Computation The particle set representing is
computed from by CONDENSATION:
1. Apply dynamics to the particle set:
2. Multiply by the observation likelihood:
)|p( :1 tt ZX
1 1: 1p( | )t t X Z
: 1: 11 p( | p(p( | ) ) | )t tt tt t X ZX XZ Z
1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
![Page 28: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/28.jpg)
Applying Dynamics Given particle set , compute1 1: 1p( | )t t X Z
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.018.gif
1 1: 1p( | )t t X Z
1: 1p( | )t t X Z
)|p( 1tt XX
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
![Page 29: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/29.jpg)
Applying Dynamics Given particle set , compute
1. Resample into
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.020.gif
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
![Page 30: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/30.jpg)
Applying Dynamics Given particle set , compute
1. Resample into
2. Predict, generating to give 1p( | ' )i i
t t t X X X{( ,1/ )}.i
t NX
1{( ' ,1/ )}it NX
{( ,1/ )}it NX
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1{( ' ,1/ )}it NX
1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X
![Page 31: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/31.jpg)
Multiplication by Likelihood Given particle set , compute
Weight particles, setting
1: 1p( | )t t X Z
:1 1 1:p( | ) p( | )p( | .) t tt tt t Z X X ZX Z
1: 1p( | )t t X Z
)|p( tt XZ
)|p( :1 tt ZX
{( ,1/ )}it NX
{( , )}i it tX
p( | ).i it t t Z X
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.021.gif
![Page 32: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/32.jpg)
Efficient Computation Review The particle set representing is
computed from by CONDENSATION:
)|p( :1 tt ZX
1 1: 1p( | )t t X Z
1 1: 1p( | )t t X Z
1 1{( , )}i it t X
1 1: 1p( | )t t X Z
1{( ' ,1/ )}it NX {( ,1/ )}i
t NX
1: 1p( | )t t X Z )|p( :1 tt ZX
{( , )}i it tX
Resample Predict Reweight
Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.022.gif
![Page 33: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/33.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 34: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/34.jpg)
People Tracking
![Page 35: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/35.jpg)
People Tracking Tracking was successful in real time on
this 53s clip except when two people crossed in front of a third.
![Page 36: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/36.jpg)
Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.
)|p( tt XZ)|p( 1tt XX
)|p( :1 tt ZX
![Page 37: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/37.jpg)
Algorithm Summary The models chosen
Are a smooth integration of foreground and background models.
Allow hypotheses with differing numbers of objects to be compared directly.
Can be quickly evaluated in a particle filtering implementation.
![Page 38: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/38.jpg)
Relationship to Previous Work Static camera blob tracking:
Classifies pixels as foreground or background.
Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts
Video from http://image.pirl.umd.edu/knkim/research/BGS/compressed_video.htm
![Page 39: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/39.jpg)
Relationship to Previous Work Static camera blob tracking:
Classifies pixels as foreground or background. Static camera blob tracking:
Finds the position in the search area Made up of foreground pixels. Matching the blob in the previous frame.
Frame t - 1 Frame t Frame t + 1
Predicted position in frame t + 1
Search area
![Page 40: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/40.jpg)
Relationship to Previous Work Improvements over blob tracking:
Integrates the foreground and background modeling.
Multiple objects can be tracked through occlusions.
Video from http://robotics.stanford.edu/~birch/headtracker/Video from http://www.robots.ox.ac.uk/~jmac/research/thesis/thesis.html
![Page 41: BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003](https://reader035.vdocuments.us/reader035/viewer/2022062519/56649e605503460f94b5b07c/html5/thumbnails/41.jpg)
Weaknesses The algorithm is sensitive to reflections.
The algorithm sometimes switches the labels when one object passes in front of another.
There are a lot of parameters to assign.