![Page 1: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/1.jpg)
A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video
Department of Electrical Engineering and Computer ScienceThe University of Kansas, Lawrence, KS
Manjunath Narayana Donna Haverkamp,Assistant Professor
IEEE International Conference on Object Tracking and Classification in and Beyond the Visual Spectrum
IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Minneapolis, MN, USA, Friday, June 22, 2007
![Page 2: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/2.jpg)
The University of Kansas 2
Main Objective
Problem:
Tracking of moving objects in surveillance video
Our solution:
A Bayesian approach to assign tracks to objects probabilistically, based on color and position observations from objects
![Page 3: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/3.jpg)
The University of Kansas 3
Outline
Motivation Background Method Illustration Results Summary Future work
![Page 4: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/4.jpg)
The University of Kansas 4
Motivation
Accurate tracking very important for surveillance applications
Major issue: object data is noisy Objects
Appear in the scene Disappear due to exit from scene or
occlusion Merge with other objects or the background Break up into two or more objects due to
occlusion A probabilistic algorithm to assign track
numbers to objects may be very useful
![Page 5: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/5.jpg)
The University of Kansas 5
Example of tracking problem
Fig 2Fig 1
previous frame current frame
![Page 6: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/6.jpg)
The University of Kansas 6
Segmentation and Tracking
Motion segmentation (background subtraction) used for object detection
Once moving objects (blobs) detected, find correspondence between tracks of previous frame and blobs of current frame
Most common method: a Match matrix used to determine correspondences
Euclidean distance between blobs commonly used as the measure for a match
![Page 7: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/7.jpg)
The University of Kansas 7
Our Tracking Approach
We propose a Bayesian approach to determine probabilities of match between blobs.
Bayesian approach results in a Belief matrix (of probabilities) instead of a Match matrix (of distances)
![Page 8: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/8.jpg)
The University of Kansas 8
Method
track
blob
What is Probability of track blob ?
Basic principle: Given a track in previous frame, we expect
a blob of similar color and position in current frame with some probability
Provides basis for Bayesian method Upon observing a blob of given color and
position, what is the posterior probability that this blob belongs on one of the tracks from the previous frame?
![Page 9: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/9.jpg)
The University of Kansas 9
O1
O2
Illustration (1)
Example:
Blobs O1 and O2 seen in current frame
What is the probability that each of these blobs belongs to the tracks in the previous frame?
Color, c1={r1,g1,b1} Position, d1={y1,x1}
Color, c2={r2,g2,b2} Position, d2={y2,x2}
t1
t3
t2
![Page 10: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/10.jpg)
The University of Kansas 10
Probabilistic network for track assignment
![Page 11: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/11.jpg)
The University of Kansas 11
Illustration (2)
Blob O1
Blob O2
“lost”
t1 0.33 0.33 0.33
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
t1
t3
O1
O2
t2
Given three tracks t1, t2, and t3 in the previous frame
There are six probabilities:
Each track can either be assigned to blob O1 or O2, or may be lost in the frame
Produces initial Belief matrix with equal likelihood for all cases
)( 21 OtAssignp )( 11 OtAssignp
)( 22 OtAssignp )( 12 OtAssignp
)( 23 OtAssignp )( 13 OtAssignp
![Page 12: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/12.jpg)
The University of Kansas 12
Consider track t1 and update first element in matrix
Observations: color (c1), position(d1)
To find:
Assumption - color and position observations are independent:
Illustration (3)
)( 11 OtAssignp
),( 1111 dcOtAssignp
t1
O1
)()(
),(
111111
1111
dOtAssignpcOtAssignp
dcOtAssignp
![Page 13: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/13.jpg)
The University of Kansas 13
Illustration (4)
Blob O1
Blob O2
“lost”
t1 0.60 0.33 0.33
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
First, color observation for first element in matrix, c1
By Bayes formula:
The Belief matrix is updated
t1
O1
)(
)()(
)(
1
11111
111
cp
OtAssignpOtAssigncp
cOtAssignp
![Page 14: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/14.jpg)
The University of Kansas 14
Illustration (5)
Blob O1
Blob O2
“lost”
t1 0.60 0.20 0.20
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
After row normalization
Blob O1
Blob O2
“lost”
t1 0.60 0.33 0.33
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
Row needs to be normalized so that sum of elements is 1
![Page 15: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/15.jpg)
The University of Kansas 15
Next, position observation for first element in matrix, d1 , is considered
By Bayes formula:
Calculation and row normalization:
t1
O1
Illustration (6)
Blob O1
Blob O2
“lost”
t1 0.90 0.05 0.05
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
)(
)()(
)(
1
11111
111
dp
OtAssignpOtAssigndp
dOtAssignp
)( 111 dOtAssignp
![Page 16: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/16.jpg)
The University of Kansas 16
Illustration (7)
Blob O1
Blob O2
“lost”
t1 0.75 0.22 0.08
t2 0.33 0.33 0.33
t3 0.33 0.33 0.33
After the first element update, we move to second element
Similar calculation and update:
Row 1 processing - complete
t1
O2
)( 21 OtAssignp
![Page 17: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/17.jpg)
The University of Kansas 17
t1
Illustration (8)
Blob O1
Blob O2
“lost”
t1 0.75 0.22 0.08
t2 0.25 0.60 0.15
t3 0.33 0.33 0.33
Similarly, processing track t2 and row 2:
t3
t2
Blob O1
Blob O2
“lost”
t1 0.75 0.22 0.08
t2 0.25 0.60 0.15
t3 0.20 0.30 0.50
Processing track t3 and row 3:
![Page 18: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/18.jpg)
The University of Kansas 18
Illustration (9)Blob O1
Blob O2
“lost”
t1 0.75 0.22 0.08
t2 0.25 0.60 0.15
t3 0.20 0.30 0.50
t1
t3
O1
O2
t2
Based on the Belief matrix, the following assignments may be made
t1 O1
t2 O2
t3 ”lost”
![Page 19: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/19.jpg)
The University of Kansas 19
Results
Blob 1 Blob 2 “lost”
track 03
0.00 0.10 0.90
track 07
0.00 1.00 0.00
track 11
0.00 0.39 0.61
track 12
1.00 0.00 0.00
Real example - frame 0240
Belief matrix
Resulting track assignments
Blob 1
Blob 2
![Page 20: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/20.jpg)
The University of Kansas 20
Results – comparison with Euclidean distance matrix
Blob 1
Blob 2
Blob 3
![Page 21: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/21.jpg)
The University of Kansas 21
Results – comparison with Euclidean distance matrix
“lost” probability can be useful Track 12 (lost in frame 0275) would be
erroneously assigned to blob 3 if Euclidean distance matrix used
![Page 22: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/22.jpg)
The University of Kansas 22
Results - tracking
![Page 23: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/23.jpg)
The University of Kansas 23
Summary
Probabilistic track assignment method using the blob color and position observations
Tracks allowed to be “lost” and recovered Provides alternative to Euclidean distance
based match matrix Accuracy similar to Euclidean distance
based matrix Probability values
Easier to interpret Beneficial for further processing
![Page 24: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/24.jpg)
The University of Kansas 24
Future Work
Incorporate blob size as an observation Extend to other spectra Use a learning approach to determine PDF’s
for p(c|Assign) and p(d|Assign)
![Page 25: A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University](https://reader035.vdocuments.us/reader035/viewer/2022081516/56649d435503460f94a20163/html5/thumbnails/25.jpg)
The University of Kansas 25
Questions