![Page 1: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/1.jpg)
Automatic Tracking of Moving Objects in Video for Surveillance Applications
Department of Electrical Engineering and Computer ScienceThe University of Kansas, Lawrence, KS
Manjunath Narayana
Committee:Dr. Donna Haverkamp (Chair)
Dr. Arvin AgahDr. James Miller
![Page 2: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/2.jpg)
The University of Kansas 2
Main Objective
Problem:Tracking of moving objects in surveillance video
Goals: Set up a system for automatic tracking at KUAdd to existing research in the field by developing new algorithms
![Page 3: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/3.jpg)
The University of Kansas 3
Outline
MotivationContributions of this thesisBackgroundSystem descriptionShortcomings in basic systemImprovementsThe Vicinity Factor – a new conceptBayesian algorithm for probabilistic track assignment ResultsSummary and ConclusionsFuture work
![Page 4: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/4.jpg)
The University of Kansas 4
Motivation
High interest in intelligent video applicationsSurveillance – Identification of interesting/anomalous behavior
Airports, highways, officesRemove or assist humans in these applications
Identification/Classification of targetsSegmentation and tracking of targets are first tasks in all above applicationsAim: Setting up a tracking framework at KU
First step that can result in great deal of research in the future
![Page 5: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/5.jpg)
The University of Kansas 5
Contributions of this thesis
Establishment of framework for Object segmentation and tracking
Object segmentation – distinguishing the object of interest in the videoTracking – following the object’s motion in successive frames
Development of the Vicinity Factor – a new conceptDesign of a new Bayesian algorithm for tracking
paper accepted for CVPR 2007 workshop
![Page 6: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/6.jpg)
The University of Kansas 6
System Overview
Input video sequenceSegment moving regionsDetect objects (or “blobs”)Track blobs over framesOutput video with tracks
Input video Motion segmentation Blob detection
Output tracks
![Page 7: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/7.jpg)
The University of Kansas 7
BackgroundObject segmentation
Color basedEdge basedMotion based
Background pixel modelingForeground pixel modelingOptical flow methodsGradient methods like moving edge detector
We use average background modeling with improvementsTracking
Once objects detected, find correspondence between objects of previous frame and objects of current frame“Match matrix” of distances between objects is used to find correspondence
![Page 8: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/8.jpg)
The University of Kansas 8
System Architecture
![Page 9: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/9.jpg)
The University of Kansas 9
Input Image Sequence
![Page 10: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/10.jpg)
The University of Kansas 10
Motion Segmentation
![Page 11: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/11.jpg)
The University of Kansas 11
Segmentation methodDevelop a model for the backgroundSubtract the current frame from background modelThreshold the difference imageChallenges:
Outdoor sequences are more difficultIllumination changesCamera jitter and noise
− =
Background model Current frame Thresholded difference
![Page 12: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/12.jpg)
The University of Kansas 12
Background modelCannot use static background (BG) model for outdoors Moving average BG model The average grayscale intensity value of each pixel over 150 frames
+ +
Previous 75 frames Next 75 framesCurrent frame
![Page 13: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/13.jpg)
The University of Kansas 13
Moving average BG modelSimple yet efficientWorks because fast moving objects do not contribute much to the average value
![Page 14: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/14.jpg)
The University of Kansas 14
Thresholding the difference image
![Page 15: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/15.jpg)
The University of Kansas 15
Shortcomings of average BG model
Tails visible in front and behind of objects
![Page 16: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/16.jpg)
The University of Kansas 16
Improvement in BG model
Intensity of BG does not change much from one frame to nextThe intensity of BG pixel in previous frame is better estimate of BG in current frame than is the average BGSecondary BG model (SBG)Use of secondary model to refine segmentation
![Page 17: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/17.jpg)
The University of Kansas 17
SBG model illustration
Basic BG model
SBG model
Original frame
![Page 18: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/18.jpg)
The University of Kansas 18
SBG model calculation
Initialization:
Update:
Segmentation:
If (y,x) is a BG pixel in frame k
otherwise
![Page 19: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/19.jpg)
The University of Kansas 19
SBG model
Sharper pictureAveraging and blurring effect of basic BG model is not present in SBG model
![Page 20: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/20.jpg)
The University of Kansas 20
SBG results
Basic Average BG model results
Secondary BG model refined results
![Page 21: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/21.jpg)
The University of Kansas 21
Noise removal
Remove blobs that are very smallGroup all connected pixels as one objectPerform Dilation and Erosion morphology operations
Small blobs get deletedEnvelope is established around each detected object in the color image
Calculate statistics for each blob (color, position, size)
![Page 22: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/22.jpg)
The University of Kansas 22
Object Tracking
![Page 23: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/23.jpg)
The University of Kansas 23
Correspondence algorithmOnce moving objects (blobs) detected, find correspondence between tracks of previous frame and blobs of current frameMost common method: a Match matrix used to determine correspondencesEuclidean distance between blobs commonly used as the measure for a matchNon-trivial because data is noisy and objects are not predictableObjects
Appear in the scene Disappear due to exit from scene or occlusion Merge with other objects or the backgroundBreak up into two or more objects due to occlusion
![Page 24: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/24.jpg)
The University of Kansas 24
Example of tracking problem
Fig 2Fig 1
previous frame current frame
![Page 25: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/25.jpg)
The University of Kansas 25
Correspondence problem
Tracks:
Blobs:
Given track set of previous frame and blob set of current frame, calculate the Match matrix of Euclidean distances between them in color space (R,G,B) and Position (Y, X) values
![Page 26: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/26.jpg)
The University of Kansas 26
Match matrix
???t3
???t2
???t1
Blob O3Blob O2Blob O1
![Page 27: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/27.jpg)
The University of Kansas 27
Match matrix example
![Page 28: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/28.jpg)
The University of Kansas 28
Shortcomings in basic algorithm – broken objects
Broken objects
![Page 29: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/29.jpg)
The University of Kansas 29
Shortcomings in basic algorithm – velocity not used
Assignment against flow of velocity
![Page 30: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/30.jpg)
The University of Kansas 30
Merge module for broken objects
![Page 31: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/31.jpg)
The University of Kansas 31
Velocity factor
Multiply all entries in the Match matrix with a scale factor that reflects velocity
![Page 32: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/32.jpg)
The University of Kansas 32
Tracking accuracy Accuracy difficult to calculateData is real-life surveillance videoNo ground truthManual observation and analysis used to determine accuracy3 sub-sequences chosen in video
Easy, medium, and difficult situationsAbout 700 frames in all
![Page 33: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/33.jpg)
The University of Kansas 33
Sub-sequences for analysis
Sub-sequence 1
Sub-sequence 3
Sub-sequence 2
![Page 34: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/34.jpg)
The University of Kansas 34
Comparison metrics
“number-of-blobs” errorTo find accuracy of segmentation
“index” error and “swap” errorTo calculate errors in tracking“index” error – number of absolute errors in track numbers“swap” error – number of times a track number for an object changes
![Page 35: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/35.jpg)
The University of Kansas 35
Error calculation
![Page 36: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/36.jpg)
The University of Kansas 36
Segmentation errors
Compares segmentation results with and without use of blob merging
![Page 37: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/37.jpg)
The University of Kansas 37
Tracking errors - index
Compares tracking index errors with and without use of Velocity factor
![Page 38: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/38.jpg)
The University of Kansas 38
Tracking errors - swap
Compares tracking swap errors with and without use of Velocity factor
![Page 39: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/39.jpg)
The University of Kansas 39
Results - summary
Blob merging is usefulImprovement by 17.2%
Accuracy of blob merging depends on the threshold usedVelocity factor is very useful
Improvement by 44.4%
![Page 40: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/40.jpg)
The University of Kansas 40
Depth variation in scenes
In videos, there is significant change in object distance from camera (“Object Depth”, OD) from one part of the image to anotherThis affects the position thresholds that are used in the algorithmEx: Car 01 far away from camera compared to Car 02. Need to use different value for threshold for each carFor instance, the threshold used for blob merging
![Page 41: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/41.jpg)
The University of Kansas 41
Object Depth variation
Need to compensate for change in ODCan use 3-D modeling or stereo-based approaches
Complicated methods
Alternative Learning approach to solve this problem
The Vicinity Factor
![Page 42: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/42.jpg)
The University of Kansas 42
The Vicinity FactorObjects closer to camera move more in pixel space than objects far away doBy observing this variation during tracking, we can learn the relative distances of objects in different parts of the scene
![Page 43: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/43.jpg)
The University of Kansas 43
Vicinity Factor (VF) conceptBreak the scene into grids of 30 by 30 pixelsSet an initial value for VFEach time a track is seen in a grid location, observe how much it movesIf motion is large, then VF for that grid location is increasedIf motion is small, then VF for that grid location is decreased
![Page 44: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/44.jpg)
The University of Kansas 44
VF Calculation (1)
Initialize:
is the VF in Y direction is the VF in X direction where
Update:
![Page 45: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/45.jpg)
The University of Kansas 45
VF Calculation (2)
Note that VF Y and VF X will not have the same values
![Page 46: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/46.jpg)
The University of Kansas 46
VF matrix example
![Page 47: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/47.jpg)
The University of Kansas 47
Use of VF in blob merging
Used VF as basis for variable thresholds in different parts of the scene for blob merging
![Page 48: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/48.jpg)
The University of Kansas 48
Use of VF in blob merging (2)
Example of improvement
![Page 49: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/49.jpg)
The University of Kansas 49
Probabilistic track assignment
Thus far, tracks assigned based on Euclidean distance matrixProbabilistic track assignment can lead to better inference in higher level applicationsBayesian algorithm to track objects was designedWork accepted and presented at CVPR 2007 workshop, Minneapolis, 18-23rd June, 2007
![Page 50: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/50.jpg)
The University of Kansas 50
Need for Probabilistic track assignment
Major issue: object data is noisyObjects
Appear in the scene Disappear due to exit from scene or occlusion Merge with other objects or the backgroundBreak up into two or more objects due to occlusion
A probabilistic algorithm to assign track numbers to objects may be very useful
![Page 51: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/51.jpg)
The University of Kansas 51
Our Bayesian Approach
We propose a Bayesian approach to determine probabilities of match between blobs. Bayesian approach results in a Belief matrix (of probabilities) instead of a Match matrix (of distances)
![Page 52: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/52.jpg)
The University of Kansas 52
Method
track
blob
What is Probability of track blob ?
Basic principle:Given a track in previous frame, we expect a blob of similar color and position in current frame with some probability
Provides basis for Bayesian method
Upon observing a blob of given color and position, what is the posterior probability that this blobbelongs on one of the tracks from the previous frame?
![Page 53: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/53.jpg)
The University of Kansas 53
O1
O2
Illustration (1)
Example:
Blobs O1 and O2 seen in current frame
What is the probability that each of these blobs belongs to the tracks in the previous frame?
Color, c1={r1,g1,b1} Position, d1={y1,x1}
Color, c2={r2,g2,b2} Position, d2={y2,x2}
t1
t3
t2
![Page 54: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/54.jpg)
The University of Kansas 54
Probabilistic network for track assignment
![Page 55: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/55.jpg)
The University of Kansas 55
Illustration (2)
0.330.330.33t3
0.330.330.33t2
0.330.330.33t1
“lost”Blob O2Blob O1
t1
t3
O1
O2
t2
Given three tracks t1, t2, and t3in the previous frame
There are six probabilities:
Each track can either be assigned to blob O1 or O2, or may be lost in the frame
Produces initial Belief matrix with equal likelihood for all cases
)( 21 OtAssignp →)( 11 OtAssignp →
)( 22 OtAssignp →)( 12 OtAssignp →
)( 23 OtAssignp →)( 13 OtAssignp →
![Page 56: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/56.jpg)
The University of Kansas 56
Consider track t1 and update first element in matrix
Observations: color (c1), position(d1)
To find:
Assumption - color and position observations are independent:
Illustration (3)
)( 11 OtAssignp →
),( 1111 dcOtAssignp →
t1
O1
)()(
),(
111111
1111
dOtAssignpcOtAssignp
dcOtAssignp
→×→
=→
![Page 57: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/57.jpg)
The University of Kansas 57
Illustration (4)
0.330.330.33t3
0.330.330.33t2
0.200.200.60t1
“lost”Blob O2Blob O1
First, color observation for first element in matrix, c1
By Bayes formula:
The Belief matrix is updated
t1
O1
)()()(
)(
1
11111
111
cpOtAssignpOtAssigncp
cOtAssignp
→×→
=→
![Page 58: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/58.jpg)
The University of Kansas 58
Next, position observation for first element in matrix, d1 , is considered
By Bayes formula:
Calculation and row normalization:
t1
O1
Illustration (5)
0.330.330.33t3
0.330.330.33t2
0.050.050.90t1
“lost”Blob O2Blob O1
)()()(
)(
1
11111
111
dpOtAssignpOtAssigndp
dOtAssignp
→×→
=→
)( 111 dOtAssignp →
![Page 59: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/59.jpg)
The University of Kansas 59
Illustration (6)
0.330.330.33t3
0.330.330.33t2
0.080.220.75t1
“lost”Blob O2Blob O1
After the first element update, we move to second element
Similar calculation and update:
Row 1 processing - complete
t1O2
)( 21 OtAssignp →
![Page 60: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/60.jpg)
The University of Kansas 60
t1
Illustration (7)
0.330.330.33t3
0.150.600.25t2
0.080.220.75t1
“lost”Blob O2Blob O1
Similarly, processing track t2and row 2:
t3
t2
0.500.300.20t3
0.150.600.25t2
0.080.220.75t1
“lost”Blob O2Blob O1
Processing track t3 and row 3:
![Page 61: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/61.jpg)
The University of Kansas 61
Illustration (8)
0.500.300.20t3
0.150.600.25t2
0.080.220.75t1
“lost”Blob O2Blob O1
t1
t3
O1
O2
t2
Based on the Belief matrix, the following assignments may be made
t1 O1
t2 O2
t3 ”lost”
![Page 62: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/62.jpg)
The University of Kansas 62
Results
0.610.390.00track 110.000.001.00track 12
0.001.000.00track 070.900.100.00track 03“lost”Blob 2Blob 1
Real example - frame 0240
Belief matrix
Resulting track assignments
Blob 1
Blob 2
![Page 63: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/63.jpg)
The University of Kansas 63
Results – comparison with Euclidean distance matrix
Blob 1
Blob 2
Blob 3
![Page 64: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/64.jpg)
The University of Kansas 64
Results – comparison with Euclidean distance matrix
“lost” probability can be useful Track 12 (lost in frame 0275) would be erroneously assigned to blob 3 if Euclidean distance matrix used Bayesian Belief matrix can be a useful alternative to distance-based match matrixMore useful in inference tasks
![Page 65: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/65.jpg)
The University of Kansas 65
Results - tracking
![Page 66: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/66.jpg)
The University of Kansas 66
Results sequence 1
![Page 67: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/67.jpg)
The University of Kansas 67
Results sequence 2
![Page 68: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/68.jpg)
The University of Kansas 68
Results sequence 3
![Page 69: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/69.jpg)
The University of Kansas 69
Summary
Designed suitable data structures, classes, and methods for BG segmentation, blobs, and tracksWorks on outdoor sequencesAverage Background modelSecondary Background modelBlob mergingVelocity factorThe Vicinity FactorBayesian algorithm
![Page 70: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/70.jpg)
The University of Kansas 70
Conclusions
Automatic tracking can lead to interesting research and applicationsWe have established a baseline system for future researchSuccessfully met research objectives
Setting up of a base systemDeveloping new algorithms
System is capable of tracking based on Euclidean distance-based match matrixBayesian Belief matrix for probabilistic assignment
![Page 71: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/71.jpg)
The University of Kansas 71
Future work (1)
Improvements in current systemObject segmentation
Gaussian BG modelTemplate matching and search methods
Object trackingUse of blob size as a feature for matchingSecondary analysis of tracks to establish longer tracksTemplate matching and search based tracking
Useful in non-static camera applications
![Page 72: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/72.jpg)
The University of Kansas 72
Future work (2)
Improvements in current system (contd..)Vicinity Factor
To identify anomalous behavior based on object motion To estimate object size in different parts of gridTo detect “active” regions of the sceneIn multi-class problems, use of separate VF matrix for each class of objects
Bayesian tracking algorithmAutomatically learn PDF’s for the assignment networkUse size as an observation, along with color and positionOther color spaces like HSV
![Page 73: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/73.jpg)
The University of Kansas 73
Future work (3)
Extensions of current systemLearn patterns of activity from tracksGait analysis for human identificationGesture recognitionBehavior analysis based on tracksDetection of events…
![Page 74: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/74.jpg)
The University of Kansas 74
Questions ?
![Page 75: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/75.jpg)
The University of Kansas 75
Probabilities – Color PDF
200 frames observedThe color difference between the track and corresponding blob is observedPDF for Red color shown hereSame PDF used for G and B
![Page 76: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/76.jpg)
The University of Kansas 76
Probabilities – Position PDF
200 frames observedThe position difference between the track and corresponding blob is notedPDF for Y position shown hereSame PDF used for X position
![Page 77: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/77.jpg)
The University of Kansas 77
Background (2)
Another common approachDevelop a model of the object being trackedUse searching and statistical methods to locate the object in each frameComputationally expensive methodKalman filtering, Joint Probabilistic Data Association Filters, Condensation
Commonly discussed tracking systemsVSAM – MITW4 – Univ of Maryland Bramble – Compaq Research
![Page 78: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/78.jpg)
The University of Kansas 78
SBG model illustration
![Page 79: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/79.jpg)
The University of Kansas 79
Track assignment procedure
Based on the Match matrixOlder tracks given preferenceIf a match is not found for a track, it is not deleted immediately, but kept alive
Declared as “lost”
In subsequent frames, if the track reemerges, it is reassigned
![Page 80: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/80.jpg)
The University of Kansas 80
Basic results (“new track” case)
![Page 81: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/81.jpg)
The University of Kansas 81
Basic results (“lost” case)
![Page 82: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/82.jpg)
The University of Kansas 82
How VF is useful?Automatically learns the distance variation that is caused due to OD changeGives basis for using variable thresholds in different parts of the imageApplications
Blob merging – the thresholds used to decide whether or not to merge two blobs can be varied based on VFObject size estimation – using VF Y and VF X valuesDetecting active regions in scene
VF changes from its initialized value only in parts of the scenewhere real trackable motion was observedRobust to noise blobs
![Page 83: Automatic Tracking of Moving Objects in Video for](https://reader031.vdocuments.us/reader031/viewer/2022020620/61e5e94d919e1871e151c715/html5/thumbnails/83.jpg)
The University of Kansas 83
Illustration (5)
0.330.330.33t3
0.330.330.33t2
0.200.200.60t1
“lost”Blob O2Blob O1
After row normalization
0.330.330.33t3
0.330.330.33t2
0.330.330.60t1
“lost”Blob O2Blob O1Row needs to be normalized so
that sum of elements is 1