vision-based multiple vehicle detection and tracking for driver assistant system
DESCRIPTION
Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System. Qing Ming [email protected] May. 11. 2012. Vision Based Driver Assistant System. Problem setting. Main Goal : Multiple vehicle detection and tracking . Challenge work:. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/1.jpg)
Qing [email protected]
May. 11. 2012
Vision-Based Multiple Vehicle Detection and Tracking for Driver
Assistant System
![Page 2: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/2.jpg)
2Intelligent Systems
Lab.
Vision Based Driver Assistant System
![Page 3: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/3.jpg)
3Intelligent Systems
Lab.
Problem setting
Main Goal: Multiple vehicle detection and tracking
Challenge work:-Different environment(illumination, distance, background…)-Tracking windows scale dynamic adjustment -Vehicle partial occlusion-Vehicle temporary missing
![Page 4: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/4.jpg)
4Intelligent Systems
Lab.
…
…
yes
System architecture
Frame 1
Vehicle detection unit
Detectednew vehicle?
yes
Frame 2
Vehicle detection unit
Detectednew vehicle?
yes
Frame n
Vehicle detection unit
Detectednew vehicle?
Particle filter 1
Internal storage
Particle filter 2
Particle filter m
Vehicle Tracking Unit Return to Vehicle
detection unit
![Page 5: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/5.jpg)
5Intelligent Systems
Lab.
Offline process
Vehicle Detection Algorithm
Image Sequence
Gabor FeatureExtraction
BP Neural NetworkTraining
BP Neural NetworkClassifier
Test Image
Vehicle Candidate Detection
Vehicle Candidate Verification
Detected Vehicle
![Page 6: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/6.jpg)
6Intelligent Systems
Lab.
Color segmentation
Morphological operation
Original image
Vehicle candidate detection
+
Threshold obtained by tail light image statistical value
![Page 7: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/7.jpg)
7Intelligent Systems
Lab.
Vehicle candidate generation
hc2
1 2min maxc cw w w 1 2c ch h d
wc1c2: width between vehicle light pair.
hc1, hc2: height of C1 and C2
d : a constant which depends on image size
hc1
wc1c2
![Page 8: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/8.jpg)
8Intelligent Systems
Lab.
Gabor feature2 2
2 212 ( )1(x,y, , ) exp exp
2x y
x y
j x y
x y
G
8 orientations5 scale
Tai Sing LEE, “Image Representation using 2D Gabor Wavelets,”IEEE Transactions on Parrern Analysis and Machine Intelligence,Vol 18,No.10, pp959-971,October 1996
![Page 9: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/9.jpg)
9Intelligent Systems
Lab.
…
…
Training
Back propagationNeural network
Non-vehicle database
vehicle database
-1
1
Stuart Russell and Peter Norvig, “Artificial Intelligence A Modern Approach”. p. 578. 1969
![Page 10: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/10.jpg)
10Intelligent Systems
Lab.
training
…
(No)
(Yes)
(Yes)BPNNclassifier
Gabor feature set
Vehicle candidate verification
![Page 11: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/11.jpg)
11Intelligent Systems
Lab.
Vehicle Tracking
Frame t
Frame t+1
Detected vehicle
Histogram generation
Colorhistogram
Particle generation
Particle selection
Similaritycomputation
TrackingWindow
estimation
Histogram updating
Updatedhistogram …
![Page 12: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/12.jpg)
12Intelligent Systems
Lab.
Target Vehicle Representation
Frame t
Color space representation
Detected vehicle
Histogram representation
Split into uniform Histogram bins
![Page 13: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/13.jpg)
13Intelligent Systems
Lab.
Color PDF
Particle Generation
Frame t+1 Particle selection Each Particle is
consider as one pixel
• Randomly generate particles at the position of tracking window in previous frame
![Page 14: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/14.jpg)
14Intelligent Systems
Lab.
Similarity Computing
……
•Bhattacharyya coefficient•Mean state of the particle set
Each selected Particle is consider as one region
Dorin Comaniciu, Visvanathan Ramesh, P eterMeer, "Real-Time Tracking of Non-Rigid Objects using Mean Shift, " IEEE Conference on Computer Vision and Pattern Recognition, June 13 -15, Hilton Head, SC, USA, 2000
![Page 15: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/15.jpg)
15Intelligent Systems
Lab.
Partial occlusion and temporary missing
Frame 1 Frame 45 Frame 25
Frame 120 Frame 80 Frame 148
Target vehicle Partial occlusion Temporary missing
Temporary missing Partial occlusion Target vehicle re-tracking
![Page 16: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/16.jpg)
16Intelligent Systems
Lab.
Temporary missing
Frame 80
Particles are generated nearby the covering vehicle bounding box
effective particles are searched
Frame 125
When enough effective particles are searched, the missing vehicle start Tracking again
![Page 17: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/17.jpg)
17Intelligent Systems
Lab.
Color Histogram Updating
Target vehicle color histogram changing under different condition
(ex: different distance, different illumination…)
![Page 18: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/18.jpg)
18Intelligent Systems
Lab.
Color Histogram Updating
Frame 1 Frame 55 Frame 62 Frame 115
Frame 1 Frame 55 Frame 62 Frame 115
Tracking without color histogram updating
Tracking with color histogram updating
![Page 19: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/19.jpg)
19Intelligent Systems
Lab.
Detection result
High way Urban Road Campus
![Page 20: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/20.jpg)
20Intelligent Systems
Lab.
Multiple vehicle detection result
Image originHigh way Urban road Campus
Total number of vehicles 56 42 36
Number of vehicle Correct detection 43 34 30
Number of vehicle fail detection 13 8 6
Detect rate (%) 76.8 80.1 83.3
![Page 21: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/21.jpg)
21Intelligent Systems
Lab.
Tracking result
![Page 22: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/22.jpg)
22Intelligent Systems
Lab.
Tracking result
Horizontal trajectory Vertical trajectory
![Page 23: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/23.jpg)
23Intelligent Systems
Lab.
Tracking result
Trajectory in image plane Tracking error
![Page 24: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/24.jpg)
24Intelligent Systems
Lab.
Partial occlusion and temporary missing result
![Page 25: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/25.jpg)
25Intelligent Systems
Lab.
Multiple vehicle tracking result
Video origin
High way Urban Road Campus
Total number of frames 5254 952 1326
Image Size 640×480 1920×1080 640×480
Tracking window size (in pixel) 40×32-130×104 60×48-320×256 32×25-130×104
AVG. of tracking trajectory error (in pixels) 10 23 7
Total number of moving vehicle 12 5 6
Number of miss tracking vehicle 3 1 0
![Page 26: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/26.jpg)
26Intelligent Systems
Lab.
Conclusion
Detected multiple vehicles in different environment (different light condition, different size vehicle, different speed) Tracked partial occluded vehicle Re-tracked temporary missing vehicle Tracking windows dynamically adapt according to target vehicle scale changing Color histogram self-updating
Advantage
Disadvantage Only color model based multiple vehicle tracking is not suitable for same color Vehicle occlusion problem
Future works -Camera stabilization for smooth trajectory generation-Combine with odometry information to predict dangerous situations
![Page 27: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/27.jpg)
27Intelligent Systems
Lab.
Publications Qing Ming, Kang-Hyun Jo, “Vehicle Detection Using Tail Light Segmentation,” International Forum on Strategic Technology, August 22-24, Harbin, 2011.Ming Qing and Kang-Hyun Jo, “Vehicle Detection and Scale-adaptive Tracking Using Tail Light Segmentation,” proc. of image and vision computing New Zealand, pp. 115-119, 2011.Ming Qing and Kang-Hyun Jo, “A novel particle filter implementation for Multiple-Vehicle Detection and Tracking System using Tail Light Segmentation”, International Journal of Control, Automation, and Systems (Reviewing)Ming Qing and Kang-Hyun Jo, “Vision Based Multiple Vehicle Detection and Tracking Using Tail Light Segmentation”, IECON Montreal (reviewing), October 25-28, 2012 Mecatronics 2012, Paris, Date line: 30 May,2012
![Page 28: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/28.jpg)
28Intelligent Systems
Lab.
![Page 29: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/29.jpg)
29Intelligent Systems
Lab.
BPNNBack propagation nerual network:
Xi : input Zi :output t1: expect output wij : weights between input layer and hidden layer wjk : weights between hidden layer and output layer. The input goes through the neural network in order to obtain the forward propagation’s output. Compare with expect output, difference value backward propagation through network to update weights
![Page 30: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/30.jpg)
30Intelligent Systems
Lab.
HSV color model
H: HueS: SaturationV: Value
![Page 31: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/31.jpg)
31Intelligent Systems
Lab.
Particle filter
Particle filter:
PropagateParticle generation evaluation …
p(xt|zt) p(xt+1|zt+1)p(xt+1|zt)
Zt+1
Zt: observation
P(xt|Zt): particle state under current observation
P(xt+1|Zt): particle state prediction
Propagate
p(xt+2|zt+1)
![Page 32: Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System](https://reader033.vdocuments.us/reader033/viewer/2022051420/56815e01550346895dcc461f/html5/thumbnails/32.jpg)
32Intelligent Systems
Lab.
Bhattacharyya coefficient
Bhattacharyya coefficient is an approximate measurement of the amount of overlap between two statistical samples. This coefficient can be used to describe the similarity of two discrete and normalized distributions.
1
[ , ]m
u uu
p q p q