detecting and tracking tractor-trailers using view-based templates
DESCRIPTION
Detecting and Tracking Tractor-Trailers Using View-Based Templates. Masters Thesis Defense by Vinay Gidla Apr 19 ,2010. Introduction. Object tracking: Sports analysis Games and gesture recognition Retail video mining Automobile driver assistance Traffic surveillance - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/1.jpg)
DETECTING AND TRACKING
TRACTOR-TRAILERS USING VIEW-BASED TEMPLATES
Masters Thesis Defense byVinay GidlaApr 19,2010
![Page 2: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/2.jpg)
Introduction• Object tracking: • Sports analysis • Games and gesture recognition • Retail video mining• Automobile driver assistance
• Traffic surveillance• Volume, individual speeds, classification• Lane changes, speed violations,
congestions
![Page 3: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/3.jpg)
Feature-based vehicle tracking
• Beymer et al. 1997 use feature point approach with motion cues to segment vehicles using homography
• Kanhere et al. 2008 use features with 3D estimation using multi-level homographyFeature_based.avi
• Drawback: These approaches track features on the vehicle, not vehicle as a whole
![Page 4: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/4.jpg)
Template-based tracking
• Model the object by 2D template of image intensities
• Compare search image with template image
• Comparison usually by discrete cross-correlation
• Good: Both spatial and appearance informationAble to retrieve shape of the object
• Bad: Encode vehicle appearance from single viewpoint
Do not adapt to changes in appearance of object
![Page 5: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/5.jpg)
Proposal
• Overcome the limitations of a single template by using a template sequence instead of a single template
• The template sequence encapsulates all of the vehicle’s perspective deformations
• As a starting step, aim to detect and track contours of tractor-trailers in multi-lane traffic
![Page 6: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/6.jpg)
Video Sequences
![Page 7: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/7.jpg)
Template creation
Training sequence:
• A portion of traffic video containing a tractor-trailer
• Process the video frames to create a template sequence
![Page 8: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/8.jpg)
Training Sequence
![Page 9: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/9.jpg)
Training frame
![Page 10: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/10.jpg)
Manual contour selection
![Page 11: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/11.jpg)
Template creation
![Page 12: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/12.jpg)
Template sequence
![Page 13: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/13.jpg)
Algorithm Overview
![Page 14: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/14.jpg)
Step 1: Background subtraction
![Page 15: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/15.jpg)
Reference background
![Page 16: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/16.jpg)
Input Video Frame
![Page 17: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/17.jpg)
Background subtracted frame
![Page 18: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/18.jpg)
Step 2: Blob-Template match
![Page 19: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/19.jpg)
Blob-Template match
![Page 20: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/20.jpg)
Plot of Blob-Template match
![Page 21: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/21.jpg)
Step 3: Trace contour
![Page 22: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/22.jpg)
Results based on template-blob correlation
![Page 23: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/23.jpg)
Plot of misalignment
![Page 24: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/24.jpg)
Gradient magnitude match
• Reduce the misalignment by including salient features such as points of high gradient magnitude
• These points are located at identical spatial locations in every tractor-trailer
![Page 25: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/25.jpg)
Training frame
![Page 26: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/26.jpg)
Gradient Magnitude
![Page 27: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/27.jpg)
Template Gradient Magnitude
![Page 28: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/28.jpg)
Template Gradient Sequence
![Page 29: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/29.jpg)
![Page 30: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/30.jpg)
Results based on template-frame gradient
match
![Page 31: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/31.jpg)
Plot of misalignment
![Page 32: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/32.jpg)
Test sequences
![Page 33: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/33.jpg)
Results(Lane 3)
![Page 34: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/34.jpg)
Results(Lane 2)
![Page 35: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/35.jpg)
Level set-based trackingfor automatic template
generation
![Page 36: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/36.jpg)
Conclusion
• The new approach accurately traces the contours of all the tractor-trailers in the traffic video
• Works for multi-lane traffic
• Minor misalignment
![Page 37: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/37.jpg)
Future extensions
• Tracking other classes of vehicles such as passenger cars, buses etc
• Compact template sequence with minimal template redundancy
• Implement matching using level set techniques
![Page 38: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/38.jpg)
Thank You
![Page 39: Detecting and Tracking Tractor-Trailers Using View-Based Templates](https://reader035.vdocuments.us/reader035/viewer/2022081514/56813652550346895d9dd74d/html5/thumbnails/39.jpg)
Questions &
Discussion