imecs 2017: in situ real-time vision-based lane detection
Post on 22-Mar-2017
25 Views
Preview:
TRANSCRIPT
In Situ Real-Time Vision-Based Lane Detection on Raspberry Pi using 1D Haar Wavelet Spikes
Vladimir KulyukinDepartment of Computer Science
Utah State University
IMECS 2017
Outline
IntroductionLane Detection with 1D Haar Wavelet Spikes EvaluationConclusions
Introduction
Self-Driving Cars
Pros– Enhanced mobility for the elderly and disabled– Reduction in traffic accidents
Cons– Major job losses in driving and transportation industries– Loss of privacy and increased risk of hacking attacks
For now, human drivers are still indispensable
Related Work
S. Mallat and W. L. Hwang. “Singularity Detection and Processing with Wavelets.” IEEE Trans on Information Theory, vol. 38, no. 2, pp. 617 – 643, March 1992
Y. Wang, E. Teoha, D. Shen. “Lane detection and tracking using B-snake.” Image and Vision Computing, vol. 22, pp. 269-280, 2008
Z. Kim. “Robust lane detection and tracking in challenging scenarios.” IEEE Trans. on Intelligent Transportation Systems, vol. 9, no. 1, pp. 16 – 26, Mar. 2008
P. Hsiao, C. Yeh, S. Huang, L. Fu. “A portable vision-based real-time lane departure warning system: day and night.” IEEE Trans. on Vehicular Technology, vol. 58, no. 4, pp. 2089 – 2094, May 2009
P. Mandlik, A. Deshmukh. “Raspberry-pi based real time lane departure warning system using image processing.” International Journal of Engineering Research and Technology, vol. 5, issue 06, June-2016, pp. 755 – 762
Lane Detection with
1D Haar Wavelet Transform Spikes
Up-Down Spikes
Up-down spikes
Down-Up Spikes
Down-up spikes
Hardware
The hardware consists of an RPi 3 Model B ARM v8 1GB RAM computer, an RPi Camera Board v2, an RPi Night Vision Camera, and a 7 inch RPi touchscreen
The hardware is placed inside a Jeep Wrangler with a camera attached to the windshield from the inside and is powered through a regular 12V-to-5V car charger
As the car moves along a road, the detected lanes are drawn in real time in a small bottom right window on the touchscreen
HaarSpiker: Spike Detection Algorithm
HaarSpiker is an algorithm that uses 1D Haar Spikes to detect lanesThe algorithm is implemented in Python 2.7.9 with OpenCV 3.0.0The algorithm consists of three phases: pre-processing, spike detection, line fitting
Preprocessing: Cropping a Region of Interest (ROI)
A 360 x 240 PNG image is taken and a 56 x 200 ROI is cropped in the bottom center portion of the image where road is likely to be
Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI
Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI
Blur with Gaussian 7x7 kernel
Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI
Blur with Gaussian 7x7 kernel
Threshold with Otsu
Spike Detection
- Take image row segments on both sides of the image- Each segment is 64 pixels long (but this is a parameter)- Apply 2 iterations of Haar Wavelet Transform to each row segment- Detect up-down spikes in each row and compute the means of the flat segments for each detected spike- Adjust the start and end row segments on the basis of where the mean is detected in the previous row
Spike Detection
- Do you go top to bottom or bottom to top? Bottom to top, because you want to detect what is closest to the car - Do you process every row? No, go up in increments of 5 rows. This is a parameter that can be adjusted
Line Fitting
1D polynomial line fitting is applied to the (x, y) spike positions The lines are filtered by inclination angles:
- Left line threshold values are -60° to -30°- Right line threshold values are 30° to 60°
The final left and right lines indicate left and right lanes
Ideal Case of Lane Detection
Evaluation
Data
All the images taken from a Jeep Wrangler going at a speed of 60 mph on State Route 30 in Northern Utah
Sample 1: October 12th, 2016, a sunny day, consists of 1000 PNG images
Sample 2: November 12th, 2016, a cloudy day, consists of 1000 PNG images
Sample 3: January 6th, 2017, a snowy night, consists of 1000 PNG images
Sample 4: January 7th, 2017, a snowy day, consists of 775 PNG images
Image Size: 360 X 240 pix
Results
Sample Num. of Images Both Lanes Detected (%) At Least 1 Lane Detected (%)
1 1000 61.90 91.20
2 1000 34.10 77.40
3 1000 16.90 64.10
4 775 15.74 57.03
Results
All true negatives in sample 1 are caused by shadows cast by other vehicles or trees and changes in road surface texture
The algorithm’s accuracy on sample 2 which is 74.1% is worse due to faded road lanes on a cloudy day
Results
The algorithm’s accuracy drops to 64.10% on sample 3 on a snowy night
The algorithm’s accuracy deteriorates to 57.03% on sample 4 on a snowy day
ConclusionsThe algorithm processes 22 images/second in situ at a speed of 60 mph
The system’s hardware can be placed inside a car, next to the windshield, and can be powered through a regular 12V-to-5V car charger, i.e. the power requirement is 10W
The hardware and software components of the presented algorithm can be replicated with off-the-shelf hardware components and open source software
The algorithm performs well in fair weather; the performance declines at night and on snowy roads
Data and Source Code Availability
Images used in this study are available at https://www.dropbox.com/sh/yqpq0adt42n54dt/AAA6m5OF4s_C2KYYDLI4kc2ra?dl=0
Py code is available at https://github.com/VKEDCO/PYPL/tree/master/haar_spiker
Thank You!
top related