indian sign board detection and recognition for driving

Indian Sign Board Detection and Recognition for Driving Assistance

E. Ramalakshmi1, A. Gowthami Radha2 & Haswika Bommena3

1Department of IT, Chaitanya Bharathi Institute of Technology,

Gandipet, Hyderabad, Telangana. 2Department of IT, Chaitanya Bharathi Institute of Technology,

Gandipet, Hyderabad, Telangana. 3Department of IT, Chaitanya Bharathi Institute of Technology,

Gandipet, Hyderabad, Telangana. .

Email: [email protected], [email protected], [email protected]

Abstract: It is necessary to bring traffic sign detection for autonomous vehicles onto our roads. Also, simultaneous traffic sign detection and recognition is certainly a very challenging task. Although traffic sign detection and recognition is a fairly archaic field of research, only a few works exist in literature that performs simultaneous detection and recognition using a dataset of realistic real world images. Existing traffic sign detection algorithms perform the task, but mostly lack realistic road conditions. The main focus in this paper is to detect traffic signs under challenging conditions using a novel dataset that contains a variety of road conditions. The traffic signs in the video are detected using shape filtering techniques. The classification module present determines the type of detected road signs by using Neural networks.

Keywords: Traffic sign, Filtering, Sign board Detection, Recognition, Neural

Networks.

1. Introduction Road sign recognition is a task just from the beginning of automation by the means of computer vision. With the help of this system, there are a wide range of applications, starting from traffic control, as well as parking management and public security. Being already subject to commercial many applications, recognition systems continued to be an interesting topic for researchers. In all cases mentioned above, one deals with a series of problems, mainly consisting of: requirement for real time processing; various illumination conditions in motion vehicles; and signs belonging to other states. Road safety is one of the most crucial issues in modern society.‘Human Factor’ is the one of the factors which is directly related to Safety issues.In attention of Driver and pedestrian is one of the main causes of road accidents. The scrutiny of traffic signs and markings on roads play an important role in active driver assistance systems (ADAS) and control systems for autonomous vehicles.Multiple computer vision systems have been established to analyze the traffic signs. But the characteristics of existing algorithms (recognition accuracy, quantity of errors) are not adequate to rule out a human operator. Main challenge of the computer vision system used in traffic sign detection

AEGAEUM JOURNAL

Volume 8, Issue 6, 2020

ISSN NO: 0776-3808

http://aegaeum.com/ Page No: 385

is detecting the area on which the sign is present. The intention of installing traffic signs on roads is to warn and provide necessary knowledge to drivers about safe journeys. Drivers are facing various risks while driving with the expansion in the number of vehicles and this may also lead to accidents. A huge amount of accidents are occuring every day all over the world. These accidents are mainly because of the driver's inability to exercise all the ocular information that is available while driving. Traffic signs not only regulate the traffic but indicate the state of the road, mentoring and forewarning drivers and pedestrians. These signs can be distributed according to its shape and color, and both these characteristics constitute their content.

The main objective of our paper is to deal with video processing. The proposed model is divided into three parts: frame conversion, shape analysis, convolutional neural network. For ADAS this paper will serve as a module for detection. The model we are dealing with has both detection and classification which enhances the model performance to detect the sign in high frame rate. By using SVM we may not get the better performance for classification as we consider video processing and image processing. So CNN proved to be a better method for image processing. The main objective is to identify the sign using shape and colour analysis and give the sign for classification to get the better output. With the help of deep learning, feature extraction and classifier has been incorporated into a learning framework. In recent years, the enhancement of Convolutional neural networks are mainly focused on the following aspects: the design of the Convolutional layer and pooling layer,loss function,the activation function, regularization and Convolutional neural network can be applied to practical problems.

2. Related Work Many algorithms and methodologies have been proposed for road traffic sign detection.The proposed system has planned and implemented an indicator by adopting the framework of faster R-CNN and the structure of MobileNet. Faster R-CNN and Mobile Nets are combined and altered to make the detection procedure efficient. A pretty good detection result of distinctive traffic signs is reported for the first time on the GTSDB database. Here, shape and color information have been used to filter the localizations of small traffic signs. An efficient and robust CNN with asymmetric kernels is designed to be the classifier.

Conventional machine learning and computer vision based methods were extensively used for traffic signs classification but these methods were soon replaced by deep learning based classifiers. Also, two benchmarks are widely used as performance metrics to evaluate the detection performance, namely PASCAL VOC and ImageNet ILSVRC. The limitation of these popular datasets is that they contain images where traffic signs occupy nearly 80% of the image and it becomes comparatively easier to detect and recognize them. However, in realistic scenarios, while trying to detect and recognize from CCTV cameras mounted on the roads, a typical traffic sign might be, for instance, 80 x 80 pixels, in a 2000 x 2000 pixel image, which is less than 0.2% of the image. Hence, it becomes inevitable that such a method that detects and recognizes such small, but significant objects in an image exists in the literature.

3. Methodology

The major steps in the proposed approach are illustrated in Fig.3. The algorithm has three main stages:

AEGAEUM JOURNAL


ISSN NO: 0776-3808


1) Preprocessing 2) Detection and 3) Recognition. The system takes videos which contain traffic signs. The output of theis approach is the video with the name of a recognized road sign.

In preprocessing, the input images formed from videos are processed to recognize the signs easily. In the detection module, the region of interest is extracted from the image based on colour segmentation and makes them ready for the recognition stage. Convolution neural networks are used to carry out the recognition of the traffic sign.

3.1 Pre-Processing Before preprocessing the frames, the video is converted into frames and color segmentation is performed on each frame. The video is converted into frames as shown in the Figure 1.

Figure 1.Video to frame conversion Colours present in the image provide useful information for human attention; usually traffic signs are colored with contrast colours against road environments. Colour Segmentation is a process of extracting required colour and removing an unwanted part from the given image and outputs the binary image. Each frame of the video is preprocessed in this step. Binary image is generated. Binary images are generated from the color images by segmentation. Segmentation can be defined as the procedure of allocating each pixel of the source image to one or more classes. The number of binary images going to form depends on the number of classes in the image. In Otsu’s method of segmentation, assign pixels to background or foreground based on grayscale intensity. Edge Detection is also a method to produce a binary image with some pixels allocated to edge pixels, and is also a first step in further segmentation as shown in Figure 2.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


Figure 2. Binary Image

3.2. Detection Module The main function in the detection module is to identify contours from the images and to get them ready for the next stage. Contours can be simply explained as a curve connecting all the points which are continuous along the boundary, having the same color. The contours are a valuable tool for shape examination and object recognition. To obtain better results, binary images are used. There are mathematical structures with shapes that are formed by joining the points having an area of similar intensity or color. We are going to use this concept on the edges detection in the pre-processing step. This will help in finding the contours or objects in the image. By changing the value of similarity related to the circle we get different shapes. By applying a value of 0.7, maximum accuracy for the shape circle is obtained and we take the coordinates of the contours. Different steps in each module of the proposed system are shown in Figure 3.

Figure 3.Block Diagram- Overall Proposed Approach

AEGAEUM JOURNAL


ISSN NO: 0776-3808


3.3 Recognition Module

The Recognition stage takes the output from the previous detection stage as its input. It then recognizes the traffic sign and gives the output with the bounding box around the traffic sign and it’s name. Neural networks are applied in this stage, which has been experimented for result comparisons.

Neural network can be defined as a collection of neurons, arranged in the form of layers, which convert he input vector into output. Each neuron takes an input, applies a function on it and then moves that output to the next layer. Generally the networks are feed-forward in which each layer proceeds its output to all the units on the next layer, but there is no feedback to the previous layer. Weights are applied on the signals which are passing from one unit to another, and it is these weightings which are used in the training process to build a neural network to the problem at hand. The proposed Convolution Neural Network model is explained in the below three data flow diagrams. The final result from the recognition module is shown in Figure 4.

Figure 4. Result of recognition module

Flow of Convolution layers is as referred in Figure 5.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


Figure 5.The flow of Convolution layer

4. Dataset German Traffic Sign Recognition Benchmark is a multiclass, single-image classification which has 43 classes having 31,367 training images, 7,842 validated images and 12,640 testing images. The dataset has reliable ground-truth data due to semi-automatic annotation. The dataset also ensures that the real world traffic symbol instances are unique within the dataset i.e., each traffic sign only occurs once. Discussing about the images and their format, every image just contains one traffic sign each. The images contain a border of at least 8% around the actual traffic sign, which accounts for a minimum of 5 pixels. This allows for edge-based approaches for detection and recognition. The image sizes vary from 15*15 to 250*250 pixels but the images are not necessarily square because it ensures there is no bias to the image size or resolution. The annotations are provided in a separate CSV file and include the height and width of the bounding box, along with the coordinates of the top-left and bottom-right corner of traffic sign bounding box. The distribution of dataset can be better explained with the help of figure as Figure 6.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


Figure 6. Dataset overview

5.EXPERIMENTAL RESULTS

The result of this approach is a video with traffic signs highlighted with a bounding box and

name of the traffic sign.

Figure 7. Binary image

Figure 7. shows the output of the image after preprocessing. In Preprocess, image is preprocessed using LOG and binarization.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


Figure 8. Detection of Traffic Sign

Figure 9. Detection of No entry sign

Figure 8 and Figure 9 shows the outputs for detected sign and no entry sign in both the binary image and frame of the video.

6.Conclusion

This paper gives a finalized method to detect and recognize traffic signs from a video sequence, taking into account all the surviving difficulties related to object recognition in outdoor environments. A detailed approach for traffic sign detection and recognition using colour segmentation and convolutional neural network is shown in this report. Also, the

AEGAEUM JOURNAL


ISSN NO: 0776-3808


report brings out several issues that need attention from researchers. The traffic signs with various types of orientation, appearance and speed of vehicle make evaluation of the video data a challenging task. The main limitation is the frame rate at which the module recognizes the sign in the frame, and also signs are different in different states and countries. The module is trained using the GTSRB dataset while there are many signs that are different with respect to the dataset we used to train the dataset. We hope to know all the information about the traffic signs in future works such as the conservation state, level of deformation, angle of rotation. In future, the work should drive towards reduction of frame dimensions and processing time per frame.

REFERENCES [1].Jia Li and Zengfu Wang, Member “Real-Time Traffic Sign Recognition Based on Efficient CNNs in the Wild” IEEE,pp-1-10. [2].Z. Liu, J. Du, F. Tian and J. Wen, "MR-CNN: A Multi-Scale Region-Based Convolutional Neural Network for Small Traffic Sign Recognition," in IEEE Access, vol. 7, pp. 57120-57128, 2019. [3].A. Gonzalez et al., “Automatic traffic signs and panels inspection system using computer vision,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 2, pp. 485–499, Jun. 2011. [4].S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel,“Detection of traffic signs in real-world images: The German traffic sign detection benchmark,” in Proc. IEEE Int. Joint Conf. Neural Netw., Aug. 2013, pp. 1–8. [5].Escalera, S., et al.: Background on traffic sign detection and recognition. Traffic-sign recognition systems, pp. 5–13. [6].J. F. Khan, S. M. A. Bhuiyan, and R. R. Adhami, ``Image segmentation and shape analysis for road-sign detection,'' IEEE Trans. Intell. Transp. Syst., vol. 12, no. 1, pp 83,March2011. [7].Sheikh D.M.A.A., Kole A., Maity T. Traffic sign detection and classification using colour feature and neural network; Proceedings of the 2016 International Conference on Intelligent Control Power and Instrumentation (ICICPI); Kolkata, India. 21–23 October 2016. [8]. V. A. Prisacariu, R. Timofte, K. Zimmermann, I. Reid, and L. V. Gool, “Integrating object detection with 3D tracking towards a better driver assistance system,” in Proc. Int. Conf. Pattern Recognit., 2010, pp. 3344–3347 [9].Y. Yang, H. Luo, H. Xu, and F. Wu, ``Towards real-time traf_c sign detection and classi_cation,'' IEEE Trans. Intell. Transp. Syst., vol. 17, no. 7, pp. 2022_2031, Jul. 2016. [10].M. Da Lio et al., “Artificial co-drivers as a universal enabling technology for future intelligent vehicles and transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 1, pp. 244–263, Feb. 2015. [11].C. Szegedy et al., “Going deeper with convolutions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1–9. [12].C. Bahlmann, Y. Zhu, V. Ramesh, M. Pellkofer, and T. Koehler, “A system for traffic sign detection, tracking, and recognition using color, shape, and motion information,” in Proc. IEEE Intell. Veh. Symp., Jun. 2005, pp. 255–260. [13]. S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel. Detection of Traffic signs in real-world images: The German Traffic sign detection benchmark. In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1–8, August 2013. [14].Y. Gu, T. Yendo, M. P. Tehrani, T. Fujii, and M. Tanimoto, “Traffic sign detection in dual-focal active camera system,” in Proc. IEEE Intell. Veh. Symp., Jun. 2011, pp. 1054–1059.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


[15].Sebastian Houben, Johannes Stallkamp, Jan Salmen, Marc Schlipsing, and Christian Igel. Detection of Traffic signs in real-world images: The German Traffic Sign Detection Benchmark. In International Joint Conference on Neural Networks, number 1288, 2013. [16].T. Wang, D. Wu, A. Coates, and A. Ng, End-to-end text recognition with Convolutional neural networks, in International Conference on Pattern Recognition (ICPR), 2012, pp. 33043308. [17].Y. Boureau, J. Ponce, and Y. Le Cun, A theoretical analysis of feature pooling in visual recognition, in ICML, 2010, pp. 111118.

AEGAEUM JOURNAL


ISSN NO: 0776-3808


indian sign board detection and recognition for driving

Documents