VEHICLE TYPE CLASSIFICATION BASED ON FRONTAL PROFILE
SANGKEETHA SUBRAMANIAM
UNIVERSITI TEKNOLOGI MALAYSIA
VEHICLE TYPE CLASSIFICATION BASED ON FRONTAL PROFILE
SANGKEETHA SUBRAMANIAM
A project report submitted in partial fulfilment of the
requirements for the award of the degree of
Master of Engineering (Electrical - Computer and Microelectronic System)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JUNE 2012
iii
Dedicated with love and thankfulness to
Amma, Appa, Kala, Mani, Patties.
To all family members,
Thanks for being there during our hardest times.
iv
ACKNOWLEDGEMENTS
In the process of completing my project, I have been assisted in many ways
by many generous individuals to whom I would like to convey my heartfelt gratitude.
First and foremost, I would like to express my appreciation to Assoc. Prof.
Dr. Syed Abdul Rahman Bin Syed Abu Bakar, my project supervisor who gave
generously of his time, wisdom and guidance. Besides providing assistance during
the completion of the project, he has also been motivating me to always be positive
and treat problems as challenges and learning process.
Tons of appreciation also to my family, especially my parents who have
never declined me any support be it emotionally and financially throughout my
education journey. My family have been very accepting to my ideas and kept
inspiring me besides pouring me the unconditional love that has been the drive in
pursuing my dreams.
I would like to convey my sublime acknowledgement to my friends and
colleagues who have enticed me by sparing some time to share their ideas that have
nurtured my thoughts in thoughts in the area of the project.
Gratitude is also extended to Faculty of Electrical Engineering, Universiti
Teknologi Malaysia and School of Graduate Studies (SPS UTM) for their support in
this project.
v
ABSTRACT
Vehicle recognition and classification is useful to many traffic management
activities and is one of the important technologies in Intelligent Transportation
System (ITS). Automatic toll collection, vehicle access control, traffic forecast and
volume, congestion management and speed monitoring are some of the systems that
have a need to classify the vehicle types. This project highlights the potential use of
vehicle dimensions and shape information for the vehicle type classification. The
system profiles the frontal view of the vehicles and extracts the basic outline shapes
and geometrical parameters and compares against the stored specification for each
class. The design utilizes Hough line detection algorithm to identify the location of
vehicle properties that needs to be extracted. Input to the system is a static image of
vehicle frontal view. The image is processed through various image processing
methods namely; median filtering, morphological reconstruction, edge detection and
Hough transform. Finally, the designed system is able to determine and classify
various vehicle types into three major classes; car, bus and truck. This design is a
useful support system to existing Plate Number Recognition System (PNRS) which
will help to increase the accuracy of the results obtained.
vi
ABSTRAK
Mengklasifikasi kenderaan amat berguna dalam kerja-kerja pengurusan trafik
dan ianya adalah salah satu teknologi penting dalam Sistem Pengangkutan Pintar
(ITS). Kutipan tol automatik, kawalan akses kenderaan, ramalan dan pengurusan
trafik dan kesesakan, dan pemantauan kelajuan adalah conthoh aplikasi yang
mempunyai keperluan untuk mengklasifikasi jenis kenderaan. Projek ini
merealisasikan penggunaan dimensi dan bentuk kenderaan untuk mengklasifikasikan
jenis kenderaan. Sistem ini memprofil pandangan hadapan kenderaan, dimana
parameter geometri dan bentuk asas kenderaan diekstrak untuk dibandingkan
denagan spefikasi bagi setiap kelas. “Hough transform” diaplikasikan untuk
mengenal pasti lokasi setiap paramter yang perlu diekstrak daripada imej kenderaan.
Selain itu, pelbagai kaedah pemprosesan imej yang lain juga digunakan dalam usaha
mengekstrak parameter geometri kenderaan. Sistem yang direka ini, mampu
mengenal pasti dan mengklasifikasi pelbagai jenis imej kenderaan kepada tiga kelas
utama; kereta, bas dan trak. Reka bentuk ini boleh digunakan sebagai sistem
sokongan kepada sistem pengenalan nombor plat (PNRS).
vii
TABLE OF CONTENTS
CHAPTER TITLE PAGE
DECLARATION ii
DEDICATION iii
ACKNOWLEDGEMENTS iv
ABSTRACT v
ABSTRAK vi
TABLE OF CONTENT vii
LIST OF TABLES x
LIST OF FIGURES xi
LIST OF ABBREVIATIONS xiii
LIST OF SYMBOLS xiv
1 INTRODUCTION 1
1.1 Project Background 1
1.2 Objective 3
1.3 The Motivation 3
1.4 Scope of the Project 4
1.5 Organization of Thesis 5
viii
2 LITERATURE REVIEW 6
2.1 Related Research Papers 6
2.1.1 Template Matching Method 7
2.1.2 Morphological Image Processing 8
Technique
2.1.3 Boosting Neural Network 8
Classifiers
2.1.4 Contour Feature Algorithm 9
2.1.5 Multi-Class and Single-Class 10
Classification Approaches
2.2 Hough Transform 10
2.3 Morphological Operations 13
2.4 Edge Detection 15
2.5 Median Filter 17
3 PROJECT METHODOLOGY 19
3.1 Overall System Design 19
3.2 Preprocessing 21
3.3 Object Segmentation 22
3.4 Feature Extraction 25
3.5 Image Recognition and Classification 31
4 RESULTS AND ANALYSIS 36
4.1 Test Samples 36
4.2 Results of Each Processing Steps 39
4.2.1 Preprocessing Results 39
4.2.2 Hough Transform and Line Detection 41
4.2.3 Image Segmentation for Width Extraction 41
4.2.4 Results of Feature Extraction 42
ix
4.2.5 Vehicle Recognition 43
4.2.6 Output Images of One Vehicle for 45
Each Class
4.3 Analysis and Discussion 47
4.3.1 Wiped Off Bottom Edge 48
4.3.2 Uncommon Geometrical Properties 50
4.3.3 Line Detection 53
4.3.3.1 Undetected Horizontal Line 53
4.3.3.2 Untrue Vehicle Property 54
4.4 Performance Analysis 55
5 CONCLUSION 58
5.1 Concluding Remark 58
5.2 Challenges and Solutions 59
5.3 Future Work 60
REFERENCES 62
x
LIST OF TABLES
TABLE NO. TITLE PAGE
3.1 Edge Based Geometrical Parameters 27
4.1 Mean and standard deviation values for each class based 37
on the test samples.
4.2 The ratio values computed for image v52.jpg and the 44
corresponding density value obtained from pdf of each
class.
4.3 Density values of each class for the obtained the 50
TBWR and WOHR.
4.4 Density Value of Each Class when TBWR =0.731. 51
4.5 Failure Types of the Unsuccessful Classification. 55
4.6 Performance Accuracy of the Vehicle Type Classification 56
System.
4.7 Grouped Similarity Percentage value for each successful 57
classification.
xi
LIST OF FIGURES
FIGURE NO. TITLE PAGE
2.1 ρand� interpretation of a line in xy-plane. 11
2.2 An example of ρ� − ��� subdivided into accumulator 12
cells.
2.3 The illustration of 2 points in xy –plane (representing non 13
zero pixels in an image) and the corresponding plot in
the ρ�-plane.
2.4 Erosion Morphological Operation applied on A Binary 14
Image
2.5 Dilation on a Binary Image with Vertical Line Structuring 15
Element.
2.6 Applying Sobel Operator for Edge Detection . 17
3.1 The Overall System Design Flow. 20
3.2 Image Processing Flow in Object Segmentation Stage. 22
3.3 Peaks of Hough Transform. 24
3.4 Image segmentation to obtain the top and bottom of the 25
input image.
3.5 The Extracted Dimensions for CAR. 26
3.6 The Extracted Dimensions for BUS. 26
3.7 The Extracted Dimension for TRUCK. 27
3.8 Height computation between Hough Lines. 29
3.9 Windscreen Height Computation. 30
3.10 Image Classification Process and Algorithm 35
xii
4.1 Normal pdf plot for WOHR, TBWR, and MHWR by 38
each class.
4.2 Resized images. 39
4.3 Resulting edge images for both resized images. 40
4.4 Vehicle images after clearing the background pixels. 40
4.5 Superimposed Hough lines on the image. 41
4.6 Cropped Image for top and bottom width extraction. 42
4.7 Image showing the geometrical interpretation of max 42
width and max height of the image.
4.8 Geometrical interpretation of windscreen height and the 43
overall height of the image.
4.9 Geometrical representation of bottom and top width of an 43
image.
4.10 Vehicle Type Classification System GUI display. 45
4.11 Car edge images for WOHR, TBWR and MHWR 46
computation.
4.12 Bus edge images for WOHR, TBWR and MHWR 46
computation.
4.13 Truck edge images for WOHR, TBWR and MHWR 47
computation.
4.14 Detected Edge for a bus image. 48
4.15 Image used for bottom width extraction. 49
4.16 Edge Image of a bus. 50
4.17 Truck Image. 51
4.18 Bus edge image with superimposed Hough lines. 52
4.19 Truck Image. 53
4.20 The location of the extracted windscreen height. 54
4.21 Car Image. 55
xiii
LIST OF ABBREVIATIONS
ITS - Intelligent Transportation System
MHWR - Maximum Height – Width Ratio
PNRS - Plate Number Recognition System
PDF - Probability Density Function
TBWR - Top-Bottom Width Ratio
WOHR - Windscreen-Overall Height Ratio
xiv
LIST OF SYMBOLS
� , � - Edge operator mask
S, s - Similarity percentage ����, �� - Median Filter Function ��� - Normal Probability Density Function (PDF) � - Angle between positive x-axis and the perpendicular
projection from the origin to the measured line in Hough
Transform � - Mean value � - Standard Deviation ρ - Distance from origin to a line along a vector perpendicular to
the line in Hough Transform
CHAPTER 1
INTRODUCTION
This project proposes the development of a software tool to determine and
classify vehicle type from its frontal profile. The software can be applied in the
traffic control and monitoring systems for the vehicle type recognition. In the first
chapter, the background of this project is discussed, providing the objective and
scope of the work.
1.1 Project Background
Vehicle type classification is one of the core technology applications in the
Intelligent Transportation System (ITS). ITS corresponds to the technology applied
in road infrastructure in order to improve the safety and efficiency of transportation.
Many countries have been involved in various researches in this field to overcome
the primary transportation issues. Malaysia has also developed the traffic
management system which includes incident detection and vehicle location system
that have been implemented in some of the main traffic roadways.
2
Vehicle recognition and classification is useful to many traffic management
activities. Automatic toll collection, vehicle access control, traffic forecast and
volume, congestion management and speed monitoring are some of the systems that
have a need to classify the vehicle types. These systems are designed for the purpose
of security, safety and reliability of transportation and vehicle classification is one of
the major components of these systems.
Vehicle type classification system can be grouped into two categories, the
sensor based system and the camera based system. The sensor based system
involves installation of inductive loop magnetic detectors onto the roadways.
Information on vehicle counts, distance between vehicles and the vehicle weights can
be obtained by analyzing the signal generated by the sensor. Applying sensor
approach for vehicle classification can be costly and sometimes the accuracy level is
low. Road congestion and high speed vehicle may cause inaccurate classification
depending on the sensitivity of the sensors. Besides, this system will be interrupted
during the road construction activities as the sensors are implanted onto the roads.
On the other hand, video camera based recognition systems are classified as
nonintrusive methods as there is no need for direct injection of any component onto
the road. Traffic cameras will be fixed on top of the roadways and the system will
not be disrupted during road construction works. The camera based vehicle
classification systems apply various image processing techniques in order to detect
and recognize the vehicle class.
This project highlights the potential use of vehicle dimensions and shape
information for the vehicle classification. In such a system, captured images will be
processed through few phases before the final shape and dimension can be extracted.
Accuracy of the extracted features is very vital for proper classification of vehicle
type. Hence, the applied image processing techniques will be one of the major
contributors to the accuracy of the system.
3
1.2 Objective
The project aims to design a vehicle type classification system based on the
frontal profile of the vehicle images. Classification is to be done, based on the
differences in the geometrical properties between the vehicle groups.
1.3 The Motivation
Accurate vehicle classification system is an important element in the measure
of enhancing road safety and transportation reliability. By integrating this system to
existing plate number recognition systems (PNRS) will further increase the accuracy
and the validity of the results obtained.
In some situations, PNRS may fail in providing the correct information due to
unsatisfactory image quality or falsified number plates. Such cases highlight the
need of a support system to increase the robustness of authentication. Vehicle type
classification can be applied as one of the support system to cross match the number
plate against the vehicle class which will directly help to increase the accuracy of the
system and indirectly prevent car theft, insurance frauds or other criminal activities.
The motivation behind this choice of developing a vehicle type classification
tool is to provide the traffic management and control systems with useful software
that can be applied in various ITS projects. The aim is to improve the safety and
security of transportation. Furthermore, the PNRS developers should be encouraged
by the evidence presented in this project in order to incorporate this system in their
new designs. In the same way, the users should be convinced with the obtained
4
results through a design that combines PNRS and the vehicle type classification
system.
1.4 Scope of the Project
In this project 3 types of vehicle classification will be considered. They are:
• CLASS 1: CAR
• CLASS 2: BUS
• CLASS 3: TRUCK
The project involves only software implementation in which MATLAB
software will be used for the system design. There is no real time image capturing in
this project. The set of vehicle images to be processed are stored and considered
available in the database prior to image classification. Input of the system is a static
vehicle frontal view image in query and the output of the system is the final vehicle
classification type. The design is targeted to process vehicle images taken in the day
time.
The system will analyze the vehicle frontal view and provide the
classification based on the highest similarity value in comparison to the
specifications set in the database for each class. The designed system will be an
offline tool. MATLAB GUI will be developed, providing users the ability to upload
an image in query and view the respective classification type.
5
1.5 Organization of the Project
This project write up is organized in five chapters. The first chapter
introduces the project background. The motivation, project objectives, scope and the
project organization are covered in this chapter.
Several previous works on the existing vehicle classification system and the
recognition methods are reviewed to provide background to the project. Chapter two
summarizes these literatures and provides some basic theory on various image
processing techniques which are relevant to this project.
Chapter three focuses on the methods and design procedures applicable in
developing the vehicle type classification system. These include the image
processing steps and techniques which will be used in this design. The methodology
and implementation plans which include preprocessing and 3 phases of image
processing is described in this chapter.
Chapter four provides the results obtained from the designed Vehicle Type
Classification System. This includes the results and discussion for each of the
important design phases, and analysis on the performance accuracy.
Finally, the fifth chapter concludes the project and recommendations for
future works are given here.
62
REFRENCES
Thiang Andre, Teguh Guntoro and Resmana Lim (2001). Type of Vehicle
Recognition Using Template Matching Method. Proc. of the International Conf. on
Electrical, Electronics, Communication, and Information. CECI’2001, March 7-8,
Jakarta.
Aisha Ajmal and Ibrahim M. Hussain (2010). Vehicle Detection Using
Morphological Image Processing Technique. MClT. MClT 2010.
Limin Xia (2006). Vehicle Recognition Using Boosting Neural Network Classifiers.
Proceedings of the 6th World Congress on Intelligent Control and Automation. June
21 - 23, 2006, Dalian, China.
Weihua Wang (2009). A Study on Contour Feature Algorithm for Vehicle Type
Recognition. 2009 International Joint Conference on Artificial Intelligence. DOI
10.1109/JCAI.2009.56.
Daniel T. Munroe and Michael G. Madden (2005). Multi-Class and Single-Class
Classification Approaches to Vehicle Model Recognition from Images.
National University of Ireland, Galway.
Mark S. Nixon, Alberto S. Aguado (2002). Feature Extraction and Image
Processing. (1st ed.) Wildwood Avenue, Woburn, MA.:Newnes.
Steven Lobregt and Max A. Viergever (1995). A Discrete Dynamic Contour Model.
IEEE Transactions on Medical Imaging. Vol. 14, No. I . March 1995.
Rafael C. Gonzalez and Richard E. Woods (2008). Digital Image Processing. (3rd
ed.) Pearson Education, Inc.: Prentice Hall.
Dr. Syed Abd. Rahman Al-Attas (2007). Advanced Digital Image Processing
(Teaching Module). Dept. of Microelectronic and Computer Engineering.