Human Gait Silhouettes Extraction Using Haar Cascade Classifier on OpenCV

Ahmad Puad Ismail1 and Nooritawati Md Tahir2

1Faculty of Electrical Engineering, Universiti Teknologi MARA (UiTM) Pulau Pinang, Permatang Pauh, Pulau Pinang, Malaysia.

2Faculty of Electrical Engineering, Universiti Teknologi MARA (UiTM) 40450, Shah Alam, Selangor DE, Malaysia

Corresponding Author: nooritawati@ieee.org

Abstract – Human Silhouette Extraction is one of the vital task in human abnormality gait detection. Hence in this study, extraction of human gait silhouette using Haar Cascade classifier is developed. This is done by detecting the region of interests (ROI) and by cropping the selected areas followed by background subtraction to eliminate unwanted background and further convert the images to silhouettes form based on human gait sequences acquired. Next, human abnormality gait detection is conducted based on front view markerless model. Classification results attained showed that the developed Haar cascade classifier is indeed suitable and robust in extracting front view gait features for abnormality gait detection with detection time of KNN surpass the SVM classifier. Keywords - Haar Cascade Classifier, Detection Time, Human Silhouette, Gait, Python and OpenCV

I. INTRODUCTION

Haar Cascade is one of the effective detection classifier

based on results attained by previous researches as mentioned in [1],[2],[3],[4],[5],[6]&[7]. For instance as reported in [1], enhancement of face detection achieved using Haar Cascade Classifiers that was based on skin color segmentation and reduction in execution time too. Further, Shujuan et al [2] developed a real time vehicle detection method using both Haar cascade classifier along with AdaBoost classifier and the algorithms are well validated using large database of both vehicles and non vehicles for training and testing. On the other hand, Goel & Agarwal [4] developed a robust gender classification technique using facial feature images based on Haar classifier and classification rate was over 98% and again reduction in computational time as well as suitability to be used in real time environment. Also, as mentioned in [8], Haar cascade was a rapid processing classifier in their research related to vehicle detection with good detection rates and proven suitable for computer vision based system. Consequently, an extension of Haar cascaded approach was also introduced by [10] specifically rotated Haar-like features for face detectors and boosting algorithms in related of performances and complex computational task.

Human detection are crucial factors in visual based surveillance system which discussed by many researchers as reported in [20], [21] and [22]. For instance, M. Paul et.al

[22] discussed the steps for human detection of gait motion in surveillance monitoring that involved background subtraction, optical flow and spatial-temporal filtering as human being classifier. Additionally, as mentioned by R. Poppe [21] that discussed characterizing of image sequences as human action label, is explored in visual surveillance, video retrieval and human computer interaction tasks. On the other hand, an example of gender recognition survey study in [20], robustness of face and whole body parts contributed to good features extraction with high classification rate in controlled environment.

Consequently, K-Nearest Neighbour (KNN) is one of the simplest classification methods as compared with other machine learning methods as mentioned by Z.Changjun and C.Yuzong in [23], and generally, it is an accurate classifier in most of situations. Other than that, KNN model is a well-known supervised learning algorithm which firstly introduced by [24] for pattern recognition task. On the other hand, Support Vector Machine (SVM) was utilized for multi-classification in [25] and SVM with radial Kernel function for abnormal gait classifications as elaborated in [26].

Additionally, tracking algorithms using Haar cascade has improved the performance among machine learning approach for visual object detection and recognition as reported in [12] and the proposed method capable to handle large database of images. Moreover, Haar classifier as reported in [13] was broadly used in celebrities related queries using images database that automatically provide information on the query person with facial image of the celebrity as inputs. Other than that, as described by Kumar et al. [14], AdaBoost with Haar cascade was utilised as biometrics based on the face detection along with Principal Component Analysis (PCA) and Latent Dirichlet Allocation (LDA) for the real time biometric system. Hierarchical face and eye detection based on Haar Cascade as classifier was explored by [15] too and reported promising findings. Therefore, in this study we deemed further to evaluate and validate the effectiveness of Haar Cascade classifier as human gait silhouette extraction and further for classification of normal and abnormal human gait.

 

2017 UKSim-AMSS 19th International Conference on Modelling & Simulation

978-1-5386-2735-8/17 $31.00 © 2017 IEEE

DOI 10.1109/UKSim.2017.25

105

II. METHODOLOGY

This section will detail and elaborate the proposed method in this study. As we are aware, extracting human gait silhouette from video sequences using image processing technique is a complex task. The two main challenges are robustness of the background subtraction algorithm and attaining perfection for foreground extraction. Figure 1 depicted the general overview of this study with the main focus is to investigate the effectiveness of Haar Cascade classifier for human gait silhouette extraction.

As depicted in Figure 1, human silhouette will be extracted from the video sequences captured by either digital camera or Raspberry Pi. Firstly, object tracking via Haar Cascade classifier is performed followed by locating the x and y coordinates of the region of interest [ROI] and cropping of silhouette image sequences prior to feature extraction and classification for anomalous gait detection.

Here, the Haar Cascade classifier is to be trained with positive and negative images. For this study, positive images are the human gait images irrelevant of different types of lighting or illuminance whilst negative images are images that without the presence of the positive image that is the human gait. In this study negative images are the background scene. Figure 2 depicted some samples of database for positive images as well as negative images used in this study. Next, the Haar classifier needs to be trained with suitable number of training images specifically the human gaits. Based on previous researches by [9], [11], [12] & [16] and using trial and error approach, it was found that for this study, 1500 images are suitable to train the Haar classifier.

The negative images acted as background subtraction samples to remove the arbitrary background within the positive images. This is to ensure that the classifier can detect the human gait accurately, thus reducing imperfect silhouette extraction. This is repeated for all other positive images to ensure that the object of interest namely the human gait is detected. Then, the gathered features or vector values are converted into a single vector file to train the Haar classifier since Haar recognised the vector values instead of images. This is because Haar classifier needs to be trained with the (extracted) feature or vector values instead of bitmap images. All the features or vector values from each single frame of the positive image are computed via the edge, line and centre features [7]. This is done by focusing those features in each image.

 Figure 1. General Overview of the proposed Human Gait Silhouette

Extraction.

Conversely, the steps involved in the Python program are as listed below:

Define Haar Cascaded classifier file that contain the feature vectors of the human gait;

Develop python file for detection of the human movement points remark on rectangular ROI;

Setting of an Open-CV environment and Python core program for feature extraction stage and classification using Support Vector Machine (SVM) and K-Nearest Neighbour (KNN);

106

Fig 2. Samples of Set of Positive Images (Top) and Set of Negative Images (Bottom) for training the Haar Cascade Classifier.

 

107

Note that the Python file developed is for detection of human movement, to locate the coordinate of ROI and to crop the human gait silhouette image in rectangular form based on the size of ROI as depicted in Figure 3. The Haar Cascade algorithm developed is able to identify mismatch area of ROI or if multiple ROI area detected. This is done via differential distance analysed computed for every multiple ROI generated prior to selection of the most optimum ROI. The next stage after image cropped is the front view based feature extraction and classification stages.

 

Fig 3. Human body area as region of interest Based on Figure 3, w and h are the expansion units of

the coordinates to create the region of interest. This is automatically generated for each human gait located during identifying the ROI and cropping.

Upon completion of silhouette gait extraction via Haar classifier, the silhouette will undergo feature extraction stage as described in [17] & [18]. Here six features of anatomical human body are evaluated to extract the gait features for modelling human walking gait. Based on front view markerless model as discussed in [19], four joint angles namely left thigh, left knee, right thigh and right knee are the feature inputs to the classifiers.

III. EXPERIMENTAL ANALYSIS AND DISCUSSION This section discussed the experimental results attained

based on the proposed method. Firstly, setting up for training and testing the Haar Cascade classifier need to be done as outlined below:

Define vector file to store feature vectors generated;

Define set of positive images – In this study 1500 sets of images of human gait;

Define set of negative images that is 1500 sets of images without the image of human gait that is the background scene;

Number of stages – The total number of Haar-training stages to be used.

Usually higher number of stages will reduce error but it

may also lead to decrease the sensitivity of Haar classifier in detecting the human gait. Next, upon completion of training the Haar Cascade Classifier, the features vectors of human gait is generated in a specific file. Further, the Haar classifier will be tested with both positive and negative

database totalling of 1500 for each case. Recall that appropriate detection of a human gait is vital in ensuring the ROI region coordinates are identified correctly.

Figure 4 depicted an example of ROI detection of a front view human gait as highlighted in the red box. As mentioned earlier, if more than one ROI are detected, the proposed shortest distance based Haar Cascade algorithm will select the most suitable ROI amongst them and identify the x and y coordinates of the ROI as well. Overall the Haar Cascade algorithm is able to identify the ROI of human gait followed by correct cropping of the human gait image. Next, the background subtraction is implemented to eliminate unnecessary background and as a result human gait silhouette is attained. Examples of human gait silhouettes extraction results are as depicted in Figure 5. On the contrary, to evaluate the effectiveness of the Haar Classifier, several more human gaits are also used as inputs. As shown, Figure 5 represented examples of normal and abnormal gait silhouette that are successfully extracted using the developed method.

Upon completion of silhouette extraction and feature extraction process, next is the detection stage using both KNN and SVM. Table 1 tabulated the accuracy rate for both classifiers along with the detection time.

TABLE 1 : OVERALL CLASSIFIER PERFORMANCE WITH HAAR

CASCADE AS HUMAN GAIT SILHOUETTE EXTRACTION Category Gait

Type Accuracy

(%) Detection Time

(sec)SVM KNN SVM KNN

Normal Normal Walking

84.2 89.5 28.22 14.06

Abnormal Tiptoed toe Drunken

84.2 84.2

89.5 89.5

39.20 32.56

18.4854.88

Results attained showed that accuracy rate for KNN and

SVM classifiers are similar for both normal and abnormal gait namely 89.5% for KNN and 84.2% for SVM respectively. In addition, KNN demonstrated higher recognition rate for both normal and abnormal gait. On the other hand, the detection time taken for detection of normal gait is shorter as compared to abnormal gait for both classifiers. Again, results attained showed that KNN required less detection time in identifying normal gait and abnormal gait as required by SVM.

IV. CONCLUSION In conclusion, a more robust human silhouette extraction

method using Haar Cascade classifier is developed. This is done via ROI identified and further cropped upon locating the ROI coordinates. The robustness of the Haar Cascade algorithm is the ability to identify variance in the ROI area or if multiple ROI detected and further capable to identify the most optimum ROI. Additionally, detection time during classification using KNN and SVM is reduced too. Future

108

work is to use the Haar Cascade classifier in real time environment.

Fig 4. Samples of ROI detection using Haar Cascade Classifier - Two ROI (Top); Three ROI (Middle) and single ROI (Bottom).

Fig 5. Example of human gait silhouette extraction based on ROI and further cropped: Top for normal gait and Bottom for abnormal gait

109

ACKNOWLEDGMENT This research is funded by Ministry of Higher Education

(MOHE) Malaysia under the Niche Research Grant Scheme (NRGS) Project No: 600-RMI/NRGS 5/3 (8/2013). The authors wished to thank Human Motion Gait Analysis (HMGA) Laboratory, IRMI Premier Laboratory, Research Management and Innovation (IRMI), Universiti Teknologi MARA (UiTM), Malaysia for the instrumentations and experimental facilities provided as well as Faculty of Electrical Engineering UiTM Shah Alam for all the support given during this research.

REFERENCES

[1] Érica K. Shimomoto, Almir Kimura and Ruan Belém, 'A faster face detection method combining Bayesian and Haar Cascade Classifiers', Proocedings of Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), pp:7-12, DOI: 10.1109/Chilecon.2015.7400344, 2015.

[2] Sun Shujuan, Xu Zhize, Wang Xingang, Huang Guan, Wu Wenqi and Xu De, 'Real-time vehicle detection using Haar-SURF mixed features and gentle AdaBoost classifier', Proceedings of 27th Chinese Control and Decision Conference (2015 CCDC),pp:1888-1894, 2015.

[3] Ali Sharifara, Mohd Shafry Mohd Rahim, and Yasaman Anisi, 'A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection'', Proceedings of 2014 International Symposium on Biometrics and Security Technologies (ISBAST), 2014.

[4] Priyanka Goel and Suneeta Agarwal, 'Hybrid Approach of Haar Cascade Classifiers and Geometrical Properties of Facial Features Applied to Illumination Invariant Gender Classification System', Proceedings of 2012 International Conference on Computing Sciences, pp 132-136, 2012.

[5] Sangkyun Park; Seonyoung Lee; Soojin Kim; Kyeongsoon Cho, 'Design of AdaBoost classifier circuit using Haar-like features for automobile applications', Proceedings of 2011 International SoC Design Conference, pp 262-265, 2011.

[6] Supratim Gupta, Anirban Dasgupta and Aurobinda Routray, ' Analysis of training parameters for classifiers based on Haar-like features to detect human faces', Proocedings of 2011 International Conference on Image Information Processing, pp:1-4, 2011.

[7] OpenCV 3.2.0 (Open Source Computer Vision), “Face Detection using Haar Cascades.” [Online]. Available: http://docs.opencv.org/trunk/d7/d8b/tutorial_py_face_detection.html. [Accessed: 16-Aug-2016].

[8] M. Oliveira and V. Santos, “Automatic Detection of Cars in Real Roads using Haar-like Features,” in 8th Portuguese Conference on Automatic Control, pp. 1–6, 2008.

[9] A. Robotics, C. Performance, and M. Vision, “Comparison Of Cascade Classifiers For Automatic Landing Pad Detection In Digital Images,” in 12th Brazilian Symposium on Intelligent Automation (SBAI), pp. 1017–1022, 2015.

[10] R. Lienhart, A. Kuranov, V. Pisarevsky, and M. R. L. T. Report, “Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection.” in Joint Pattern Recognition Symposium, pp. 297–304. 2003.

[11] V. Singh, V. Shokeen, and B. Singh, “Face Detection By Haar Cascade Classifier With Simple And Complex Backgrounds Images Using Opencv Implementation,” International Journal of Advanced Technology in Engineering and Science, vol. 1, no. 12, pp. 33–38, 2013.

[12] J. Paulo Menezes , Jos´e Carlos Barreto, Dias, “Face Tracking Based On Haar-Like Features And Eigenfaces,” in IFAC/EURON Symposium on Intelligent Autonomous Vehicles , pp. 1–6, 2004.

[13] M. J. M. Jadhav and M. D. V Gore, “Introducing Celebrities in an Images Using HAAR Cascade Algorithm,” in International Journal of Advanced Research in Computer Science and Software Engineering 3, vol. 8, pp. 571–577, 2013.

[14] K. Kumar, V. Semwal, and R. Tripathi, “Real time face recognition using adaboost improved fast PCA algorithm,” International Journal of Artificial Intelligence & Applications (IJAIA), vol. 2, no. 3, pp. 45–58, 2011.

[15] A. Schmidt and A. Kasiński, “The performance of the haar cascade classifiers applied to the face and eyes detection,” in Computer Recognition Systems 2, vol. 45, pp. 816–823, 2007.

[16] N. Seo, “Tutorial: Open CV Haar training (Rapid Object Detection With A Cascade of Boosted Classifiers Based on Haar-like Features).” [Online]. Available: http://note.sonots.com/SciSoftware/haartraining.html. [Accessed: 17-Aug-2016].

[17] A.P. Ismail, N. M. Tahir, and A. Hussain, “Frontal view gait analysis of gender,” in Control System, Computing and Engineering (ICCSCE), 2012 IEEE International Conference on, 2012, pp. 574–579.

[18] A.P. Ismail and N. Tahir, “Abnormal gait detection using Hexagonal method on Model based Front view model”, in Journal of Electrical Systems, 2015, vol 2, pp. 223-229.

[19] A.P. Ismail and N. Tahir, “Embedded platform performance of image based abnormal gait detection”, International Journal of Control Theory and Applications, 2016, vol 9, issue 6, pp 2977-2984.

[20]  C. B. Ng, Y. H. Tay, and B. Goi, “Recognizing Human Gender in Computer Vision : A Survey,” in 12th Pacific Rim International Conference on Artificial Intelligence, Kuching, Malaysia, 2012, pp. 335–346.

[21] R. Poppe, “A survey on vision-based human action recognition,” Image Vis. Comput., vol. 28, no. 6, pp. 976–990, Jun. 2010.

[22] M. Paul, S. M. E. Haque, and S. Chakraborty, “Human detection in surveillance videos and its applications - a review,” EURASIP J. Adv. Signal Process., vol. 2013, no. 1, p. 176, 2013.

[23] Z. Changjun and C. Yuzong, “The Research of Vehicle Classification Using SVM and KNN in a Ramp,” 2009 Int. Forum Comput. Sci. Appl., pp. 391–394, 2009.

[24] A. D. Number and N. E. W. L. Change, “Discriminatory analysis – Nonparametric discrimination: Small sample performance,” Proj. No. 21-49-004, Rep. No. 11, Contract No. AF 41(129)-31, USAF Sch. Aviat. Randolph Field, Texas., no. May, 1952.

[25] C. Meng, H. Bufu, and X. Yangsheng, “Intelligent shoes for abnormal gait detection,” in Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, 2008, pp. 2019–2024.

[26] C. Bauckhage, J. K. Tsotsos, and F. E. Bunn, “Detecting abnormal gait,” in Computer and Robot Vision, 2005. Proceedings. The 2nd Canadian Conference on, 2005, pp. 282–288.

110