spie proceedings [spie spie defense, security, and sensing - baltimore, maryland, usa (monday 29...

Human Gait Recognition by Pyramid of HOG feature onSilhouette Images

Guang Yang, Yafeng Yin, Jeanrok Park, Hong Man

ECE Department, Stevens Institute of Technology, Hoboken, NJ, 07030, USA

ABSTRACT

As a uncommon biometric modality, human gait recognition has a great advantage of identify people at adistance without high resolution images. It has attracted much attention in recent years, especially in thefields of computer vision and remote sensing. In this paper, we propose a human gait recognition frameworkthat consists of a reliable background subtraction method followed by the pyramid of Histogram of Gradient(pHOG) feature extraction on the silhouette image, and a Hidden Markov Model (HMM) based classifier.Through background subtraction, the silhouette of human gait in each frame is extracted and normalized fromthe raw video sequence. After removing the shadow and noise in each region of interest (ROI), pHOG featureis computed on the silhouettes images. Then the pHOG features of each gait class will be used to train acorresponding HMM. In the test stage, pHOG feature will be extracted from each test sequence and used tocalculate the posterior probability toward each trained HMM model. Experimental results on the CASIA GaitDataset B1 demonstrate that with our proposed method can achieve very competitive recognition rate.

1. INTRODUCTION

Human gait recognition, as to identify different people from their silhouette of walking styles, has been studiedin recent decades in computer vision research. As a biometric modality, gait recognition can be used to identifypeople at a distance without a high resolution image, which can be a great complementary to other popularbiometric technologies, such as face recognition. However, gait recognition remains to be a challenging problemdue to the variations of human appearances, viewing angles and scales.

Most previous works have been focused on the gait cycle modeling to identify different gait styles. Tocapture the dynamic uniqueness of each person’s walking style, different feature such as Energy Intensity,2

Moving Motion Silhouette Image(MMSI),3 Gait Entropy Image (GEnI) citeBashir2010, have been proposed.In addition, many classification methods, such as Hidden Markov Model (HMM),4,5 hierarchical structuralmodel,6 manifold learning7 are utilized to increase the robustness and effectiveness in gait recognition.

Reliably capturing the human silhouette dynamics is the key element for gait recognition. HOG feature8

has been proven as a robust feature for pedestrian detection. As a local description feature, HOG capturesthe detailed gradient variation of small image patches. This inspired us to adapt HOG feature to describe thevariation of human silhouette during a walking cycle. Recently, Bosch et.al9 extended HOG feature to a spatialpyramid representation. Pyramid HOG (pHOG) descriptor can effectively capture the shape feature for imageclassification. In this paper, we propose a pHOG feature for the human gait recognition framework. To extracthuman silhouette, background subtraction is applied on the human gait images. The human silhouette imagesare then segmented to binary images and normalized to the same size. Instead of computing HOG at one scale,the HOG feature at different scale will be calculated and a pyramid for human silhouette representation willbe constructed. The extracted pHOG features for each person will be used to train a Gaussian Mixture Model(GMM) based HMM for modeling the person’s gait cycle.

Compared with other features for gait recognition, the proposed pHOG feature on the binary silhouetteimage can effectively capture the shape characteristic of each person. As the human silhouette rather thanother texture features plays an essential role for gait recognition, the binary image representation accuratelycharacterizes the human body movement during each gait cycle, while the Pyramid of HOG provides an effective

Further author information: (Send correspondence to Guang Yang.)Guang Yang: E-mail: [email protected], Telephone: 1 201 216 8019

Optical Pattern Recognition XXIV, edited by David Casasent, Tien-Hsin Chao, Proc. of SPIE Vol. 8748, 87480J · © 2013 SPIE · CCC code: 0277-786X/13/$18 · doi: 10.1117/12.2015981

Proc. of SPIE Vol. 8748 87480J-1

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 10/03/2013 Terms of Use: http://spiedl.org/terms

spatial encoding approach to represent the human body shapes at different scale. Besdies its descriptiveness,the proposed pHOG feature also reduces the computational cost.

The rest of this paper is organized as follows: Section 2 will review the previous works on human gaitrecognition, Section 3 introduces the details of our proposed gait recognition framework. The experimentalresults will be presented in Section 4, and the paper is concluded in Section 5.

2. RELATED WORKS

Human gait recognition is also known as human recognition at a distance. Most previous gait recognitionalgorithms fall into two big categories: model-based or model-free methods. For model-based methods, HiddenMarkov Model,4,5 hierarchical structural model6 have demonstrated great effectiveness in gait recognition. Inthe model free category, a GEI feature was introduced for Gait recognition by Han,2 and the Gait EntropyImage (GEnI) was proposed to select the best feature for gait recognition without subject cooperation, whichmeans the probe data and training data don’t have to be under the same situation.10

Histogram of Oriented Gradient (HOG) feature is derived from SIFT feature, and it was introduced by Dalalet.al in 2005 for pedestrian detection. By implementing HOG feature with SVM classifier, the work in8 hasshown impressive results on human detection. To extract HOG features, the whole detection window is firstlydivided into grid of small cells. Similar to the cells for computing SIFT feature, the gradient in each cell is alsoaccumulated at different directions in a histogram and form a normalized vector description. Usually cells arethen integrated into blocks. Different from SIFT feature computation, blocks are overlapped with each otherin a sliding window fashion in HOG feature computation. The overlapping-block approach connects all thefeature descriptions in local regions. Bosch et.al9 extended HOG feature to a spatial pyramid representation.

3. FRAMEWORK OVERVIEW

The proposed Human gait recognition framework can be divided to three parts: human silhouette extraction,pHOG feature computation and HMM model training and testing.

3.1 Human silhouette extraction

Given an input video, the silhouette extraction process is shown in Figure 1 below. First, the backgroundimage is assumed to be known from the video as shown in Figure 2(a). Following the background subtractionapproach in,11 the raw silhouette images are obtained at this step, by subtracting the background image fromthe original frame. Then all the raw silhouette images will be refined in the shadow and noise removal process.

3.1.1 Background subtraction and Pixel classification

Each sequence is consist of one person and one sequential movement. The background scene is obtained from thebeginning of each sequence. Each pixel in the background image is modeled by < Ei, si, ai, bi >. The arithmeticmean and standard deviation is calculated. Ei is expected color value of pixel i in the background image forR, G, B channel, i.e. Ei = [μR(i), μG(i), μB(i)]; and and si is the standard deviation si = [σR(i), σG(i), σB(i)].

Then the background image is normalized at each R, G, B channel and horizontally aligned. Each imagewill be decomposed into brightness and chromaticity components. The measurement of brightness distortion(αi) and chromaticity distortion (CDi), which will be used for classifying each pixel are defined as below:

αi =

(IR(i)μR(i)

σ2R

(i)+ IG(i)μG(i)

σ2G

(i)+ IB(i)μB(i)

σ2B

(i)

)([

μR(i)σR(i)

]2

+[

μG(i)σG(i)

]2

+[

μB(i)σB(i)

]2)

CDi =

√√√√ ∑C=R,G,B

(IC(i) − αiμC(i)

σC(i)

)2



Each pixel M(i), in current frame

Calculate the difference betweenreference image and current image

Chromaticit Brightness

M(i) = Highlight

M(0= Foreground

M(0= Background

MO) = Shadow

Figure 1. Pixel classification flow chart

In the next step, the variation of the brightness distortion ai and chromaticity distortion bi on ith pixel isshown as below:

ai =

√∑Ni=0 (αi − 1)2

N

bi =

√∑Ni=0 (CDi)

2

N

Figure 2 shows an overview of the approach of background subtraction.

In the last step, the pixel classification is performed. As shown in Figure 2, each pixel is classified intoforeground, background, shadow or highlight by comparing the normalized brightness distortion α̂i = αi−1

αiand

normalized chromaticity distortion ˆCDi = CDi

biwith the thresholds. Some examples of background subtraction

results are show Figure 2.

3.1.2 Noise remove and silhouette extraction

After pixel classification, there are still holes or some isolated parts in human silhouette images. To connectthe isolated parts in the silhouette image, morphological operation is applied. After mathematical subtrac-tion the background image and foreground image with respect to the shadow, the human silhouette image isreconstructed with morphological operation dilation.

After morphological operation, all the images are normalized to the same size for feature extraction. Abounding box (size 150 × 200) is used to crop the human silhouette in each image. All the images with smallsize silhouettes will be discarded. Finally, the size normalize and horizontal alignment within bounding boxwill be performed and and only the silhouette images within the bounding box only will be kept. Some of theextracted silhouette images are shown in the Figure 3:



f

l>" ". ._...

A

1111E11111

(a) Background (b) Original frame (c) Pixel classification (d) Morphologicaldilation

(e) Final result

Figure 2. Human silhouette image extraction illustration. These are the human gait viewed in 54, 90 and 126 degreefrom the top row to the bottom, respectively.

Figure 3. Sampled human gait silhouette images

3.2 pHOG feature computation

HOG (Histogram of Oriented Gradient) is derived from SIFT feature,12 and it was introduced by Dalal et.al13 in2005. By implementing HOG feature with SVM classifier, it has shown impressive results on human detection.To extract HOG features, the whole detection window is firstly divided into grid of small cells. Similar tothe cells for computing SIFT feature, the gradient in each cell is also accumulated at different directions in ahistogram and form a normalized vector description. Pyramid HOG is an extension of HOG feature and itconsists of HOG features at different levels with different block sizes. The pyramid HOG feature is utilizedto describe the silhouette on all the binary images. In our experiment, the pyramid has two levels for eachsilhouette image and each block has 8 bins for directions, the total size of pHOG description is 648 for eachbinary silhouette image.

3.3 HMM model training and testing

To evaluate the extracted pHOG feature, a standard Gaussian Mixture Model based Hidden Markov Model(HMM) is applied for gait recognition. HMM have been widely used in the spatial-temporal dynamic recog-nition. Each subject’s pHOG feature sequences will be used to train its corresponding HMM model. Aftertraining, each subject will have a trained HMM model to characterize its gait cycle. Given a test gait sequencewith pHOG feature, its likelihood probability with regard to each trained HMM will be calculated and the onewith the maximum probability will be selected as the test result.



Table 1. Gait recognition with Different number of statesHMM state number Recognition rate

3 93.55%4 94.84%5 95.81%6 94.51%7 91.61%

0 18 36 54 72 90 108 126 144 162 180

60

70

80

90

100Gait Recognition on CASIA Dataset

0 ≤ Θ ≤ 180

Rec

ogni

tion

Rat

e

State =3State =4State =5State =6State =7

Figure 4. Sampled human gait silhouette images

4. EXPERIMENTAL RESULTS

4.1 Experiment on the CASIA

To evaluate the proposed framework, we conduct gait recognition experiments on the CASIA Gait DatasetB. In our experiments, we are using 31 subjects with 11 angles from 0◦ ∼ 180◦ and each has 6 sequences.Sequences includes 6 ∼ 8 walking cycles. Each sequence is divided into clips which contain one walking cyclein each.

Hidden Markov Models are used for classification in our framework, which are set with three Mixture ofGaussian in each state. The recognition rate vary with the different number of states, as we can observe fromFigure 4. The accuracies are very high and can even reach 100% in some cases. Accuracy is little less at 180◦

angle, which is the back view of a subject. The overall accuracy is shown in the Table 1. When Hidden MarkovModel with 5 states, our proposed framework has the best results. Results are based on 5-fold cross validation.

4.2 Comparison with other methods

As shown in the Table 2, while HMM with 5 states and 3 mixture of Gaussian, our proposed framework canachieved 95.81% recognition rate. Compared to MMSI method: 73.00% and GEI method: 63.33% under the

Table 2. Comparison with other methodOur method MMSI3 GEI2

95.33 73.00% 63.33%



same experimental setting, our proposed framework demonstrated clear superior performance in human gaitrecognition.

5. CONCLUSION

In this paper, a binary silhouette subtraction and pHOG features based human gait recognition frameworkis proposed. The Pyramid of HOG feature on the binary silhouette images effectively captures the shapedynamics during each gait cycle and the experimental results verified that the proposed framework can achievea competitive recognition rate in comparison with other existing human gait recognition methods. The proposedframework can be extended and applied to many gait recognition systems.

REFERENCES[1] Zheng, S., Zhang, J., Huang, K., He, R., and Tan, T., “Robust view transformation model for gait

recognition,” in [Image Processing (ICIP) ], 2073 –2076 (2011).[2] Han, J. and Bhanu, B., “Individual recognition using gait energy image,” in [IEEE Trans. Pattern Anal.

Mach. Intell. ], 28, 316–322 (Feb. 2006).[3] Nizami, I. F., Hong, S., Lee, H., Lee, B., and Kim, E., “Automatic gait recognition based on probabilistic

approach,” in [International Journal of Imaging Systems and Technolog ],[4] Chen, C., Liang, J., Zhao, H., and Hu, H., “Gait recognition using hidden markov model,”[5] Sarkar, S., Phillips, P., Liu, Z., Vega, I., Grother, P., and Bowyer, K., “The humanid gait challenge

problem: data sets, performance, and analysis,” in [IEEE Trans. Pattern Anal. Mach. Intell. ], 27, 162–177 (feb. 2005).

[6] Dockstader, S. L., Bergkessel, K. A., and Tekalp, A. M., “Feature extraction for the analysis of gait andhuman motion,” in [Proceedings of ICPR’02 Volume 1 - Volume 1 ], ICPR ’02, 10005– (2002).

[7] Cheng, M.-h., Ho, M.-f., and Huang, C.-l., “Gait analysis for human identification through manifoldlearning and hmm,” in [Pattern Recognition ], 41, 2541–2553 (2008).

[8] Dalal, N. and Triggs, B., “Histograms of oriented gradients for human detection,” in [CVPR’05 ], 2, 886–893 (June 2005).

[9] Bosch, A., Zisserman, A., and Munoz, X., “Representing shape with a spatial pyramid kernel,” in [Pro-ceedings of the ACM CIVR ],

[10] Bashir, K., Xiang, T., and Gong, S., “Gait recognition without subject cooperation,” in [Pattern Recog-nition Letters ], 31, 2052–2060 (October 2010).

[11] Horprasert, T., Harwood, D., and Davis, L. S., “A robust background subtraction and shadow detection,”in [In Proceedings of ACCV ], (2000).

[12] Lowe, D. G., “Object Recognition from Local Scale-Invariant Features,” in [The Proceedings of the SeventhIEEE International Conference on Computer Vision ],

[13] Dalal, N. and Triggs, B., “Histograms of oriented gradients for human detection,” in [IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition ], 2, 886–893 (2005).



spie proceedings [spie spie defense, security, and sensing - baltimore, maryland, usa (monday 29...

Documents