Gait Recognition

Guy Bar-henTal Reis

Introduction

• Gait – is defined as a “manner of walking”.

• Gait recognition – – is the term typically used to refer to the

automatic extraction of visual cues that characterize the motion of a walking person in video and is used for identification purposes.

Motivation

• Gait is particularly an attractive modality for passive surveillance since, unlike most biometrics, it can be measured at a distance, hence not requiring interaction with or cooperation of the subject.

Usage Examples

• Classify moving objects – person / non – person

• Biometric recognition of a person

Gait Recognition Methods

• Two main classes of gait recognition -– Holistic

• characterizes body movement by the statistics of the raw spatiotemporal patterns generated by the person’s motion [XYT].

– Feature-based• recovers explicit features (or parameters)

describing gait dynamics, such as stride dimensions and the kinematics, of joint angles.

Gait Recognition Using Image Self-Similarity

Chiraz BenAbdelkaderRoss G. CulterLarry S. Davis

Assumption

• People walk with constant velocity for about 3 – 4 seconds.

• People are located sufficiently far from the camera.

• The frame rate is greater than twice the frequency of the walking.

• The camera is stationary.

Method Outline

• The includes three main modules –– Preprocessing module - segmentation and

tracking of the moving person in each frame of a given sequence.

– Feature measurement – computing SSP from silhouette sequence,[scaling alignment] obtaining a set of normalized feature vectors.

– Pattern classification – identity is determined with standard classification techniques.

Method Overview

Preprocessing

• Segmentation is achieved via nonparametric background modeling or subtraction technique.

• Using cadence-based technique to determine whether a foreground blob corresponds to a moving person.

• Creating from N frame N silhouette templates.

Preprocessing

• Extract the blob region enclose with its bounding box either from –– Original color / grayscale image– Foreground image– Binary image

Feature measurementsilhouette template scaling

• The silhouette templates need to be first scaled to a standard size to normalize for depth variations.

Feature measurementsilhouette template scaling

• According to gait analysis the width and height of a person can be approximated as sinusoidal function –

noscillatioofamplitudeA

widthmeanm

gaitofphase

gaitoffrequency

w

w

nAnmnh

nAnmnw

hh

ww

sin)()(

sin)()(

Feature measurementsilhouette template scaling

• De-trend due to the changing camera depth and the differences between non-fronto / fronto parallel walking as –

• The templates have equal mean width and height.• The width-to-height aspect ratio need to remain

constant throughout the sequence.

nAnnhnh

nAnnwnw

hhh

www

sin)()()(ˆ

sin)()()(ˆ

Feature measurementsilhouette template scaling

• Final scaling so that the mean is equal to some given constant.

• Where H0 = 50 pixels

nAHH

nhnh hh

sin~

)(ˆ)(~

00

Feature measurementcomputing the self-similarity plot [SSP]

• In a periodic motion the SSP is also periodic, a good way to detect and characterize the periodic motion.

• Computation of SSP is done by –

iI Ith scaled template with size w(I) * h(I) in pixels

Feature measurementcomputing the self-similarity plot [SSP]

• Intersection = local minima of S• S encodes both the frequency and phase of gait cycle

Feature measurementNormalizing the SSP

• SSP is needed later to compare two different walking sequences that contain equal number of walking cycle starts at the same phase.

• SSU – self-similarity units.• The SSP can be viewed as a contiguous of SSU.• For gait recognition using only the SSU

corresponding to the left and right double-support poses.

• Scaling the SSU to uniform size m*m to be able to compare.– Double-support – cycle corresponds to when the feet are max apart.

Feature measurementNormalizing the SSP

SSU start at pose A and C

Feature measurementComputing the frequency and phase of the gait

• Frequency –– Apply autocorrelation

method on the SSP.– Smoothes the matrix of

SSP– Computes peaks– Find the best fitting regular

2D lattice and then the period obtained as the width of best fitting lattice

Feature measurementComputing the frequency and phase of the gait

• Phase –– Compute by locating the local minima of the SSP that

corresponds to A and C poses.

Pattern classification

• Definition – – Gallery – labeled SSU.– Probe – determine the person corresponding to a

set of novel SSU.

• Two steps –– Pattern matching using statistical pattern

classification, between each pair of probe.– Determine the probe’s correct class using KNN.

Results

• The methods was tested on 4 data set with low resolution video with recognition rate of 100% for fronto-paralle data set of 6 people and 70% for data set of 54 people.

Gait Analysis for Recognition and Classification

L. Lee W.E.L. Grimson

Gait appearance representation

• Canonical view - perpendicular to the direction of walk.

• The silhouette of the walker is segmented from the background

• Walker is located sufficiently far from the camera.

• The camera is stationary.

Gait Dynamic Feature Vector

• Ability to describe appearance at a level finer than whole body description without having to segment individual limbs.

• Robustness to noise in video foreground segmentation.

• And simplicity of representation.

Ellipses Representation

Ellipses Representation

(xi, yi) – The center of the ellipseli – The aspect ratio of major and minor axisi – The orientation of major axis

For each frame

• The frame feature vector of the j-th frame is

• In addition to these 28 features we also use h, the height of the center of mass.

For each Sequence

• The mean and standard deviation of region features across time.

• Magnitudes and phases of each region feature related to the dominant walking frequency.

Gait Average Appearance Feature Vector

))(),(),(( jjjjjj FstdFmeanhmeans

j = 1…last frame; s is 57-dimentional.

Gait Spectral Component Feature Vector

• Ωd is the dominant walking frequency of a given sequence.• t is 56-dimentional.

Are all features equally useful?

• A good feature should minimize within class variance and maximize interclass variance.

• Assuming that the features are independent from each other.

• The features can be ranked by p-value generated by analysis of variance (ANOVA).

Person Identification

• Gait data was gathered on 4 separated days with different backgrounds.

• 10 women, 14 men• 194 walking sequences.• A minimum of 3 complete walking cycles at 15

frames per second for each walking sequence.

Recognition using Averaged Appearance Feature

• Two gait recognition tests were performed using each of the two gait feature vectors

• p < 10-9 41 feature parameters for gait averaged appearance feature set, and 32 parameters for gait spectral components feature set.

Any-day test

• 100% correct identification for the 1st match, using the best 41 features.

• 97% for the 1st match and 100% by the 3rd match, using all 57 features.

Cross Day Test

• The gait recognition performance was highest when comparing data collected on day A against all other sequences.

Cross Day Test

• The average appearance gait features contain the mean shape of the 7 regions of the silhouettes. The shapes of these regions are very much affected by drastic changes in clothing style, such as pants vs. dress, shorts vs. pants.

Recognition using Spectral Components Features

• The gait spectral component features only contain the changes in shape and the time delay between the different regions, hence it is less affected by the change of clothing style.

Gender Classification

• Each of the 57 features was ranked based on the p-value of ANOVA in separating the genders and set a threshold of p < 10−9, which resulted in the best 6 features

Gender Classification

• SVM’s were trained using the 57 and the 6 gender features.

• Experimented with the linear, Gaussian and the 2nd degree polynomial kernels

• Random-person vs. Random-sequence

Gender Classification

Questions

Backup

Mahalanobis distanceIt is based on correlations between variables by which different patterns can be identified and analyzed. It is a useful way of determining similarity of an unknown sample set to a known one. It differs from Euclidean distance in that it takes into account the correlations of the data set and is scale-invariant, i.e. not dependent on the scale of measurements.

http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_mahalanobis.htm