Gait Recognitionby Deformable Registration

Yasushi Makihara1, Daisuke Adachi1, Chi Xu2,1, Yasushi Yagi1

1: The Institute of Scientific and Industrial Research, Osaka Univ.2: School of Computer Science and Technology, Nanjing University of Science and Technology

1

IEEE Computer Society Workshop on Biometrics 2018, Jun. 18th, 2018

Gait recognition: Overview

2Distance from sensor

DNA FaceFinger print IrisVein Gait

Near Far

Recognition at a distanceCriminal investigation

Gait recognition: Use casesAdmitted as evidence in courts for the first time UK in 2008 Japan in 2016

3

Gait verification systemfor criminal investigation [Iwama+ 2013]

[1] http://news.bbc.co.uk/2/hi/programmes/click online/7702065.stm, ”How biometrics could change security,” BBC News, 31 Oct. 2008.

Challenge of gait recognition

4Challenge: Intra-subject posture changes

Distracted walking is often seen in the real world.

Probe(inclined forward)

Gallery(normal)

Largedissimilarity

Falselyrejected

Related work

5

Combination of appearance-basedgait feature and metric learning

RankSVM[Martin-Felez+ PR2014]

Deformable registration model for gait recognition is required.

Robust gait recognition

Gait energy image (GEI)[Han+ TPAMI2006]

+

Not direct way to handle geometric deformation such as posture change

Registration model for face

w/ active shape model (ASM) [Thai+ IJBB2011]

Expression-invariant face recognition

w/ robust constrained local models [Boddeti+ 2017]

Outstanding landmarks such as eyes, nose, mouth, are unavailable for gait

Objective Gait recognition by deformable registration

6

Probe(Inclined forward)

Gallery(Normal)

Dissimilarity(Large)

Falselyrejected

Probe(Deformed)

Deformable registration model

Trulyaccepted

Dissimilarity(Small)

Free-form deformation (FFD)

Deformation vector on the control points (CPs)

Overall deformation field:Bilinear interpolation from adjacent CPs

7

Probe image Ip Deformed probe image F(Ip; u)

ui,j uij

Deformable registration

pi,jpi,j pijuijpi,jui,j

ui,j pi,j pijuij

Computing FFD between two images

8

Intra-/inter-subject FFDs

9

Intra-subject

Inter-subject

Forward inclination

Body shape

Translation

Head-torso ratio Arm swing Stride

Different trend Learn intra-subject deformation

# Deformation is represented by morphing.

Eigen FFD

10

Matching by eigen FFD

11

Metric learningConvolutional neural network:Matching local features at the bottom layer [Wu+ TPAMI 2016]

12

128

W1

16

122

W1’

16

122

128

Def

orm

ed p

robe

Gal

lery

16

122

W3

7

256

16

61

W2

Black digits: SizeBlue digits: #Channels

Green digits: Stride: Pooling

: Convolution filter

: Normalization

64

55 28

64

7 16

7 16

7 64 2 22 2256

22

W4

: Full connection w/ dropout

01

Experimental setup

13

Matching example

14

(a) Probe(inclined forward)

Fals

e m

atch

True

mat

ch

(b) Gallery

Dissimilarity score

1.42

1.31

0.60

Difference (a) – (b)

1.11

(a’) Deformed probe

Difference (a’) – (b)

(a) Probe(inclined forward)

Evaluation w/o metric learning

15

Receiver operating characteristics(ROC) curves

Better

Cumulative matching characteristics(CMC) curves

Better

False acceptance rate (FAR)

Fals

e re

ject

ion

rate

(FR

R)

Evaluation w/ metric learning

16

ROC curves CMC curves

Better

Better

Fals

e re

ject

ion

rate

(FR

R)

False acceptance rate (FAR)

Summary Gait recognition by deformable registration robust

against intra-subject posture change

Future work Handle more posture changes (e.g., climbing up stairs) Jointly optimize deformable registration and metric

learning 17

World’s largest gait database

18

Data set #Subjects Covariates

OUMVLP[Takemura+2017]

10,307

14 views

OULP-Bag[Uddin + 2018] 62,528

Carried objects in the wild

OULP-Age[Xu+ 2018] 63,846

Wide age range

World’s largest gait database Available at

http://www.am.sanken.osaka-u.ac.jp/BiometricDB/index.html

19