PhD Thesis

Biometrics

Science studying measurements and statistics of biological data Most relevant application: id.

recognition

2

Why Facial Biometrics ?

Most intuitive way of identification Socially and culturally accepted

worldwide It may work without collaboration

2006

43.6 %

19.2 %

2001

3

Facial Biometrics

Challenges ahead Less accurate than iris and

fingerprint Problems with uncontrolled

environments (illumination, viewpoint…) Best system

AverageFully automatic

4

Active Shape Models

Automatic training from examples

User-defined template based on landmarks

Model-based parametrization

Generative models5

T.F. Cootes, C. J, Taylor, D.H. Cooper, J. Graham (1995)Computer Vision and Image Understanding, 61(1):38–59

This thesis…

Focus on 3 contributions to ASMs on relevant aspects for facial feature localization: More accurate

segmentation invariant to in-plane rotations

Add robustness to out-of-plane rotations

Estimate the Reliability of the segmentation

1

2

3

4

6

ASM: Construction of the model

Face outlines based on landmarksShape statistics to learn spatial

relationsTexture statistics for image search

Landmarked Training Set

Local texture statistics

Shape statistics

PDM

IIMs

1

7

Point Distribution Model

1.- The input shapes are aligned to remove scale, translation and rotation effects.

1 1 2 2, , , , ... , 1TL L

i i i i i i ix y x y x y i N u

iv iui i i is u R v t

Image Coordinates Model Coordinates

11

8

Point Distribution Model2.- Principal Component Analysis (PCA) on the

aligned shapes

iijji

iT

i

j

N

i

T

ii

N

ii

diagS

NS

N

Φbuub

uuΦb

ΛΛΦΦ

uuuu

uu

)(

1

1

1

1

1

(2L)-space representationPCA-space

representation

11

9

Point Distribution Model (PDM)

iijji Φbuub

• Can determine valid shapes

• Can get closest valid shape

• Introduces a representation error

11

10

Point Distribution Model (PDM)

iijji Φbuub

More specific

More general

11

11

PDM: Modes of variation

Variation from 1st Principal Component

11

12

PDM: Modes of variation

Variation from 2nd Principal Component

11

13

ASM: Local Texture Statistics (1)

First order derivatives of the pixel intensity For each landmark Sampled perpendicularly to the contour

1:

k

kj

jigNormalized

i-th landmark

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

11

14

ASM: Local Texture Statistics (2) Second order statistics for each landmark

N

sisi

T

isii

N

ssii

N

N

1,,

1,

1

1

1

ggggG

gg

i-th landmark

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

11

15

ASM: Model Matching

1. The average shape is placed on the image, roughly matching the face position

11

16

2. Displacement of each landmark to minimize the Mahalanobis distance to the mean profil

3. Apply shape model restrictions

ASM: Model Matching

Steps 2 and 3 are repeated a fixed number of iterations at different resolutions, increasing detail

11

17

ASM: Model Matching

11

18

ttjjt

tht

Tt iterationt

Φbuub

uuΦb

1

),(

ASM: Model Matching

11

19

ASM: Complex textures

Several factors modify facial appearance beard, hair cut, glasses, teeth.

The distribution of the normalized gradient is often non Gaussian nor unimodal.

11

20

ASM: Complex textures

11

21

Optimal Features ASM

Texture description based on Taylor series

Grids centered at the landmarks for local analysis

Non linear classifier (kNN) for inside-outside labeling

0

00

0)(

)(!

)()(

xx

xxn

xfxf

n

n

inside

outside

1 2

22

B. van Ginneken, A.F. Frangi, J.J. Staal, B.M. ter Haar Romeny, and M.A. Viergever (2002)IEEE Transactions on Medical Imaging, 21(8):924–933