real-time detection, alignment and recognition of human faces

Real-Time Detection, Alignment Real-Time Detection, Alignment and Recognition of Human Facesand Recognition of Human Faces

Rogerio Schmidt FerisRogerio Schmidt Feris

Pattern Recognition Project

June 12, 2003June 12, 2003

OverviewOverview IntroductionIntroduction FERET DatasetFERET Dataset Face DetectionFace Detection Face AlignmentFace Alignment Face RecognitionFace Recognition ConclusionsConclusions

IntroductionIntroduction

Detection Alignment Recognition

IntroductionIntroduction Why this is a difficult problem?Why this is a difficult problem?

Facial Expressions, Illumination Changes, Pose, etc. Facial Expressions, Illumination Changes, Pose, etc.

Assumption: Frontal view facesAssumption: Frontal view faces

Objectives:Objectives: Develop a fully automatic system, suitable for real-time Develop a fully automatic system, suitable for real-time

applications.applications. Evaluate it on a large dataset.Evaluate it on a large dataset.

FERET DataSetFERET DataSet

1196 different individuals1196 different individuals

Probe Sets:Probe Sets:

FB: Different facial expressionsFB: Different facial expressions FC: Different illumination conditionsFC: Different illumination conditions DUP1: Different daysDUP1: Different days DUP2: Images taken at least 1 year afterDUP2: Images taken at least 1 year after

Face DetectionFace Detection

State-of-the-art: Learning-based approachesState-of-the-art: Learning-based approaches

Neural Nets [Rowley et al, PAMI 98]Neural Nets [Rowley et al, PAMI 98] SVMs [Heisele and Poggio, CVPR 01]SVMs [Heisele and Poggio, CVPR 01] Boosting [Viola and Jones, ICCV 01]Boosting [Viola and Jones, ICCV 01]

Want to know more?Want to know more?Detecting Faces in Images: a Survey [M. Yang, PAMI 02]Detecting Faces in Images: a Survey [M. Yang, PAMI 02]

Face DetectionFace Detection[Viola and Jones, 2001]

Simple features, which can be computed very fast.

A variant of Adaboost is used both to select the features and to train the classifier.

Classifiers are combined in a “cascade” which allows background regions of the image to be quickly discarded.

Face DetectionFace Detection

Time: 100ms (PIV 1.6Ghz)

Face AlignmentFace Alignment

State-of-the-art: Deformable ModelsState-of-the-art: Deformable Models

Bunch-Graph approach [Wiskott, PAMI 98]Bunch-Graph approach [Wiskott, PAMI 98]

Active Shape Models [Cootes, CVIU 95]Active Shape Models [Cootes, CVIU 95]

Active Appearance Models [Cootes, PAMI 01]Active Appearance Models [Cootes, PAMI 01]

Face AlignmentFace Alignment Active Appearance Model (AAM)Active Appearance Model (AAM)

Statistical Shape Model (PCA)

Statistical Texture Model (PCA)

AAM SearchAAM Search

Face AlignmentFace Alignment

Problem: Partial OcclusionProblem: Partial Occlusion

Active Wavelet Networks (AWN) Active Wavelet Networks (AWN) (submitted to BMVC’03)(submitted to BMVC’03)

Main idea: Replace AAM texture model by a wavelet networkMain idea: Replace AAM texture model by a wavelet network

Face AlignmentFace Alignment

Similar performance to AAM in images under normal conditions.

More robust against partial occlusions.

Face AlignmentFace Alignment

Using 9 wavelets, the system requires only 3 ms per Using 9 wavelets, the system requires only 3 ms per iteration (PIV 1.6Ghz). In general, at most 10 iterations are iteration (PIV 1.6Ghz). In general, at most 10 iterations are sufficient for good convergence.sufficient for good convergence.

Face RecognitionFace Recognition

State-of-the-art: Subspace TechniquesState-of-the-art: Subspace Techniques

PCA, FDA, Kernel PCA, Kernel FDA, ICA, etc.PCA, FDA, Kernel PCA, Kernel FDA, ICA, etc.

Want to know more?Want to know more?Face Recognition: a Literature Survey [W. Zhao, 2000]Face Recognition: a Literature Survey [W. Zhao, 2000]

Face RecognitionFace Recognition

www.cs.colostate.edu/evalfacerec/ www.cs.colostate.edu/evalfacerec/

PreprocessingPreprocessingLine up eyes, histogram equalization, maskingLine up eyes, histogram equalization, masking

Subspace Training (PCA)Subspace Training (PCA)

Classification (Nearest-neighbor)Classification (Nearest-neighbor)

Face RecognitionFace Recognition

Face RecognitionFace Recognition

Face RecognitionFace Recognition

Face RecognitionFace Recognition

ConclusionsConclusions An efficient, fully automatic system for face recognition was An efficient, fully automatic system for face recognition was

presented and evaluated.presented and evaluated. Future Work:Future Work: Alignment: multiresolution searchAlignment: multiresolution search View-based face recognitionView-based face recognition Explicit illumination modelExplicit illumination model Live demoLive demo

Face RecognitionFace Recognition

Face RecognitionFace Recognition

Face RecognitionFace Recognition