A Brief Introduction on Face Detection

Mei-Chen Yeh04/06/2010

P. Viola and M. J. Jones, Robust Real-Time Face Detection, IJCV 2004.

Object Detection

• Find the location of an object if it appear in an image– Does the object appear?– Where is it?

Applications

Applications

George colony

Challenges 1: view point variation

Michelangelo 1475-1564 Slide (5-10) credit: Fei-Fei Li

Challenges 2: illumination

slide credit: S. Ullman

Challenges 3: occlusion

Magritte, 1957

Challenges 4: scale

Challenges 5: deformation

Xu, Beihong 1943

Challenges 6: background clutter

Klimt, 1913

Detection Framework

• Viola and Jones’ face-detection algorithm – The first object detection framework to provide

competitive object detection rates in real-time– Implemented in OpenCV

• Components– Features

• Haar-features • Integral image (speed up the feature calculation)

– Learning• AdaBoost algorithm

– Cascade method (speed up the detection process)

Haar-features

• The difference between pixels’ sum of the white and black areas

• Based on the face symmetry

Haar-features

• Too many features!– Different locations– Different scales

• Speed-up strategy– Fast calculation of haar-features– Selection of good features

24

24

Integral image

Sum of pixel values in the blue area

Example:

2 1 2 3 4 33 2 1 2 2 34 2 1 1 1 2

Image

2 3 5 8 12 155 8 11 16 22 289 14 18 24 31 39

Integral image

Integral image

1

3

2a b

c d

a = sum(1) b = sum(1+2) c = sum(1+3) d = sum(1+2+3+4)

Sum(4) = ?

4

d + a – b – cFour-point calculation!

A, B: 2 rectangles => C: 3 rectangles => D: 4 rectangles =>

6-point8-point9-point

A 24x24 detection window

Four types of haar features

Type A

The feature pool

Feature selection

• A very small number of features can be combined to from an effective classifier!

• Example: The 1st and 2nd features selected by AdaBoost

Feature selection

• A week classifier h

f1 f2

f1 < θ (a threshold) => Face!

f2 > θ (a threshold) => Not a Face!

h = 1 if fi < θ0 otherwise

Feature selection

• Idea: Combining several weak classifiers to generate a strong classifier

w1 w2

w3 wn

……

w1h1+w2h2+w3h3+…+wnhn>< T

a week classifierh1 = 1 or 0

~performance of the weak classifier on the training set

Feature selection

• Training Dataset– 4916 face images– non-face images cropped from 9500 images

non-face images

positive samples negative samples

AdaBoost

• Each training sample may have different importance!

• Focuses more on previously misclassified samples– Initially, all samples are assigned equal weights– Weights may change at each boosting round

• misclassified samples => increase their weights• correctly classified samples => decrease their weights

BoostingRound 1 + + + -- - - - - -

0.0094 0.0094 0.4623B1

= 1.9459

AdaBoost

decreased decreased increased

fi

Initial weights for each data point

OriginalData + + + -- - - - + +

0.1 0.1 0.1

-∞ ∞

misclassified

AdaBoost

……

Learning the classifier

• Initialize equal weights to training samples• For T rounds

– normalize the weights – select the best weak classifier in terms of the

weighted error– update the weights (set increased weights to

misclassified samples)• Linearly combine these T week classifiers to

form a strong classifier

Detection procedure• Scans the detector at multiple locations and scales

Sub-window

m

m

m

mn

n

n

n

Image

the detection window

Cascade method

• Most are non-face images!• Rejects negative sub-windows in an early stage

Strong Classifier = (W1xL1 + W2xL2) + (…)+ (…+ WnxLn) 1 2 3

>< T

Summary

Feature Extraction

Cascade Detection

Face Detection

BoostingRound 1 + + + -- - - - - -

0.0094 0.0094 0.4623B1

= 1.9459

Resources

• OpenCV Library• C codes

– apps/haartraining– samples/c

• Detectors– data/haarcascades

Assignment #3

• Due 04/20 11:59pm• Build a face detector using the OpenCV

resources• Send to TA

– A picture of yours with detected faces– Subject: Multimedia System Design-Assignment #3-your

student id-your name– File name: Assignment #3-your student id-your name

For Windows users, you might want to try an older version of OpenCV

Reminder

• Brainstorm report (a 2-page formal paper) is due on 4/20.