FACE RECOGNITION

CSE 402 - BIOMETRICS AND PATTERN RECOGNITION THOMAS SWEARINGEN

�1

CSE 402

OUTLINE

▸ Introduction

▸ Face Detection

▸ Matching Algorithms

▸ Principal Component Analysis (PCA)

▸ Linear Discriminant Analysis (LDA)

▸ Review

�2

CSE 402

WHAT IS THE FACE?

▸ Front portion of the human head

▸ Ranges from the forehead to the chin

▸ Includes mouth, nose, cheeks, and eyes

�3

CSE 402

HOW DO HUMANS RECOGNIZE FACES?

▸ Neuro-cognition & psychology indicates that certain parts of the human brain are geared towards perceiving the face

▸ Humans struggle to recognize inverted faces even though they can easily identify other inverted objects

▸ Humans likely perceive the face based on high-level characteristics (e.g., object with two eyes, nose, & mouth)

�4

Image from https://www.seventeen.com/celebrity/news/a38505/this-picture-of-adele-turned-upside-down-will-freak-you-the-freak-out/

CSE 402

TYPICAL FACE RECOGNITION PIPELINE

�5

CSE 402

FACE VARIATIONS

▸ Face images may vary in: ▸ Pose ▸ Illumination ▸ Expression ▸ Age

▸ Appearance may also change due to: ▸ Makeup ▸ Facial Hair ▸ Accessories (e.g.

Sunglasses)

▸ These variations make face recognition more difficult

�6

Images from Introduction to Biometrics, 2011.

CSE 402

FACE SENSING SPECTRUMS

▸ Face Recognition can be performed in different spectrums (e.g., Visible or Near-Infrared)

▸ Visible Spectrum Face Recognition is most common

�7

Image from Introduction to Biometrics, 2011.

CSE 402

2D AND 3D

2D

▸ May be a photo obtained from a typical camera

▸ Pixel intensities in a 2D grid (X & Y)

3D

▸ Adds depth information (Z)

▸ Requires special camera (or many 2D images)

�8

Images from https://i.pinimg.com/736x/84/89/66/8489665a096544f24e48ceadd7f2817a.jpg and https://s3.envato.com/files/233305038/Face%20Recognition%20Search%20590x.jpg

CSE 402

FACE IMAGE SENSORS

▸ General purpose cameras can be used for FR

▸ Some cameras can be used for FR, but typical use makes it more difficult

▸ E.G., surveillance camera captures a face at odd angle

▸ Some cameras designed specifically for FR

▸ Apple iPhoneX

▸ Bellus3D Face Camera

�9

Surveillance CameraApple iPhoneX FaceID

Bellus3D Face Camera

General Purpose Camera

Images from https://support.apple.com/en-us/ht208109, http://freepngimages.com/cctv-camera-transparent-background/, https://images-na.ssl-images-amazon.com/images/I/71Ml%2B1BrwQL._SX425_.jpg, and https://pisces.bbystatic.com/image2/BestBuy_US/images/products/8896/8896132_sa.jpg;maxHeight=460;maxWidth=460

CSE 402

TYPES OF FACE RECOGNITION

Identification (1-to-N)

▸ Given a probe, find a match (if it exists) among a set of face images

Verification (1-to-1)

▸ Given a face image and a claimed identity, match the given face image to known image of the claimed identity

�10

FACE DETECTIONCSE 402

�11

Image from https://d1mfcqjbhp6mmy.cloudfront.net/technology/panes/crowd-face-detection-sh.jpg

CSE 402

FACE DETECTION

▸ Localize face(s) in an image

▸ Detection generally provided as a rectangular bounding box

▸ Viola-Jones is a long-standing method

�12

Image from https://d1mfcqjbhp6mmy.cloudfront.net/technology/panes/crowd-face-detection-sh.jpg

CSE 402

VIOLA-JONES FACE DETECTOR

▸ A series of windows containing part of an image are subjected to filters

▸ The filters responses are input to a series of classifiers to determine if the window contains a face or not

▸ Viola-Jones is used for face detection, but it is a general object detection method

�13

CSE 402

HAAR-LIKE FILTERS

▸ No single filter can perform face detection

▸ A set of filters with varying shapes and sizes are used

▸ An integral image is used to reduce computation

▸ Result of filter is single value ▸ Σ (pixels in black area)

- Σ (pixels in white area)

�14

2-rectangle filters

3-rectangle filter 4-rectangle filter

CSE 402

INTEGRAL IMAGE

▸ Create integral image at position (x, y) by summing above and to the left:

▸ Use integral image to get sum of values in a rectangle by:

�15

A B

DC

I(x, y) = ∑x′ �≤x,y′�≤y

i (x′�, y′�)integral image

original image

Green Rectangle Sum = I(D) + I(A) − I(B) − I(C)

CSE 402

SKETCH OF FEATURE SELECTION (BOOSTING)

1.Create weak classifier for a Haar-like filter

2.for t = 1,2,…,T (a)Normalize weights (b)Compute weighted error for each weak

classifier (c)Select best weak classifier (d)Update weights

3.Construct Final Classifier from the T weak classifiers

�16

CSE 402

VIOLA-JONES CASCADE CLASSIFIER

▸ A series of strong classifiers

▸ The first classifier has low false negative rate and a high false positive rate

�17

CLASSIFIER 1

ALL WINDOWS

CLASSIFIER 2 CLASSIFIER 3 FURTHER PROCESSING

REJECT WINDOW

Face Detected

Face Detected

Face Detected

No Face Detected

No Face Detected

No Face Detected

Illustration adapted from Introduction to Biometrics, 2011.

CSE 402

VIOLA-JONES EXAMPLE

�18

https://youtu.be/L0JkjIwz2II

▸ Varying windows sizes, they get progressively smaller

▸ Some windows take less time to process as they are rejected earlier in the cascade pipeline

CSE 402

SOME APPROACHES FACE MATCHING

▸ Appearance-Based ▸ Principal Component Analysis ▸ Linear Discriminant Analysis

▸ Model-Based ▸ Elastic Bunch Graph Matching

▸ Texture-Based ▸ Local Binary Pattern (LBP)

▸ Deep Learning ▸ Convolutional Neural Network

�19

CSE 402

TERMINOLOGY

▸ Label: a specific attribute of a data sample (e.g., an image of a dog is labeled as a Labrador Retriever)

▸ Training Data: labeled examples used by a classifier to learn

▸ Test Data: data (separate from training data) used to evaluate classifier performance; a test label is compared to classifier predicted label to evaluate classifier performance

▸ Supervised Learning Method: a method that uses class labels of the training data to learn

▸ Unsupervised Learning Method: a method that doesn’t use the class labels of the training data to learn

�20

PRINCIPAL COMPONENT ANALYSIS (PCA)

FACE RECOGNITION

�21

CSE 402

PRINCIPAL COMPONENT ANALYSIS (PCA)

▸ Early automated face recognition method

▸ Goal: learn a subspace that accounts for as much variability in the training data as possible

▸ Does not use identity information during training (Unsupervised Learning)

�22

CSE 402

PCA DATA SETUP

1. Let denote each training sample

2. Compute average of the training set

3. Define the data matrix

�23

μ =1N

N

∑i=1

xi

x1, x2, …, xN

X = [(x1 − μ) (x2 − μ)⋯(xN − μ)]

d × 1 column vector

CSE 402

PCA SUBSPACE LEARNING

4. Calculate the covariance matrix

5. Compute the Eigenvectors of the covariance matrix by solvingwhere are the Eigenvectors

�24

C = XXT

CE = λE

d × d matrixd × N matrix N × d matrix

E = [e1, e2, …, ed]

CSE 402

PCA COMPARING FACES

1. Represent two unseen images, and , as a weighted sum of Eigenvectors

2. Compare difference of weighted sums to threshold (t)Faces are considered a match if the difference is less than the threshold, otherwise they are not a match

�25

xi

ωi = ET (xi − μ)

xj

ωj = ET (xj − μ)

ωi − ωj2 ≶ t

d × 1 vector

CSE 402 �26

Algorithm 1 Principal Component Analysis for Faces

Xtrain LoadTrainData() . Xtrain =⇥xtrain1 , . . . ,xtrain

N

⇤

Xtest LoadTestData() . Xtest = [xtest1 , . . . ,xtest

M ]

V ,µ Eigenfaces (Xtrain) . Learn Subspace

D CompareFaces (Xtest,V ,µ) . Obtain distances scores for test data

function EigenFaces(X)

µ 1N

PNI=1 xi . calculate mean

Xc X � [µ, . . . ,µ] . center data

C XcXTc . compute covariance matrix

V Eigenvectors(C) . Find Eigenvectors

return V ,µend function

function CompareFaces(X,V ,µ)⌦ = V T

(X � [µ, . . . ,µ]) . ⌦ = [!1, . . . ,!M ]

for each !i,!j pair in ⌦ do

Di,j = k!i � !jk2 . Compare test samples

end forreturn D

end function<latexit sha1_base64="A3eAinbxEF/50XQowRpaj0vKfmo=">AAAKEnicnVbPb9NIFHaBhSS7y1I4chlokIpUqriHBQ6VgJYIJH50oS2V6iSajCfJwHjGmhlXrYwvXOHIX7O3FVf+Af6Qve8bx61sZ9Ks8CGevBm/75tvvvfsYcyZNp3Oj6ULFy/9cvlKo9n69bffr/5xbfn6vpaJInSPSC7VwRBrypmge4YZTg9iRXE05PTd8MOWnX93RJVmUuyak5j2IjwWbMQINhAaLF/5NxjSMRMp5mOpmJlEWSsgOLaz6Y5igrAYc7Qlo1gKKgx6LDA/0UyjkVSoiwnV8EAtBSMQe2uwoagdDCUP9UkEt/QgGwSGHpvUKMxEhoIxNRpNQy8kDndteBsbnK3ebaMAQCOATM/LsRlwOjKH5RXH2cDvl9esoSCURsOtuupVv8pGsfHE9NqLuFNtHNQh+r+Z2wwLiNsl83i/7FeYnNG+4+Jtk67VAhsz9J+DLajelfacsaJZ26nBflblEkRJVYmnbEzFKDcFyve3es7RTYmX5XpBsRLobTLUMeRwn8P2KWIBWTDulkHRfNmrG3BvaIbX66EBwiiEcsQCcJAmUtFpCdisKLQnX9BtBd1EkLx+cjmmzOpWmOOysqIjhUnqZ+kriOgkGqTPN/2sn/+tHicrUSWYk4TbvBHFYoGVyRlWZRbdQ7PmtMychixp1isTgR+qKspUWWy5wUvU3PHC/LtZGQxMkAAAkUdYMXtGKMJGsWM39L7DtUeUGKl0FtxyeGjL5YouEyGqPHsKFryhJgEnLyweKLOnIjz1y6yBKuauW2ixleeZ7HVExzhDm7UEZ8q6y+jnbeEQbx6hWQCZz0ETc6IUsy/L3bsrVUoxmdR2XSxl7gzv2yjGTCEmnGKVpNwepGzt/Rnb4KMzH7NyuXAKpsHH/kZJj+Kkp+1E4yjm+YvVukOqc321PeMiKsLau7gSyVqt1uDaSmf9Ycd/+KePZgf+eie/Vrzi2hksL10C+UliyRKOtT70O7HppVgZRrjt1Ymm0LU/wHvkEIYCR1T30vwDJUN3IBLm7XIk4Rsij5afSHGk7XZgJdTtRNfnbNA1d5iY0YNeyoStfkGmQKOEIyOR/dqBlq2gNvkJDDCB3TOCyARDY4XeVEPh41jTBPYow+p2ysqBbqfioPmD/Y11v7Pu/7Wx8uhJoWDDu+nd9lY937vvPfKeeTvenkcaQeNT43PjS/Nr8+/mP81v06UXlopnbniVq/n9P8Jbrcg=</latexit><latexit sha1_base64="A3eAinbxEF/50XQowRpaj0vKfmo=">AAAKEnicnVbPb9NIFHaBhSS7y1I4chlokIpUqriHBQ6VgJYIJH50oS2V6iSajCfJwHjGmhlXrYwvXOHIX7O3FVf+Af6Qve8bx61sZ9Ks8CGevBm/75tvvvfsYcyZNp3Oj6ULFy/9cvlKo9n69bffr/5xbfn6vpaJInSPSC7VwRBrypmge4YZTg9iRXE05PTd8MOWnX93RJVmUuyak5j2IjwWbMQINhAaLF/5NxjSMRMp5mOpmJlEWSsgOLaz6Y5igrAYc7Qlo1gKKgx6LDA/0UyjkVSoiwnV8EAtBSMQe2uwoagdDCUP9UkEt/QgGwSGHpvUKMxEhoIxNRpNQy8kDndteBsbnK3ebaMAQCOATM/LsRlwOjKH5RXH2cDvl9esoSCURsOtuupVv8pGsfHE9NqLuFNtHNQh+r+Z2wwLiNsl83i/7FeYnNG+4+Jtk67VAhsz9J+DLajelfacsaJZ26nBflblEkRJVYmnbEzFKDcFyve3es7RTYmX5XpBsRLobTLUMeRwn8P2KWIBWTDulkHRfNmrG3BvaIbX66EBwiiEcsQCcJAmUtFpCdisKLQnX9BtBd1EkLx+cjmmzOpWmOOysqIjhUnqZ+kriOgkGqTPN/2sn/+tHicrUSWYk4TbvBHFYoGVyRlWZRbdQ7PmtMychixp1isTgR+qKspUWWy5wUvU3PHC/LtZGQxMkAAAkUdYMXtGKMJGsWM39L7DtUeUGKl0FtxyeGjL5YouEyGqPHsKFryhJgEnLyweKLOnIjz1y6yBKuauW2ixleeZ7HVExzhDm7UEZ8q6y+jnbeEQbx6hWQCZz0ETc6IUsy/L3bsrVUoxmdR2XSxl7gzv2yjGTCEmnGKVpNwepGzt/Rnb4KMzH7NyuXAKpsHH/kZJj+Kkp+1E4yjm+YvVukOqc321PeMiKsLau7gSyVqt1uDaSmf9Ycd/+KePZgf+eie/Vrzi2hksL10C+UliyRKOtT70O7HppVgZRrjt1Ymm0LU/wHvkEIYCR1T30vwDJUN3IBLm7XIk4Rsij5afSHGk7XZgJdTtRNfnbNA1d5iY0YNeyoStfkGmQKOEIyOR/dqBlq2gNvkJDDCB3TOCyARDY4XeVEPh41jTBPYow+p2ysqBbqfioPmD/Y11v7Pu/7Wx8uhJoWDDu+nd9lY937vvPfKeeTvenkcaQeNT43PjS/Nr8+/mP81v06UXlopnbniVq/n9P8Jbrcg=</latexit><latexit sha1_base64="A3eAinbxEF/50XQowRpaj0vKfmo=">AAAKEnicnVbPb9NIFHaBhSS7y1I4chlokIpUqriHBQ6VgJYIJH50oS2V6iSajCfJwHjGmhlXrYwvXOHIX7O3FVf+Af6Qve8bx61sZ9Ks8CGevBm/75tvvvfsYcyZNp3Oj6ULFy/9cvlKo9n69bffr/5xbfn6vpaJInSPSC7VwRBrypmge4YZTg9iRXE05PTd8MOWnX93RJVmUuyak5j2IjwWbMQINhAaLF/5NxjSMRMp5mOpmJlEWSsgOLaz6Y5igrAYc7Qlo1gKKgx6LDA/0UyjkVSoiwnV8EAtBSMQe2uwoagdDCUP9UkEt/QgGwSGHpvUKMxEhoIxNRpNQy8kDndteBsbnK3ebaMAQCOATM/LsRlwOjKH5RXH2cDvl9esoSCURsOtuupVv8pGsfHE9NqLuFNtHNQh+r+Z2wwLiNsl83i/7FeYnNG+4+Jtk67VAhsz9J+DLajelfacsaJZ26nBflblEkRJVYmnbEzFKDcFyve3es7RTYmX5XpBsRLobTLUMeRwn8P2KWIBWTDulkHRfNmrG3BvaIbX66EBwiiEcsQCcJAmUtFpCdisKLQnX9BtBd1EkLx+cjmmzOpWmOOysqIjhUnqZ+kriOgkGqTPN/2sn/+tHicrUSWYk4TbvBHFYoGVyRlWZRbdQ7PmtMychixp1isTgR+qKspUWWy5wUvU3PHC/LtZGQxMkAAAkUdYMXtGKMJGsWM39L7DtUeUGKl0FtxyeGjL5YouEyGqPHsKFryhJgEnLyweKLOnIjz1y6yBKuauW2ixleeZ7HVExzhDm7UEZ8q6y+jnbeEQbx6hWQCZz0ETc6IUsy/L3bsrVUoxmdR2XSxl7gzv2yjGTCEmnGKVpNwepGzt/Rnb4KMzH7NyuXAKpsHH/kZJj+Kkp+1E4yjm+YvVukOqc321PeMiKsLau7gSyVqt1uDaSmf9Ycd/+KePZgf+eie/Vrzi2hksL10C+UliyRKOtT70O7HppVgZRrjt1Ymm0LU/wHvkEIYCR1T30vwDJUN3IBLm7XIk4Rsij5afSHGk7XZgJdTtRNfnbNA1d5iY0YNeyoStfkGmQKOEIyOR/dqBlq2gNvkJDDCB3TOCyARDY4XeVEPh41jTBPYow+p2ysqBbqfioPmD/Y11v7Pu/7Wx8uhJoWDDu+nd9lY937vvPfKeeTvenkcaQeNT43PjS/Nr8+/mP81v06UXlopnbniVq/n9P8Jbrcg=</latexit><latexit sha1_base64="A3eAinbxEF/50XQowRpaj0vKfmo=">AAAKEnicnVbPb9NIFHaBhSS7y1I4chlokIpUqriHBQ6VgJYIJH50oS2V6iSajCfJwHjGmhlXrYwvXOHIX7O3FVf+Af6Qve8bx61sZ9Ks8CGevBm/75tvvvfsYcyZNp3Oj6ULFy/9cvlKo9n69bffr/5xbfn6vpaJInSPSC7VwRBrypmge4YZTg9iRXE05PTd8MOWnX93RJVmUuyak5j2IjwWbMQINhAaLF/5NxjSMRMp5mOpmJlEWSsgOLaz6Y5igrAYc7Qlo1gKKgx6LDA/0UyjkVSoiwnV8EAtBSMQe2uwoagdDCUP9UkEt/QgGwSGHpvUKMxEhoIxNRpNQy8kDndteBsbnK3ebaMAQCOATM/LsRlwOjKH5RXH2cDvl9esoSCURsOtuupVv8pGsfHE9NqLuFNtHNQh+r+Z2wwLiNsl83i/7FeYnNG+4+Jtk67VAhsz9J+DLajelfacsaJZ26nBflblEkRJVYmnbEzFKDcFyve3es7RTYmX5XpBsRLobTLUMeRwn8P2KWIBWTDulkHRfNmrG3BvaIbX66EBwiiEcsQCcJAmUtFpCdisKLQnX9BtBd1EkLx+cjmmzOpWmOOysqIjhUnqZ+kriOgkGqTPN/2sn/+tHicrUSWYk4TbvBHFYoGVyRlWZRbdQ7PmtMychixp1isTgR+qKspUWWy5wUvU3PHC/LtZGQxMkAAAkUdYMXtGKMJGsWM39L7DtUeUGKl0FtxyeGjL5YouEyGqPHsKFryhJgEnLyweKLOnIjz1y6yBKuauW2ixleeZ7HVExzhDm7UEZ8q6y+jnbeEQbx6hWQCZz0ETc6IUsy/L3bsrVUoxmdR2XSxl7gzv2yjGTCEmnGKVpNwepGzt/Rnb4KMzH7NyuXAKpsHH/kZJj+Kkp+1E4yjm+YvVukOqc321PeMiKsLau7gSyVqt1uDaSmf9Ycd/+KePZgf+eie/Vrzi2hksL10C+UliyRKOtT70O7HppVgZRrjt1Ymm0LU/wHvkEIYCR1T30vwDJUN3IBLm7XIk4Rsij5afSHGk7XZgJdTtRNfnbNA1d5iY0YNeyoStfkGmQKOEIyOR/dqBlq2gNvkJDDCB3TOCyARDY4XeVEPh41jTBPYow+p2ysqBbqfioPmD/Y11v7Pu/7Wx8uhJoWDDu+nd9lY937vvPfKeeTvenkcaQeNT43PjS/Nr8+/mP81v06UXlopnbniVq/n9P8Jbrcg=</latexit>

CSE 402

VISUALIZING PCA EIGENVECTORS

▸ Train on 4,000 aligned images from the Oxford VGG Face dataset

▸ Obtain the Eigenvectors corresponding to the 4 largest Eigenvalues

▸ Called Eigenfaces

▸ Reshape from 1D vector to 2D image

�27

Mean

Largest Eigenvalue

2nd Largest Eigenvalue

3rd Largest Eigenvalue

4th Largest Eigenvalue

CSE 402

COMPRESSED REPRESENTATION

▸ Not all Eigen vectors needed to adequately represent the data

▸ Eigenvectors with small Eigenvalues mostly model noise rather than discriminant info relating to identity

▸ Represent data using only Eigenvectors

�28

5th Smallest Eigenvalue

4th Smallest Eigenvalue

3rd Smallest Eigenvalue

2nd Smallest Eigenvalue

Smallest Eigenvalueωi = ET* xd* × 1 vector

(d* < d)

E* is d × d*

d*

LINEAR DISCRIMINANT ANALYSIS (LDA)

FACE RECOGNITION

�29

CSE 402

LINEAR DISCRIMINANT ANALYSIS (LDA)

▸ Unlike PCA, LDA makes use of identity information (Supervised Learning)

▸ Goal: learn a subspace that minimizes intra-class variation and maximizes inter-class variation

▸ Images from the same subject are closer together

▸ Images from different subjects further apart

�30

CSE 402

PCA VS LDA

�31

C1 C2

= Principal Axis

PCA LDAPrincipal axis follows the direction with the most

variation for all data.

Principal axis follows the direction with the most class variation.

Illustration adapted from Introduction to Biometrics, 2011.

CSE 402

LDA INITIALIZATION

1. Let be the training set where is the data and is the class label

2. Compute mean for each class (identity)where is the number of samples with class

�32

(x1, y1), (x2, y2), …, (xN, yN)xi yi = {1,…, C}

μc =1Nc ∑

{i | yi=c}xi

Ncc

CSE 402

LDA SCATTER MATRICES

3. Use training data to calculate between class and within class scatter matrices where (mean of all data)

�33

Sw =C

∑c=1

∑{i | yi=c}

(xi − μc) (xi − μc)T

Sb =C

∑c=1

Nc (μc − μ) (μc − μ)T

μ =1N

N

∑i=1

xi

within class

between class

CSE 402

LDA SUBSPACE

4. Calculate Eigenvectors by solving

5. Represent any data vector by

6. Compare representations like in PCA (norm of difference)

�34

S−1w SbE = λE

ωi = ET (x − μ)d × 1 vector

x

minimized

maximized

CSE 402 �35

Algorithm 1 Linear Discriminant Analysis for Faces

Xtrain,ytrain LoadTrainData() . ytrain =⇥ytrain1 , . . . , ytrainN

⇤

Xtest LoadTestData()V ,µ Fisherfaces (Xtrain,ytrain) . learn subspaceD CompareFaces (Xtest,V ,µ) . obtain distances scores for test data

function FisherFaces(X,y)for each c 2 [1, . . . , C] do

Nc | {i|yi = c} | . num. samples with class cµc 1

Nc

P{i|yi=c} xi . calculate class mean

end forµ 1

N

PNi=1 xi . calculate overall mean

Sw =PC

c=1

P{i | yi=c} (xi � µc) (xi � µc)

T

Sb =PC

c=1 Nc (µc � µ) (µc � µ)T

V Eigenvectors�S�1

w Sb

�. find Eigenvectors

return V ,µend function

function CompareFaces(X,V ,µ)⌦ = V T (X � [µ, . . . ,µ]) . ⌦ = [!1, . . . ,!M ]for each !i,!j pair in ⌦ do

Di,j = k!i � !jk2 . compare test samplesend forreturn D

end function<latexit sha1_base64="cE+o3aSCEDnOXZ+jom3i4vYnobY=">AAALD3icnVbNbttGEKaTtLXYn8TpsZdNrQIO4AiiD22DwEBaK0IPaZo2tuPClIjlciVtstwldpdJBJrItT32aXoreu0j9DX6BB3+SOWf7La8aDXLme+bb2eG60ecaTMc/rl17fqNd959b7tnv//Bhx/dvLVz+1TLWBF6QiSX6szHmnIm6IlhhtOzSFEc+pw+918eZfvPX1GlmRTHZhnRSYjngs0YwQZM3s72X65P50wkmM+lYmYRprZLcJTtJo8hKFZoxDRRLGQCC4O+EpgvNdNoJhUaY0I1ODRCMAK2ZwYbivquL3mglyH8JGep5xr6xiRGYSbSfVTdXNY3kTunRqPC9Fji4Dgzj7DB6d7dPnKPZBhSYZL+JTEOXU5n5nzpOdMGbiCN3kdL78m0jqnYfGEm/avoU206CIJ1xa/T/7SRsRvG9ShjphdUzXJRUc597//KV2RSFYrDUQqkY19HANCd4WjFpyQEvhFWdFxlhDZLUmfUnW6Ll/QNEEYB1DoWgIM0kYoW9ZVFRUGmaUnXdsexIHlxFmIV1JLGOTWlyQ50LFVCMVmgPkEuABa1gZx1NRx1HP8Tj5SKXBQOboIYrJceQ4eIlB5uCqZKSiIOB0jjMOKQx2voCEQ41hqQN1RWJs0ayZ0pTBInTQAcBNNx6CWXYae1ZN+kHqtQIZiTmGeABYWQYgEcHokA9NjIpcVkxYMdOuk0//tvICVMHsz5CrQD7FnqvYZkiuAkC35Uz5gh98GF+2CVdCXnVntkNNC9Vsl5pdPd/+5RDofjtI82sfeb7POKaSGVB9zG2sjtKo+ruJ3WR8sjNqfiFSVGKp26dzpaOTuKaXLPSZtWH3V07YyJANWCrli4P1ATw6S5cvT1izIs+7nd4LXhc3mHd4bf0GrfhXSO0+zYagHWenaPuewg8oHRhFmPjw2jbtIh3iZCbQCZ73lON0q5+211cP0z5rpeZd0RXvRRhJlCTHSKVZFyBDNg/8WarXvRGa/VVSuc1dC6mB5UJ0Zx0sW4L+dma0h11tWoVUVUBI2LSM2S2rbt3dodDu4PnfufO6i9cAbD/Nm1yuept7N1A+QncUY2H6PnzjAykwQrwwjPvqWxpvBVfYnn9ByWAodUT5L8dpaiz8AS5J+zmYQLVG6teiQ41Fk68GaIzUI39zJj1955bGZfThImothQQQqgWcyRkSi76sEnVUFv8iUsMFzggCsiCwwj3cCFsI7C55GmMeQog3o6VeVAt5U4aPPi9GDgDAfO9we7D78uFdy2PrE+tfYsx/rCemh9Yz21TizS+7H3tvdT72f7F/tX+zf79+LVa1ulz8dW7bH/+BuV+wLt</latexit><latexit sha1_base64="cE+o3aSCEDnOXZ+jom3i4vYnobY=">AAALD3icnVbNbttGEKaTtLXYn8TpsZdNrQIO4AiiD22DwEBaK0IPaZo2tuPClIjlciVtstwldpdJBJrItT32aXoreu0j9DX6BB3+SOWf7La8aDXLme+bb2eG60ecaTMc/rl17fqNd959b7tnv//Bhx/dvLVz+1TLWBF6QiSX6szHmnIm6IlhhtOzSFEc+pw+918eZfvPX1GlmRTHZhnRSYjngs0YwQZM3s72X65P50wkmM+lYmYRprZLcJTtJo8hKFZoxDRRLGQCC4O+EpgvNdNoJhUaY0I1ODRCMAK2ZwYbivquL3mglyH8JGep5xr6xiRGYSbSfVTdXNY3kTunRqPC9Fji4Dgzj7DB6d7dPnKPZBhSYZL+JTEOXU5n5nzpOdMGbiCN3kdL78m0jqnYfGEm/avoU206CIJ1xa/T/7SRsRvG9ShjphdUzXJRUc597//KV2RSFYrDUQqkY19HANCd4WjFpyQEvhFWdFxlhDZLUmfUnW6Ll/QNEEYB1DoWgIM0kYoW9ZVFRUGmaUnXdsexIHlxFmIV1JLGOTWlyQ50LFVCMVmgPkEuABa1gZx1NRx1HP8Tj5SKXBQOboIYrJceQ4eIlB5uCqZKSiIOB0jjMOKQx2voCEQ41hqQN1RWJs0ayZ0pTBInTQAcBNNx6CWXYae1ZN+kHqtQIZiTmGeABYWQYgEcHokA9NjIpcVkxYMdOuk0//tvICVMHsz5CrQD7FnqvYZkiuAkC35Uz5gh98GF+2CVdCXnVntkNNC9Vsl5pdPd/+5RDofjtI82sfeb7POKaSGVB9zG2sjtKo+ruJ3WR8sjNqfiFSVGKp26dzpaOTuKaXLPSZtWH3V07YyJANWCrli4P1ATw6S5cvT1izIs+7nd4LXhc3mHd4bf0GrfhXSO0+zYagHWenaPuewg8oHRhFmPjw2jbtIh3iZCbQCZ73lON0q5+211cP0z5rpeZd0RXvRRhJlCTHSKVZFyBDNg/8WarXvRGa/VVSuc1dC6mB5UJ0Zx0sW4L+dma0h11tWoVUVUBI2LSM2S2rbt3dodDu4PnfufO6i9cAbD/Nm1yuept7N1A+QncUY2H6PnzjAykwQrwwjPvqWxpvBVfYnn9ByWAodUT5L8dpaiz8AS5J+zmYQLVG6teiQ41Fk68GaIzUI39zJj1955bGZfThImothQQQqgWcyRkSi76sEnVUFv8iUsMFzggCsiCwwj3cCFsI7C55GmMeQog3o6VeVAt5U4aPPi9GDgDAfO9we7D78uFdy2PrE+tfYsx/rCemh9Yz21TizS+7H3tvdT72f7F/tX+zf79+LVa1ulz8dW7bH/+BuV+wLt</latexit><latexit sha1_base64="cE+o3aSCEDnOXZ+jom3i4vYnobY=">AAALD3icnVbNbttGEKaTtLXYn8TpsZdNrQIO4AiiD22DwEBaK0IPaZo2tuPClIjlciVtstwldpdJBJrItT32aXoreu0j9DX6BB3+SOWf7La8aDXLme+bb2eG60ecaTMc/rl17fqNd959b7tnv//Bhx/dvLVz+1TLWBF6QiSX6szHmnIm6IlhhtOzSFEc+pw+918eZfvPX1GlmRTHZhnRSYjngs0YwQZM3s72X65P50wkmM+lYmYRprZLcJTtJo8hKFZoxDRRLGQCC4O+EpgvNdNoJhUaY0I1ODRCMAK2ZwYbivquL3mglyH8JGep5xr6xiRGYSbSfVTdXNY3kTunRqPC9Fji4Dgzj7DB6d7dPnKPZBhSYZL+JTEOXU5n5nzpOdMGbiCN3kdL78m0jqnYfGEm/avoU206CIJ1xa/T/7SRsRvG9ShjphdUzXJRUc597//KV2RSFYrDUQqkY19HANCd4WjFpyQEvhFWdFxlhDZLUmfUnW6Ll/QNEEYB1DoWgIM0kYoW9ZVFRUGmaUnXdsexIHlxFmIV1JLGOTWlyQ50LFVCMVmgPkEuABa1gZx1NRx1HP8Tj5SKXBQOboIYrJceQ4eIlB5uCqZKSiIOB0jjMOKQx2voCEQ41hqQN1RWJs0ayZ0pTBInTQAcBNNx6CWXYae1ZN+kHqtQIZiTmGeABYWQYgEcHokA9NjIpcVkxYMdOuk0//tvICVMHsz5CrQD7FnqvYZkiuAkC35Uz5gh98GF+2CVdCXnVntkNNC9Vsl5pdPd/+5RDofjtI82sfeb7POKaSGVB9zG2sjtKo+ruJ3WR8sjNqfiFSVGKp26dzpaOTuKaXLPSZtWH3V07YyJANWCrli4P1ATw6S5cvT1izIs+7nd4LXhc3mHd4bf0GrfhXSO0+zYagHWenaPuewg8oHRhFmPjw2jbtIh3iZCbQCZ73lON0q5+211cP0z5rpeZd0RXvRRhJlCTHSKVZFyBDNg/8WarXvRGa/VVSuc1dC6mB5UJ0Zx0sW4L+dma0h11tWoVUVUBI2LSM2S2rbt3dodDu4PnfufO6i9cAbD/Nm1yuept7N1A+QncUY2H6PnzjAykwQrwwjPvqWxpvBVfYnn9ByWAodUT5L8dpaiz8AS5J+zmYQLVG6teiQ41Fk68GaIzUI39zJj1955bGZfThImothQQQqgWcyRkSi76sEnVUFv8iUsMFzggCsiCwwj3cCFsI7C55GmMeQog3o6VeVAt5U4aPPi9GDgDAfO9we7D78uFdy2PrE+tfYsx/rCemh9Yz21TizS+7H3tvdT72f7F/tX+zf79+LVa1ulz8dW7bH/+BuV+wLt</latexit><latexit sha1_base64="cE+o3aSCEDnOXZ+jom3i4vYnobY=">AAALD3icnVbNbttGEKaTtLXYn8TpsZdNrQIO4AiiD22DwEBaK0IPaZo2tuPClIjlciVtstwldpdJBJrItT32aXoreu0j9DX6BB3+SOWf7La8aDXLme+bb2eG60ecaTMc/rl17fqNd959b7tnv//Bhx/dvLVz+1TLWBF6QiSX6szHmnIm6IlhhtOzSFEc+pw+918eZfvPX1GlmRTHZhnRSYjngs0YwQZM3s72X65P50wkmM+lYmYRprZLcJTtJo8hKFZoxDRRLGQCC4O+EpgvNdNoJhUaY0I1ODRCMAK2ZwYbivquL3mglyH8JGep5xr6xiRGYSbSfVTdXNY3kTunRqPC9Fji4Dgzj7DB6d7dPnKPZBhSYZL+JTEOXU5n5nzpOdMGbiCN3kdL78m0jqnYfGEm/avoU206CIJ1xa/T/7SRsRvG9ShjphdUzXJRUc597//KV2RSFYrDUQqkY19HANCd4WjFpyQEvhFWdFxlhDZLUmfUnW6Ll/QNEEYB1DoWgIM0kYoW9ZVFRUGmaUnXdsexIHlxFmIV1JLGOTWlyQ50LFVCMVmgPkEuABa1gZx1NRx1HP8Tj5SKXBQOboIYrJceQ4eIlB5uCqZKSiIOB0jjMOKQx2voCEQ41hqQN1RWJs0ayZ0pTBInTQAcBNNx6CWXYae1ZN+kHqtQIZiTmGeABYWQYgEcHokA9NjIpcVkxYMdOuk0//tvICVMHsz5CrQD7FnqvYZkiuAkC35Uz5gh98GF+2CVdCXnVntkNNC9Vsl5pdPd/+5RDofjtI82sfeb7POKaSGVB9zG2sjtKo+ruJ3WR8sjNqfiFSVGKp26dzpaOTuKaXLPSZtWH3V07YyJANWCrli4P1ATw6S5cvT1izIs+7nd4LXhc3mHd4bf0GrfhXSO0+zYagHWenaPuewg8oHRhFmPjw2jbtIh3iZCbQCZ73lON0q5+211cP0z5rpeZd0RXvRRhJlCTHSKVZFyBDNg/8WarXvRGa/VVSuc1dC6mB5UJ0Zx0sW4L+dma0h11tWoVUVUBI2LSM2S2rbt3dodDu4PnfufO6i9cAbD/Nm1yuept7N1A+QncUY2H6PnzjAykwQrwwjPvqWxpvBVfYnn9ByWAodUT5L8dpaiz8AS5J+zmYQLVG6teiQ41Fk68GaIzUI39zJj1955bGZfThImothQQQqgWcyRkSi76sEnVUFv8iUsMFzggCsiCwwj3cCFsI7C55GmMeQog3o6VeVAt5U4aPPi9GDgDAfO9we7D78uFdy2PrE+tfYsx/rCemh9Yz21TizS+7H3tvdT72f7F/tX+zf79+LVa1ulz8dW7bH/+BuV+wLt</latexit>

Largest Eigenvalue

CSE 402

VISUALIZING LDA EIGENVECTORS

▸ Train on images from the ORL Face dataset

▸ Obtain the Eigenvectors corresponding to the 5 largest Eigenvalues

▸ Called Fisherfaces

▸ Reshape from 1D vector to 2D image

�36

2nd Largest Eigenvalue

3rd Largest Eigenvalue

4th Largest Eigenvalue

5th Largest Eigenvalue

Images from Introduction to Biometrics, 2011.

CSE 402

REVIEW

▸ Face Images are typically acquired using a VIS spectrum camera like a camera on your phone

▸ Viola-Jones uses a cascade classifier to detect faces

▸ PCA learns a subspace from the training data

▸ LDA learns a subspace similar to PCA but takes into account the class label (i.e., the subject ID)

�37