The Effect of Distance Measures on the Recognition Rates of PCA

and LDA Based Facial Recognition Philip Miller, Jamie Lyle

Digitial Image Processing Clemson Universtiy

{pemille, jlyle}@clemson.edu

Abstract Many components affect the success of a facial

recognition system. While some research attempts

to improve on PCA or LDA algorithms, an often

overlooked component is the distance measure. In

this paper we show that the choice of distance

measure greatly affects the recognition rate.

Experiments are performed using the FRGC and

FERET face databases. Recognition rates of ten

distance measures are compared. There is an

inconsistency of performance for each distance

measure across each algorithm and face database.

This shows that being able to determine the best

distance measure before running the recognition

algorithm will make the recognition system more

successful.

1. Introduction Increasing the effectiveness of facial recognition

systems is important to biometrics researchers.

There are a few ways to accomplish this.

Improvements could be made to the matching

algorithm like PCA and LDA. A better training set

can be used. This paper looks at different distance

measures.

Distance measures are the last component of facial

recognition. Images are projected into an

eigenspace or fisherspace and represented as

vectors. The distance between the vectors of two

images is the similarity of the images.

For each recognition algorithm used with each

data set, one distance measure will be superior for

the experiment as a whole. That same distance

measure will not be superior for all experiments. A

single subject is not guaranteed to be matched

best with the superior distance measure either.

Therefore, the need exists for adaptive distance

measure selection.

In section 2 and 3 we will discuss the PCA and LDA

algorithms respectively. Section 4 will give an

explanation of each of the distance measures used

in the experiments. The face images used will be

described in section 5. The results of the

experiments are discussed in section 6. Finally, the

paper ends with our conclusion in section 7.

2. Eigenfaces Eigenfaces is the approach proposed by Turk and

Pentland in [1]. In this approach, face images are

projected into a lower dimensional space using

Principal Component Analysis (PCA). Each image is

represented by a vector of weights needed to

reconstruct the image. Although, in order to

simplify the problem and reduce data storage,

some information is lost in the algorithm and it

may not be possible to completely reconstruct the

images.

The system must first be initialized using a training

set of face images. These images should be

preprocessed to show only the face in order to

reduce recognizing the background or a particular

hairstyle. The eyes in the images should also be in

the same location for normalization. The first step

is to find the mean of all the training images 𝑇1,

𝑇2 ,…, 𝑇𝑀where 𝑇𝑖 is a column vector. The mean can

be found by the equation ψ =1

𝑀 Ti

Mi=1 . The mean

is then subtracted from all training images

𝜙𝑖 = 𝑇𝑖 − 𝜓 for 𝑖 = 1 𝑡𝑜 𝑀. The covariance matrix

of the images is found to be 𝐶 = 𝐴𝐴𝑇 , where 𝐴 =

[𝜙1 𝜙2 …𝜙𝑀]. If the face images started off being

of size 𝑁 by 𝑁, then 𝐶 is a matrix of size 𝑁2 by 𝑁2.

This is a large matrix and it is not feasible to solve

for the eigenvalues of 𝐶. Since the number of

images is less than the dimensions there will only

be 𝑀 − 1 meaningful eigenvectors as opposed

to 𝑁2. So instead of solving for the eigenvectors of

𝐶, we can solve for the eigenvectors of 𝐿,

where 𝐿 = 𝐴𝑇𝐴. 𝐿 is a matrix of size 𝑀 by 𝑀, so

solving for the eigenvectors will find

𝑀 eigenvectors. Choosing the desired number of

eigenvectors of 𝐿 (from the largest eigenvalues),

build the matrix 𝑉. Following the equation, 𝑈 =

𝐴𝑉, the eigenfaces of the system are the columns

of 𝑈 [1].

A face image 𝑇 is projected into the face space (or

PCA space) by the operation Ω = 𝑈𝑇(T − ψ). Ω is

a vector composed of the weights describing the

contribution of each eigenface in representing the

input image. To recognize the face, a distance

measure 𝐷 is used to find the distance between Ω

and Ω𝑘 , for each subject , where Ω𝑘 is the face

image of subject 𝑘 projected into PCA space. If the

minimum of these distances is below the

threshold, then the new face is classified as

belonging to subject 𝑘 [2].

3. Fisherfaces Fisherfaces is the face recognition approach

proposed by Belhumeur et al. in [3]. Where

Eigenfaces is based on PCA, the Fisherfaces

approach is based on Fisher’s Linear Discriminant

Analysis (LDA). Fisherfaces seeks to maximize the

ratio of between-class scatter to within-class

scatter. Eigenfaces maximizes total scatter across

all images, which makes it susceptible to lighting

variations. Fisherfaces seeks to reduce errors due

to lighting variations by maximizing the ratio

mentioned above.

First let the between-class scatter matrix be

defined as 𝑆𝐵 = 𝑁𝑖 𝜇𝑖 − 𝜇 (𝜇𝑖 − 𝜇)𝑇𝑐𝑖=1 where 𝑐

is the number of classes or subjects, 𝜇𝑖 is the mean

image of subject 𝑋𝑖 , and 𝑁𝑖 is the number of

training images for subject 𝑋𝑖 . Also let the within-

class scatter be defined as 𝑆𝑊 = (𝑥𝑘 −𝑥𝑘𝜖𝑋𝑖

𝑐𝑖=1

𝜇𝑖)(𝑥𝑘−𝜇𝑖)𝑇. If 𝑆𝑊 is nonsingular, the

eigenvectors can be found by solving the

equation 𝑆𝐵 − 𝜆𝑖𝑆𝑊𝑤𝑖 = 0. If 𝑆𝑊 is singular, the

images can be reduced by running PCA and then

LDA (Fisherface) can be performed [2]. Building 𝑊

from the eigenvectors found, the images can be

projected into LDA space by 𝑊𝑇𝜇𝑖 . Recognition of

a face image follows much the same as in the

Eigenfaces approach [2].

4. Distance Measures Distance measures are used to compute the

difference between two vectors. The CSU Face

Identification Evaluation System includes many

common distance measures that are used to

compute the similarity between two images. Some

of them are studied in this paper. A few

uncommon ones were also implemented. In the

definitions of the distance measures in the

following subsections, let 𝑢 and 𝑣 be vectors

representing arbitrary images in PCA or LDA space

[4].

In order to compute distances in Mahalinobis

space vectors 𝑢 and 𝑣 must be transformed.

Remember that the set of eigenvalues, 𝜆𝑖 , from

PCA are the sample variances, 𝜎𝑖2, along the

dimensions represented by the eigenvectors. In

Malahinobis space, sample variances along those

dimensions are one. Let 𝑚 and 𝑛 be the vectors in

Mahalinobis space corresponding to 𝑢 and 𝑣. The

vectors are related though the following equations

[4].

𝑚𝑖 =𝑢𝑖

𝜎𝑖 (1)

𝑛𝑖 =𝑣𝑖

𝜎𝑖 (2)

Equations 3 through 9 are explained in [4].

Equation 10 is explained in [5]. Equation 11 and 12

are explained in [6].

4.1 CityBlock

𝐷𝐶𝑖𝑡𝑦𝐵𝑙𝑜𝑐𝑘 𝑢, 𝑣 = 𝑢𝑖 − 𝑣𝑖

𝑖

(3)

4.2 Euclidean

𝐷𝐸𝑢𝑐𝑙𝑖𝑑𝑒𝑎𝑛 𝑢, 𝑣 = (𝑢𝑖 − 𝑣𝑖)2

𝑖

(4)

4.3 Correlation

𝐷𝐶𝑜𝑟𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛 𝑢, 𝑣

= 𝑢𝑖 − 𝑢 (𝑣𝑖 − 𝑣 )𝑖

(𝑁 − 1) 𝑢𝑖 − 𝑢 2

𝑖

𝑁 − 1 𝑣𝑖 − 𝑣 2

𝑖

𝑁 − 1

(5)

4.4 Covariance

𝐷𝐶𝑜𝑣𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑢, 𝑣 = 𝑢𝑖𝑖 𝑣𝑖

𝑢𝑖2

𝑖 𝑣𝑖2

𝑖

(6)

4.5 Mahalinobis CityBlock

𝐷𝑀𝑎ℎ𝐿1 𝑢, 𝑣 = 𝑚𝑖 − 𝑛𝑖

𝑖

(7)

4.6 Mahalinobis Euclidean

𝐷𝑀𝑎ℎ𝐿2 𝑢, 𝑣 = (𝑚𝑖 − 𝑛𝑖)2

𝑖

(8)

4.7 Mahalinobis Cosine

𝐷𝑀𝑎ℎ𝐶𝑜𝑠𝑖𝑛𝑒 𝑢, 𝑣 = 𝑚 ∙ 𝑛

𝑚 𝑛 (9)

4.8 Hellinger

𝐷𝐻𝑒𝑙𝑙𝑖𝑛𝑔𝑒𝑟 𝑢, 𝑣 = ( 𝑢𝑖 − 𝑣𝑖 )2

𝑖

(10)

4.9 Canberra

𝐷𝐶𝑎𝑛𝑏𝑒𝑟𝑟𝑎 𝑢, 𝑣 = 𝑢𝑖 − 𝑣𝑖

𝑢𝑖 + 𝑣𝑖 𝑖

(11)

4.10 Czekanowski

𝐷𝐶𝑧𝑒𝑘𝑎𝑛𝑜𝑤𝑠𝑘𝑖 𝑢, 𝑣 = 2 ∗ min(𝑢𝑖 , 𝑣𝑖)𝑖

𝑢𝑖 + 𝑣𝑖𝑖

(12)

5. Data The Facial Recognition Grand Challenge Data is a

set of multi-modal biometric data collected at the

University of Notre Dame. The experiments used

only the subset of 2D frontal face images with

neutral facial expression and controlled lighting.

We used 3939 images from 193 subjects for

training and 18370 images from 563 subjects for

the probe/gallery set. Because the data was

collected on the Notre Dame campus, a large

majority of the images are of people 18 to 22 years

of age [7].

The Facial Recognition Technology (FERET)

Database was created on the behest of the

Defense Advanced Research Projects Agency and

National Institute of Standards and Technology.

The experiments used 600 images from 300

subjects for training and 1536 images from 680

subjects for the probe/gallery set [8].

6. Results Experiments were run using PCA and LDA based

facial recognition with the FRGC and FERET

databases. These experiments produced distances

for each face from each of the other faces. From

these distances, cumulative match characteristic

(CMC) curves were created to show the success of

each distance measure as a whole. Figures 1

through 4 are the CMC curves for the experiments.

Figure 1: PCA using FERET

Figure 2: LDA using FERET

Figure 3: PCA using FRGC

Figure 4: LDA using FRGC

In Figure 1, the top curve is MahCosine. In Figure 2,

the top curves are both Correlation and

Covariance. In Figure 3, the top curve is MahCosine

and Czekanowski. In Figure 4, the top curves are

both Correlation and Covariance.

For PCA using the FERET database, the top curve,

MahCosine, gave a top rank image to subject

match in all but 204 images. When MahCosine did

not produce a rank of 1 for an image the best

ranking distance measure was recorded in table 1.

Ties were given to MahCosine. If any of the other

distance measures tied for best rank they all

received credit. There is some overlap. If the best

matching distance measure was a rank 1 match it

was also recorded.

Best Match Rank 1 Match

CityBlock 33 19

Euclidean 38 20

Correlation 35 20

Covariance 33 21

MahL1 26 12

MahL2 36 18

MahCosine 51 -

Hellinger 7 7

Canberra 6 2

Czekanowski 54 28

Table 1: Non Rank 1 MahCosine best match

0.8

0.9

1

0 200 400 600

CityBlockCorrelationCovarianceEuclideanMahCosineMahL1MahL2CanberraCzekanowskiHellinger

0.8

0.9

1

0 200 400 600

CityBlockCorrelationCovarianceEuclideanMahCosineMahL1MahL2CanberraCzekanowskiHellinger

0.8

0.9

1

0 200 400 600

CityBlockCorrelationCovarianceEuclideanMahCosineMahL1MahL2CanberraCzekanowskiHellinger

0.8

0.9

1

0 200 400 600

CityBlockCorrelationCovarianceEuclideanMahCosineMahL1MahL2CanberraCzekanowskiHellinger

7. Conclusions MahCosine and Czekanowski performed best with

PCA while Correlation and Covariance performed

best with LDA. This does not mean, though, that

these distance measures should be used in all

circumstances and ignore the others. Certain

individuals were matched better with a distance

measure other than the top one for the system.

A system that uses different distance measures for

each image will perform better than a system that

only uses one. In the case of PCA based facial

recognition using the FERET Database, 153 images

would have been matched better using a mixed

distance measure approach.

This data proves that a need exists for a system

that uses different distance measures for each

image. Further research is required into how to

accomplish this task.

8. References [1] M. Turk and A. Pentland, Eigenfaces for

Recognition, Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.

[2] Lecture slides from Cpsc 881-03 “Biometrics” at Clemson University with Dr. Damon Woodard. “Appearance based facial recognition”. Summary of PCA and LDA. http://people.clemson.edu/~woodard/protected/Lecture21.pdf

[3] P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, Eigenfaces vs. Fisherfaces: recognition using class-specific linear projection, IEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, 1997.

[4] Beveridge, R., Bolme, D., Teixerira, M., and Draper, B., 2003, The CSU Face Identification Evaluation System User's Guide: Version 5.0, Colorado State University.

[5] Abdi, H., 2007, Distance, in Salkind, N., ed., Encyclopedia of Measurement and Statistics, Sage Publications , Thousand Oaks, CA.

[6] Androutsos, D., Plataniotiss, K.N., and Venetsanopoulos, A.N., 1998, Distance measures for color image retrieval, Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on, vol.2, no., pp.770-774 vol.2, 4-7 Oct 1998.

[7] Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Jin Chang, Hoffman, K., Marques, J., Jaesik Min, Worek, W., 2005, Overview of the face recognition grand challenge, Computer Vision and Pattern Recognition. CVPR 2005. IEEE Computer Society Conference on , vol.1, no., pp. 947-954 vol. 1, 20-25 June 2005.

[8] Rallings C., Thrasher M., Gunter C., Phillips P.J., Wechsler H., Huang J., and Rauss P.J., 1998, The FERET database and evaluation procedure for face-recognition algorithms, Image and Vision Computing, Volume 16, Number 5, 27 April 1998, pp. 295-306(12).