1

M. Abdulrahman, T. R. Gwadabe, F. J. Abdu & A. Eleyan

Department of Electrical & Electronics Engineering

Mevlana University Konya, Turkey

Presented by

MUZAMMIL ABDULRAHMAN

2013

IntroductionApplication of FERBasic Steps of FERPrincipal Component AnalysisLocal Binary PatternGabor Wavelet TransformClassificationSimulation ResultsConclusionReferences

3

DEFINITION

Face Recognition (FR) can be described as classifying a face either known or unknown, after comparing it with known individuals stored in a database

Facial Expression Recognition (FER) system is a computer application for automatically identifying or verifying people’s emotions reflected on their faces from a digital image or a video frame from a video source by comparing it with database.

4

5

Facial recognition utilizes distinctive features of the face such as distinct micro elementsmicro elements like: Mouth, Nose , Eye, Cheekbones, Chin, Lips, Forehead, EarsThe distance between the eyes, the length of the nose, and the angle of the jaw give rise to the type of expression. Below are the 7 Facial Expressions types

Angry Angry Disgust Disgust Fear Fear Happy Happy Neutral Neutral Sad Sad Surprise Surprise

Human computer interaction

Automated access control

Video surveillance

6

7

Games

SecurityPatient condition monitoring

FER involve the following steps: Face detection Facial expression data extraction Facial expression classification

The following algorithms can be used in a Holistic-based approach to extract the facial expression features: Principal Component Analysis PCA Linear Discriminant Analysis LDA Local Binary Patterns LBP Discrete Wavelet Transform DWT Gabor Wavelet Transform GWT Discrete Cosine Transform DCT

The aim of the PCA is to reduce the dimensionality of the raw data (features) while retaining as much as possible of the variation present in the dataset.

Speeds up the computational time.

9

10

PCA

The database

2

1

2

N

b

b

b

11

2

1

2

N

c

c

c

2

1

2

N

d

d

d

2

1

2

N

a

a

a

2 2 2

1 1 1

2 2 21, 213

N N N

a b h

a b hm where M

M

a b h

Then subtract it from the training faces

2 2 2 2 2 2 2 2

2 2

1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2

1 1 1 1

2 2

, , , ,

,

m m m m

N N N N N N N N

m m

N N

a m b m c m d m

a m b m c m d ma b c d

a m b m c m d m

e m f m

e m fe f

e m

2 2 2 2 2 2

1 1 1 1

2 2 2 2 2 2, ,m m

N N N N N N

g m h m

m g m h mg h

f m g m h m

12

Now we build the matrix which is N2 by M

The covariance matrix which is NN22 by NN22

m m m m m m m mA a b c d e f g h

Cov AA

13

Find eigenvalues of the covariance matrixThe matrix is very largeThe computational effort is very big

We are interested in at most M eigenvaluesWe can reduce the dimension of the matrix

Compute another matrix which is M by M

Find the M eigenvalues and eigenvectors• Eigenvectors of Cov and L are equivalentequivalent

Build matrix V from the eigenvectors of L

L A A

14

Eigenvectors of Cov are linear combination of image space with the eigenvectors of L

Eigenvectors represent the variation in the faces

U AVm m m m m m m mA a b c d e f g h

V is Matrix of Eigenvectors

15

A: collection of the training faces

U: Face Space / Eigen Space

Compute for each face its projection onto the face space

1 2 3 4

5 6 7 8

, , , ,

, , ,

m m m m

m m m m

U a U b U c U d

U e U f U g U h

To recognize a Facial Expression

Subtract the average face from it

2 2

1 1

2 2

m

N N

r m

r mr

r m

2

1

2

N

r

r

r

16

Compute its projectionprojection onto the face space Uface space U

17

mU r

Different illumination

18

19

Different head pose Different alignment Different facial expression

20

The LBP operator was originally designed for texture description. The operator assigns a label to every pixel of an image by thresholding the 3x3-neighborhood of each pixel with the center pixel value and considering the result as a binary number.

21

Advantages of Uniform LBP Save memory: With a non uniform pattern there is Possible combinations while for uniform LBP there are patterns of Uniform LBP detects only the important local textures like spots,

edges and corners

22

2P

( 1) 2P P

Uniform Pattern: An LBP is called uniform if the binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa when the bit pattern is considered circular

Example The patterns 00000000 (0 transitions) 01110000 (2 transitions) 11001111 (2 transitions)

The patterns 11001001 (4 transitions) and 01010011 (6 transitions) are not uniform.

are uniform

Divide the examined face image to cells For each pixel in a cell, compare the pixel to each of

its neighbors. Follow the pixels along a circle, i.e. clockwise or counter-clockwise.

Where the center pixel's value is greater than the neighbor, write "1". Otherwise, write "0". This gives an 8-digit binary number (which is converted to decimal).

Compute the histogram, over the cell, of the frequency of each "number“ occurring.

Optionally normalize the histogram. Concatenate normalized histograms of all cells. This

gives the feature vector for the face image.

23

A GW filter is an essential tool used to extract local features which can be applied on images to extract features aligned at particular angles (orientations).

The GWs filter captures significant visual features such as spatial localization, orientation selectivity, frequency selectivity, and phase relationship

The GWs kernel can be defined by the following equation:

24

' ' 2

'2( )

22

1, , ,

2

X Yi Xx y e e

where (x,y) denote the pixel position in the spatial domain , is the central frequency of a sinusoidal plane wave, θ is the orientation of the Gabor filter and σ is the standard deviation along x and y directions. The parameters and can be defined by the following equations:

25

' 'cos sin , sin cosX X X

26

Having an input image I(x,y) of size MxN and a Gabor wavelets kernel of

The Gabor feature representation is obtained by convolving the input image with 40 Gabor wavelet kernels given by

Concatenate the magnitude of the convolved output images of all the 40 feature vectors for each input face image

Optionally before concatenation each image output is down-sample by a factor of 16 or 32 and normalized to zero mean and unit variance.

Apply Any dimensionality Reduction Algorithm to reduce the size of the feature vector.

27

, ( , , , )u v x y

, ,( , ) ( , ) ( , , , )u v u vx y I X Y x y

Gabor Wavelet Transform posses many properties which make them attractive for many applications. Directional selectivity Invariance to shifts and rotations Insensitive and robust to facial expression changes Insensitive to illumination variations

Despite many advantages of Gabor wavelet based algorithms in face recognition, it has major disadvantages. High computational complexity High memory capacity requirement Feature vectors dimensions are extremely large

28

Compute the Euclidian distance in the Compute the Euclidian distance in the face space between face space between the test face the test face and and all all faces faces in the in the Training dataTraining data

The expression with the minimum distance from Test face to the Training will be matched as the best expression of the Test face.

29

22 1..i i for i M

JAFFE facial expression database was used to conduct our experiments.

It contains 213 images of 10 different females each with 7 expressions posed by 3 or 4 examples of each of the seven facial expressions under different illumination and head position. The images are of the size 256x256

Each original image has been aligned by normalizing it.

A total of 137 images (64%) were used as training data, while the remaining 76 images(36%) as testing data

The K-nearest neighbour, Euclidean distance (L2) was used as a similarities measure to classify the facial expressions images.

 

31

32

Experiment PCA [1] PCA+LDA [1] ASM & HMM [2] LBP [3] SVM [4] PCA NMF LNMF[5]

Recognition

Rate (%)

80.00 95.11 88.79 85.57 94.5 63.25 65.50 64.50

Face Dimension 128x96 128x96 230X250 64X64 44X32 40X30

FRR(%) Comparisons For Different FER FRR(%) Comparisons For Different FER Technique Using JAFFETechnique Using JAFFE

Gabor wavelets were used as a pre-processing stage followed by dimensionality reducing using PCA/LBP for facial expression recognition in this paper.

Experimental evaluations the proposed approach were conducted on JAFFE database.

The results obtained showed that pre-processing with Gabor wavelets improves the performance of directly applying both PCA and LBP.

Also the variation in illumination, hair and head position affect the facial recognition rate.

Facial expression recognition proposed in this paper has an improved performance when compared with the previous works using different algorithms using the same JAFFE database as seen in tables.

33

[1] H. Deng, L. Jin And L. Zhen, “A New Facial Expression Recognition Method Based On Local Gabor Filter Bank And PCA Plus LDA”, International Journal Of Information Technology Vol. 11 No. 11 2005, pp. 93

[2] W. Zhao And J. Zhang, “Using ASM-Optical Flow Method And Hmm In Facial Expression Recognition”, IERI International Conference On Affective Computing And Intelligent Interaction, Lecture Notes In Information Technology, Vol.10, 2012 Pp. 268.

[3] S. Liao, W. Fan And D. Yeung, “Facial Expression Recognition Using Advanced Local Binary Patterns, Tsallis Entropies And Global Appearance Features”, IEEE, 2006 pp. 668.

[4] A. Bouzerdoum, S.L. Phung And P. Li, “Feature Selection For Facial Expression Recognition”, IEEE, 2nd European Workshop On Visual Information Processing USA, 2010 pp. 39

[5] I. Buciu And I. Pitas, “Application Of Non-Negative And Local Non Negative Matrix Factorization To Facial Expression Recognition”, IEEE Proceedings Of The 17th International Conference On Pattern Recognition , 2004 1051-4651.

34

35