2004_fgr_facial feature extraction using pca and wavelet multi-resolution images

7/28/2019 2004_FGR_Facial Feature Extraction Using PCA and Wavelet Multi-Resolution Images

1/6

Facial Feature Extraction

Using PCA and Wavelet Multi-Resolution Images

Kyung-A Kim 1 , Se-Young Oh 2 , Hyun-Chul Choi 3

Dept. of Electrical Eng., Pohang University of Science and Technology,

Pohang, Kyungbuk, 790-784, South Korea1 [email protected] [email protected] [email protected] 3

Abstract

This paper presents a novel algorithm for theextraction of the facial feature (eyebrow, eye, nose and

mouth) fields from 2-D gray-level face images. Thefundamental philosophy is that eigenfeatures, derived

from the eigenvalues and eigenvectors of the gray-leveldata set constructed from the feature fields, are veryuseful to locate these fields efficiently. In addition,multi-resolution images, derived from a 2-D DWT

(Discrete Wavelet Transform), are used to save thesearch time of the facial features. The experimental

results indicate that the proposed algorithm is robustagainst facial feature size and slight variations of pose.

1. Introduction

Face recognition is useful for many applications, suchas identity authentication, security access, e-commerceetc., which has received greater interest among experts

recently. To recognize face, automatic extraction offacial features from a persons face is a very importantprocess. Many researchers have proposed methods tofind the facial feature regions [1,3-5,8] or to locate theface region [6-7,9-10] in an image. These methods canbe classified by their use of three types of information:template matching, intensity and geometrical features.

In general, template matching requires many templates

to accommodate varying pose whereas the intensitymethod requires good lighting conditions. Andgeometrical method is not robust enough for thevarious pose.In this paper, we present a novel algorithm for the

extraction of the facial feature fields from 2-D gray-level face images. We use a sliding window template ofthe facial features represented in the eigenfeature space

to locate facial features in face images. Here, theeigenfeature space is defined as eigenvector space fromthe training set which consists of a particular facialfeature one of eyes, eyebrows, nose and mouth. Tospeed up execution time, multi-resolution images and

heuristic assumptions are used. Also, to enhanceperformance of the system, we combine a few single

feature detectors.In Section 2, the basic idea of the proposed algorithm

is reviewed briefly. A facial feature extractor, whichuses the proposed algorithm, is introduced in Section 3.And methods to improve the performance of theextraction system are presented in Section 4.Experimental results are reported in Section 5.

2. The Basic Idea

There are many disadvantages in pixel-based sample

representation approach, for example, the data is notcompact and there is much redundant information.Besides, when we search for a facial feature in thewhole face image (original size), computational cost is

very expensive. Therefore, we use a 2-D discretewavelet transform (2-D DWT) to emphasize the change

information in the gray scale and to reduce the searchregion. After the transform, the dimension could bereduced from 256x256 to 128x128, 64x64 and 32x32,which can save the computational cost. Figure 1 givesan example of the decomposition of an image. TheHaar wavelet is adopted in our experiment.On the other hand, the theory of the principal

component analysis (PCA) is widely used in pattern

recognition as well as in classification. The PCA has aspecial characteristic which can retrieve original image

using dominant eigenvectors (eigenfeatures). We use asliding window template of the facial featuresrepresented in the eigenfeature space to locate facialfeatures in face images. Images of facial features do notchange radically when projected into the feature space,while the projection of non-feature images that are not

facial features appears very different. We define thefacial feature map based upon the face map byM.Turk and A.Pentland [2]. The facial feature map iscalculated as the distance between the local subimageand facial feature template after PCA filtering at everylocation in the image. The PCA filter is defined as

follows. The original image is PCA transformed into


2/6

the eigenfeature space. The dominant PCA componentsare then mapped back into the original image space andthe result is called a PCA filtered image. In the facialfeature map, the lowest value indicates the presence of

a feature (see Figure 2).

3. Facial Component Extraction

We use knowledge of the eigenfeature space and a2D-DWT to detect facial features in face images.Figure 3 represents the system block diagram. Thedetailed algorithm follows:

Figure 1. Example of the Multi-Resolution Image.

Figure 2. Example of the feature map for the eye.

Step1 : Compute eigenvectors (eigenfeatures) of thefacial features (eyebrow, eye, nose, mouth)

with manually marked feature from the trainingset.

Step2 : Perform a Haar transform of the input image atseveral levels (In our experiments, 3 levels)with scale-frequency resolution. Among theresults of the transform, LL2, LH2, HL2 and

LL3 images are used to detect facial features.To improve the accuracy of detection withhigher cost, LL1, LH1 and HL1 images can beused. Therefore a tradeoff between accuracy

and computational cost exists. In our test,because the use of the former images is enoughto obtain good performance, latter images arenot used.

Step3 : Determine coarse regions of eyebrows, eyes,nose and mouth using the LL3(32x32) image.

Here, a feature detection system is made up of 2double feature detectors. One gets coarse regionof eyebrows and eyes, while the other obtainsthe region containing both nose and mouth. Thecoarse region detectors were trained onrandomly selected 30 frontal images with

Figure 3. The system block diagram

manually marked samples. Size of each sampleis 6x15, 8x8 for the eyebrow-eye and the nose-mouth region respectively. This step not only

reduces the size of the search window in thenext step but also improves performance of the

facial feature detection. Figure 4 showsexamples of the detection result.

Step4 : Compute the feature map as follows.Step4-1: Generate a binary edge image (canny

HL1

HL2

HL3

HL3 HL2 LH1

LL1

LL2

Decompose the image using 2D-DWT(Multi-Resolution Image).

Determine the coarse regions for eyebrows, eyes, nose and mouth.

Generate a binary edge image or

image with certain threshold within coarse region.

Compute the facial feature map and detect features.

Input image

Compute eigenvectors (eigenfeatures) of facial features.

Double Feature Detector 1

(eyes and eyebrows region)Double Feature Detector 2

(nose and mouth region)

Facial features

SingleFeatureDetector

(eye)


(eyebrow)


(nose)


(mouth)

If

failed


3/6

or sobel edge) or an image with certainthreshold within a coarse region (result of theprevious step) using summation of LH2 andHL2 images (see Figure 5), which contain high

frequency features. To save time, only thosepixels with the 1s value within the coarse

region (result of the previous step) arecomputed for the facial feature map.Step4-2: Compute the facial feature map usinga PCA information in LL2 image. In the featuremap, the lowest value indicates the presence ofa feature [2].

(a) (b)

Figure 4. Example of the coarse feature region by

Step3. (a) and (b) represent the coarse region

containing eyebrow-eye and nose- mouth.

(a) (b)

Figure 5. (a) From left to right, top to bottom,

clockwise, LL2, LH2, (LH2+HL2) and HL2 images.

(b) binary image obtained by applying a certain

threshold to the summed image of the LH2 and HL2.

4. Performance improvement of the feature

detection system

To improve the feature detection performance, thesystem combined with a few detectors, rather than thesingle feature detector, is more proper because each

system assists each other to avoid defects of the single

detector system. Also, the characteristic information ofthe facial morphology simplifies the process therebyreducing the execution time.

4.1. Combined usage of the two single

feature detectors for increased robustness

We developed a feature detector system whichconsists of the two double feature detectors and four

single feature detectors (see Figure 3). Double featuredetector1 extracts a region containing the eyebrows and

eyes while double feature detector2 extracts that of thenose and mouth. Second row of the Figure 6 shows theadvantage of detecting both eyes and eyebrows over thesingle feature detectors. The single eye detector cannot

extract the right eyebrow, whereas the eyebrow detectormakes upforthe eye detectors defects. First row of the

Figure 7 also demonstrates the advantage of using thenose-mouth detector. The single nose detector cannotextract the exact nose, however, this problem is solvedas the mouth detector detects a nose as well as mouth.The inverse case may appear.

(a) (b) (c)

Figure 6. Example of the results of (a) Eye Detector,

(b) Eyebrow Detector and (c) Combined Detector

(a) (b) (c)

Figure 7. Example of the results of (a) Nose Detector,

(b) Mouth Detector and (c) Combined Detector

4.1.1 Eye-eyebrow detector

(1)Eye detectorFirst, select four candidates (two eyes and two

eyebrows) that have the lowest value in the featuremap. This extracts both eyes and eyebrows withover 80% of accuracy. If four candidates satisfyheuristic assumptions (the relations betweeneyebrows and eyes), the eyebrow detector does not
http://engdic.empas.com/show.tsp/@72508http://engdic.empas.com/show.tsp/uphttp://engdic.empas.com/show.tsp/forhttp://engdic.empas.com/show.tsp/@31873http://engdic.empas.com/show.tsp/@31873http://engdic.empas.com/show.tsp/forhttp://engdic.empas.com/show.tsp/uphttp://engdic.empas.com/show.tsp/@72508


4/6

operate (see the first row of the Figure 6).Otherwise, the eyebrow detector operates to detectmissed features.

(2)Eyebrow detector

Considering both heuristic assumptions and theresult of the eye detector, it works within the coarse

region. If the lowest value in the eyebrow featuremap is below some chosen threshold, the position isselected for the remaining feature to be found (seethe second row of the Figure 6).

4.1.2 Nose-mouth detector

(1) Nose and mouth detector

Select two candidates that have the lowest value inthe feature map of each single feature detectorsystem. Considering both heuristic assumptions (therelations between nose and mouth) and the results ofnose-mouth detector, features are extracted (see

Figure 7).

4.2. Heuristic Assumptions

made to simplify the detection process

The following assumptions are made in order tosimplify the process of finding the coarse region (seeStep3) and also to improve the feature detectionaccuracy.

(1)The eyebrows and eyes remain within the upper halfof the face region. Their exact position, however,can vary according to the height of the forehead and

the facial pose.(2)The nose and mouth remain within the lower half of

the face region.(3)The relation between eyebrows and eyes

a. The eyebrows are above eyes.b. Left-right eyes and eyebrows are symmetric.c. Inter-eye and eyebrow distances are similar.d. Eyes keep a certain distance from eyebrows. The

certain distance between eyes and eyebrows isdefined as the averaged distance from the

experimental result.(4)The relation between nose and mouth

a. The nose is above the mouth.b. The horizontal center of the nose and the mouth is

near the horizontal center line of the face.c. The nose is kept apart from the mouth by a certain

distance which is taken to be an averaged distancefrom experimental statistics.

5. Experimental Results

We have used the IMDB (Intelligent Multimedia Lab.Database) face data, which consists of 107 Asian faces(56 males and 51 females) [11]. All images are

256x256 pixels in 256 grey levels and are taken againsta homogeneous background, with the person in anupright frontal position, with tolerance for some tiltingand rotation. The eye and eyebrow detectors were

trained on selected 20 frontal images while the noseand mouth detectors were learned from 30 frontal

images with manually marked features. Size of eachfeature in the training samples is 20x50, 20x40, 26x50,30x60 for eyebrow, eye, nose and mouth, respectively.Figure 8 shows the examples of the training feature set.

In this experiment, our algorithm was applied todifferent people with various poses. The algorithm is

very fast (with the average execution time of0.0862sec) and produced good results.The feature detectors were trained as described before,

using 20~30 frontal images. To improve thegeneralization performance, selection of the goodsamples which represent various shapes is important.The proposed algorithm is robust enough for the test

data, which is the rotated facial images. Table 1 showsthe experimental results. The performance of featureextraction have achieved correct hit rate of92.23~98.13% for the training feature samplesconsisting of frontal faces. Also, the system achievedgeneralization of 90.17~96.78% for the test set.Especially, the extraction performance of the eyes ishigher than any other algorithms [1,3]. Figure 9 shows

the representative experimental results. The resultsdemonstrate the robust performance of the detectorwhich extracts narrow eyes as well as eyes inspectacles. Dotted rectangles represent examples ofmis-extraction in rotated faces. To improve the

performance in the rotated face, we can select thesample features in the rotated space as well as the

frontal space. For simplicity, facial features hidden byhair were not considered.

Table 1. Extraction performance.

A: Number of missed features in the frontal face DB.

B: Number of missed features in the rotated face DB.

C: Performance of the facial feature extraction in

the frontal face DB.

D: Performance in the rotated face DB.

(Figures within parentheses represent the total

number of the features)

eyebrow eye nose Mouth

A 16/(206) 4/(214) 5/(107) 7/(107)

B 160/(1628) 55/(1710) 40/(856) 47/(856)

C 92.23% 98.13% 95.33% 93.46%

D 90.17% 96.78% 95.33% 94.51%


5/6

Figure 8. Examples of the training feature set

Figure 9. Examples from the results of the facial detector. Cases include male and female faces, with

and without eyeglasses. Dotted boxes indicate errors.


6/6

6. Conclusion

A facial feature extraction algorithm is presentedbased on eigenfeatures and multi-resolution imageswith the following three merits. First, the training andextraction time of the proposed system is less than that

of the existing that we know of any algorithms [1,3].The training time is needed only to computeeigenfeatures, making it faster than SVM or MLP [1,3].The extraction time takes less than 0.09 second for eachimage through the use of the 2-D DWT, coarse regionextraction, binary images (edge or threshold) andheuristic assumptions. Second, although the detector

system is trained using a relatively small feature sampleset of 2~3% of the total data, it has good generalizationperformance. Third, the eigenfeatures and geometricinformation of the features, that is, the result of theproposed feature detection system, can be directlyapplied to face recognition without additional

processing.The proposed system can be applied to 3D face

modeling, face tracking and detection in mobile robots

and face recognition.

References

[1] Yeon-Sik Ryu and Se-Young Oh. Automatic Extraction ofEye and Mouth Fields from a Face Images usingEigenfeatures and Ensemble Networks. Applied

Intellignece, Vol.17(2), 2002, pp.171-185.[2] Matthew Turk and Alex Pentland. Eigenfaces for

Recognition. Journal of Cognitive Neuroscience,

Massachusetts Institute of Technology, 1991, Volume3,Number 1.

[3] Dihua Xi, Igor T. Podolak and Seong-Whan Lee. FacialComponent Extraction and Face Recognition with

Support Vector Machines. Proceedings of the FifthInternational Conference on Automatic Face and Gesture

Recognition, IEEE Computer Society, 2002, pp.76-81.[4] Yankang Wang, H.Kuroda, M. Fujumura, A. Nakamura,

Automatic extraction of eye and mouth fields from

monochrome face image using fuzzy technique.Proceedings of the 1995 Fourth IEEE InternationalConference on Universal Personal Communications

Record, Tokyo, Japan, 1995, pp.778-782.[5] R.Pinto-Elias, J.H. Sossa-Azuela. Automatic facial feature

detection and location. Proceeding of the 14thInternational Conference on Pattern Recognition,Volume2, 1998, pp.1360-1364.

[6] R. Brunelli, T. Poggio. Face recognition: features versustemplates. Pattern Analysis and Machine Intelligence,

IEEE Transactions on, Volume:15 Issue:10, October 1993,pp.1042 -1052.

[7] P.Juell, R.Marsh. A hierarchical neural network for humanface detection. Pattern Recognition, Volume:29 Issue:5,

1996, pp.781-787.[8] Weimin Huang, Q. Sun, C.P. Lam, J.K. Wu. A robust

approach to face and eyes detection from images with

cluttered background. Proceedings of the 14th Interna-

tional Conference on Pattern Recognition, Volume 1,1998, pp.110-113.

[9] Kin Choong Yow, R. Cipolla. Feature-based human face

detection. Image Vision Computer, Volume:15 Issue:9,1997, pp.713-735.

[10] Takeo Kanade. Picture processing by computer complex

and recognition of human faces. Technical Report,

Department of Information Science, Kyoto University,1973.

[11] Intelligent Multimedia Laboratory. Asian Face Image

Database PF01. Technical Report, Department ofComputer Science and Engineering, Pohang University

of Science and Technology, 2001.
http://web.mit.edu/http://web.mit.edu/

2004_fgr_facial feature extraction using pca and wavelet multi-resolution images

Documents