face detection and recognition - unc computer science
TRANSCRIPT
![Page 1: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/1.jpg)
Face detection and recognition
Many slides adapted from K. Grauman and D. Lowe
![Page 2: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/2.jpg)
Face detection and recognition
Detection Recognition “Sally”
![Page 3: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/3.jpg)
Consumer application: iPhoto 2009
http://www.apple.com/ilife/iphoto/
![Page 4: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/4.jpg)
Consumer application: iPhoto 2009Can be trained to recognize pets!
http://www.maclife.com/article/news/iphotos_faces_recognizes_cats
![Page 5: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/5.jpg)
Consumer application: iPhoto 2009Things iPhoto thinks are faces
![Page 6: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/6.jpg)
Outline• Face recognition
• Eigenfaces
• Face detection• The Viola & Jones system
![Page 7: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/7.jpg)
The space of all face images• When viewed as vectors of pixel values, face
images are extremely high-dimensional• 100x100 image = 10,000 dimensions
• However, relatively few 10,000-dimensional vectors correspond to valid face images
• We want to effectively model the subspace of face images
![Page 8: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/8.jpg)
The space of all face images• We want to construct a low-dimensional linear
subspace that best explains the variation in the set of face images
![Page 9: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/9.jpg)
Principal Component Analysis• Given: N data points x1, … ,xN in Rd
• We want to find a new set of features that are linear combinations of original ones:
u(xi) = uT(xi – µ)(µ: mean of data points)
• What unit vector u in Rd captures the most variance of the data?
Forsyth & Ponce, Sec. 22.3.1, 22.3.2
![Page 10: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/10.jpg)
Principal Component Analysis• Direction that maximizes the variance of the projected data:
Projection of data point
Covariance matrix of data
The direction that maximizes the variance is the eigenvector associated with the largest eigenvalue of Σ
N
N
![Page 11: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/11.jpg)
Principal component analysis• The direction that captures the maximum
covariance of the data is the eigenvector corresponding to the largest eigenvalue of the data covariance matrix
• Furthermore, the top k orthogonal directions that capture the most variance of the data are the k eigenvectors corresponding to the klargest eigenvalues
![Page 12: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/12.jpg)
Eigenfaces: Key idea• Assume that most face images lie on
a low-dimensional subspace determined by the first k (k<d) directions of maximum variance
• Use PCA to determine the vectors or “eigenfaces” u1,…uk that span that subspace
• Represent all face images in the dataset as linear combinations of eigenfaces
M. Turk and A. Pentland, Face Recognition using Eigenfaces, CVPR 1991
![Page 13: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/13.jpg)
Eigenfaces example
Training images
x1,…,xN
![Page 14: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/14.jpg)
Eigenfaces example
Top eigenvectors: u1,…uk
Mean: μ
![Page 15: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/15.jpg)
Eigenfaces example
Principal component (eigenvector) uk
μ + 3σkuk
μ – 3σkuk
![Page 16: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/16.jpg)
Eigenfaces example• Face x in “face space” coordinates:
=
![Page 17: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/17.jpg)
Eigenfaces example• Face x in “face space” coordinates:
• Reconstruction:
= +
µ + w1u1+w2u2+w3u3+w4u4+ …
=
x̂ =
![Page 18: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/18.jpg)
Reconstruction demo
![Page 19: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/19.jpg)
Recognition with eigenfacesProcess labeled training images:• Find mean µ and covariance matrix Σ• Find k principal components (eigenvectors of Σ) u1,…uk• Project each training image xi onto subspace spanned by
principal components:(wi1,…,wik) = (u1
T(xi – µ), … , ukT(xi – µ))
Given novel image x:• Project onto subspace:
(w1,…,wk) = (u1T(x – µ), … , uk
T(x – µ))• Optional: check reconstruction error x – x to determine
whether image is really a face• Classify as closest training face in k-dimensional
subspace
^
M. Turk and A. Pentland, Face Recognition using Eigenfaces, CVPR 1991
![Page 20: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/20.jpg)
Recognition demo
![Page 21: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/21.jpg)
Limitations• Global appearance method: not robust to
misalignment, background variation
![Page 22: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/22.jpg)
Limitations• PCA assumes that the data has a Gaussian
distribution (mean µ, covariance matrix Σ)
The shape of this dataset is not well described by its principal components
![Page 23: Face detection and recognition - UNC Computer Science](https://reader031.vdocuments.us/reader031/viewer/2022021211/620654658c2f7b173006ba1d/html5/thumbnails/23.jpg)
Limitations• The direction of maximum variance is not
always good for classification