face collections
DESCRIPTION
Face Collections. 15-463: Rendering and Image Processing Alexei Efros. Nov. 2: Election Day!. Your choice!. Figure-centric averages. Antonio Torralba & Aude Oliva (2002) Averages : Hundreds of images containing a person are averaged to reveal regularities - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/1.jpg)
Face Collections
15-463: Rendering and Image Processing
Alexei Efros
![Page 2: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/2.jpg)
Nov. 2: Election Day!
Your choice!
![Page 3: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/3.jpg)
Figure-centric averages
Antonio Torralba & Aude Oliva (2002)Averages: Hundreds of images containing a person are averaged to reveal regularities
in the intensity patterns across all the images.
![Page 4: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/4.jpg)
Cambridge, MA by Antonio Torralba
![Page 5: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/5.jpg)
More by Jason Salavon
More at: http://www.salavon.com/
![Page 6: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/6.jpg)
“100 Special Moments” by Jason Salavon
Why blurry?
![Page 7: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/7.jpg)
Face Averaging by MorphingPoint Distribution Model
Average faces
![Page 8: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/8.jpg)
Manipulating Facial Appearance through Shape and Color
Duncan A. Rowland and David I. Perrett
St Andrews University
IEEE CG&A, September 1995
![Page 9: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/9.jpg)
Face ModelingCompute average faces
(color and shape)
Compute deviations between male and female (vector and color differences)
![Page 10: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/10.jpg)
Changing gender
Deform shape and/or color of an input face in the direction of “more female” original shape
color both
![Page 11: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/11.jpg)
Enhancing gender
more same original androgynous more opposite
![Page 12: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/12.jpg)
Changing age
Face becomes “rounder” and “more textured” and “grayer”
original shape
color both
![Page 13: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/13.jpg)
Change of Basis (PCA)
From k original variables: x1,x2,...,xk:
Produce k new variables: y1,y2,...,yk:
y1 = a11x1 + a12x2 + ... + a1kxk
y2 = a21x1 + a22x2 + ... + a2kxk
...
yk = ak1x1 + ak2x2 + ... + akkxk
such that:
yk's are uncorrelated (orthogonal)y1 explains as much as possible of original variance in data sety2 explains as much as possible of remaining varianceetc.
![Page 14: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/14.jpg)
Subspace Methods
How can we find more efficient representations for the ensemble of views, and more efficient methods for matching?
Idea: images are not random… especially images of the same object that have similar appearance
E.g., let images be represented as points in a high-dimensional space (e.g., one dimension per pixel)
![Page 15: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/15.jpg)
Linear Dimension Reduction
Given that differences are structured, we can use ‘basis images’ to transform images into other images in the same space.
+=
+ 1.7=
![Page 16: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/16.jpg)
Linear Dimension Reduction
What linear transformations of the images can be used to define a lower-dimensional subspace that captures most of the structure in the image ensemble?
![Page 17: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/17.jpg)
Principal Component Analysis
Given a point set , in an M-dim space, PCA finds a basis such that
coefficients of the point set in that basis are uncorrelated
first r < M basis vectors provide an approximate basis that minimizes the mean-squared-error (MSE) in the approximation (over all bases with dimension r)
x1
x0
x1
x0
1st principal component
2nd principal component
![Page 18: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/18.jpg)
Principal Component Analysis
RemarksRemarks
If the data is multi-dimensional Gaussian, then its marginals are Gaussian, and the PCA coefficients are statistically independent
If the marginal PCA coefficients are Gaussian, then the maximum entropy joint distribution is multi-dim Gaussian but the true joint distribution may NOT be Gaussian
Choosing subspace dimension r: look at decay of the eigenvalues
as a function of r Larger r means lower expected
error in the subspace data approximation
r M1
eigenvalues
![Page 19: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/19.jpg)
EigenFaces
First popular use of PCA for object recognition was for the detection and recognition of faces [Turk and Pentland, 1991]
Collect a face ensemble
Normalize for contrast, scale, & orientation.
Remove backgrounds
Apply PCA & choose the first N eigen-images that account for most of the variance of the data. mean
face
lighting variation
![Page 20: Face Collections](https://reader030.vdocuments.us/reader030/viewer/2022032606/56812e12550346895d937cfa/html5/thumbnails/20.jpg)
Blinz & Vetter, 1999
show SIGGRAPH video