lec 15: graph embedding & laplacianface · lpp formulation – affinity supervised how to...
TRANSCRIPT
Spring 2019: Venu: Haag 315, Time: M/W 4-5:15pm
ECE 5582 Computer VisionLec 15: Graph Embedding & Laplacianface
Zhu LiDept of CSEE, UMKC
Office: FH560E, Email: [email protected], Ph: x 2346.http://l.web.umkc.edu/lizhu
Z. Li: ECE 5582 Computer Vision, 2019. p.1
slides created with WPS Office Linux and EqualX LaTex equation editor
Outline
Recap: Eigenface Fisherface HW-3: Fisherface and Laplacian face
Graph Embedding Laplacianface Graph Fourier Transform
Summary
p.2Z. Li: ECE 5582 Computer Vision, 2019.
Subspace Learning for Face Recognition
Project face images to a subspace with basis A Matlab: x=faces*A(:,1:kd)
eigf1
eigf2
eigf3
= 10.9* + 0.4* + 4.7*
p.3Z. Li: ECE 5582 Computer Vision, 2019.
PCA & Fisher’s Linear Discriminant
• Between-class scatter
• Within-class scatter
• Where– c is the number of classes– i is the mean of class i
– | i | is number of samples of i..
Tii
c
iiBS ))((
1 --=å
=
å å= Î
--=c
i x
TikikW
ik
xS1
))((
1
2
1 2
p.4Z. Li: ECE 5582 Computer Vision, 2019.
Eigen vs Fisher Projection
p.5
• PCA (Eigenfaces)
Maximizes projected total scatter
• Fisher’s Linear Discriminant
Maximizes ratio of projected between-class to projected within-class scatter, solved by the generalized Eigen problem:
WSWW TT
WPCA maxarg=
WSW
WSWW
WT
BT
Wfld maxarg=
1 2
PCA
Fisher
Z. Li: ECE 5582 Computer Vision, 2019.
LDA implementation myLDA.m
p.6
% compute class meanmx = mean(x); ids = unique(y); m = length(ids);Sb = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % class mean m_cx(k,:) = mean(x(indx, :)); % between class scatter Sb = Sb + nk(k)*(m_cx(k,:) - mx)'*(m_cx(k,:) - mx);end
% compute intra-class scatterSw = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % remove mean xk = x(indx, :) - repmat(m_cx(k,:), [nk(k), 1]); % adding up Sw = Sw + (xk'*xk);end
Z. Li: ECE 5582 Computer Vision, 2019.
LDA Implementation
Find projection by generalized Eigen problem solution
p.7
% generalized eigen problem[A, v]=eigs(Sb, Sw); if dbg figure(31); subplot(2,2,1); imagesc(Sb); colormap('gray'); title('S_b'); subplot(2,2,2); imagesc(Sw); colormap('gray'); title('S_w'); z = x*A; dist = pdist2(z, z); subplot(2,2,3); imagesc(dist); title('dist(j,k)'); subplot(2,2,4); stem(diag(v), '.'); grid on; hold on; title('eig v');end
Z. Li: ECE 5582 Computer Vision, 2019.
Fisherface Basis
It is interesting to compare Fisherface with Eigenface basis
p.8
FisherfaceEigenface
Z. Li: ECE 5582 Computer Vision, 2019.
Fisherface Performance
Fisher vs Eigenface performance: (fisherface.m) 1200 face images, 144 subjects Eigen kd=32
p.9
144 subjectsROC
Z. Li: ECE 5582 Computer Vision, 2019.
Outline
Recap: Eigenface Fisherface
Graph Embedding Locality Preserving Projection Laplacian Face Graph Fourier Transform
Summary
p.10Z. Li: ECE 5582 Computer Vision, 2019.
Locality Preserving Projection
Recall the dimension reduction formulations: find w, s.t y=wx: PCA:
LDA:
p.11
S = SB + SW
Z. Li: ECE 5582 Computer Vision, 2019.
LPP Formulation – Affinity
To preserve local affinity relationship Affinity map
Selection of heat kernel size and threshold are important Hint: affinity matrix should be sparse
p.12
affinity histogramZ. Li: ECE 5582 Computer Vision, 2019.
LPP Formulation – Affinity Supervised
How to utilize the label info ? Heat map mapping is good for intra-class affinity modelling,
but how about intra-class affinity ? One direct solution is to set affinity to zero for intra class
pairs
p.13
% LPP - compute affinityf_dist1 = pdist2(x1, x1);% heat kernel sizemdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); id_dist = pdist2(ids, ids);subplot(2,2,3); imagesc(id_dist); title('label distance');S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(S1); colormap('gray'); title('affinity-supervised');
Z. Li: ECE 5582 Computer Vision, 2019.
LPP- Affinity Preserving Projection
Find a projection that best preserves the affinity matrix
p.14Z. Li: ECE 5582 Computer Vision, 2019.
LPP Formulation
L = D-S: nxn matrix, called graph Laplacian
Normalizing factor: nxn D Diagonal matrix, entry Dii = sum of col/row affinity The larger the value, the more important data point is
Constraint on D:
p.15Z. Li: ECE 5582 Computer Vision, 2019.
Generalized Eigen Problem
Now the formulation is,
Lagranian
by KKT (Karush-Khun-Tucker) Condition, it is solved by a generalized Eigen problem
p.16Z. Li: ECE 5582 Computer Vision, 2019.
X He, S Yan, Y Hu, P Niyogi, HJ Zhang, “Face Recognition Using Laplacianface”, IEEE Trans PAMI, vol. 27 (3), 328-340, 2005.
Matlab Implementation
laplacianface.m
p.17
%LPPn_face = 1200; n_subj = length(unique(ids(1:n_face)));
% eigenface kd=32; x1 = faces(1:n_face,:)*A1(:,1:kd); ids=ids(1:n_face); % LPP - compute affinityf_dist1 = pdist2(x1, x1);% heat kernel sizemdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); figure(32); subplot(2,2,1); imagesc(f_dist1); colormap('gray'); title('d(x_i, d_j)');subplot(2,2,2); imagesc(S1); colormap('gray'); title('affinity'); %subplot(2,2,3); grid on; hold on; [h_aff, v_aff]=hist(S(:), 40); plot(v_aff, h_aff, '.-'); % utilize supervised infoid_dist = pdist2(ids, ids);subplot(2,2,3); imagesc(id_dist); title('label distance');S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(S1); colormap('gray'); title('affinity-supervised');
% laplacian facelpp_opt.PCARatio = 1; [A2, eigv2]=LPP(S2, lpp_opt, x1);
Z. Li: ECE 5582 Computer Vision, 2019.
Laplacian Face
Now, we can model face as a LPP projection:
p.18
Eigenface Laplacian Face
Z. Li: ECE 5582 Computer Vision, 2019.
Laplacian vs Eigenface
1200 faces, 144 subjects
p.19Z. Li: ECE 5582 Computer Vision, 2019.
LPP and PCA Graph Embedding is an unifying theory on dimension
reduction PCA becomes special case of LPP, if we do not enforce local
affinity
p.20Z. Li: ECE 5582 Computer Vision, 2019.
LPP and LDA
How about LDA ? Recall within class scatter:
p.21
This is i-th classData covariance
Li has diagonal entry of 1/ni,Equal affinity among data points
Z. Li: ECE 5582 Computer Vision, 2019.
LPP and LDA
Now consider the between class scatter C is the data covariance,
regardless of label L is graph Laplacian computed
from the affinity rule that,
p.22Z. Li: ECE 5582 Computer Vision, 2019.
LDA as a special case of LPP
The same generalized Eigen problem
p.23Z. Li: ECE 5582 Computer Vision, 2019.
Graph Embedding Interpretation of PCA/LDA/LPP
Affinity graph S, determines the embedding subspace W, via
PCA and LDA are special cases of Graph Embedding PCA:
LDA
LPP
p.24Z. Li: ECE 5582 Computer Vision, 2019.
Applications: facial expression embedding
Facial expressions embedded in a 2-d space via LPP
p.25
frown
sad
happy
neutral
Z. Li: ECE 5582 Computer Vision, 2019.
Application: Compression of SIFT
Compression of SIFT, preserve matching relationship, rather than reconstruction:
Z. Li: ECE 5582 Computer Vision, 2019. p.26
Homework-3: Subspace Methods
Objective: Understand the graph embedding connections among popular
subspace methods like PCA, LDA and LPP Practical experiences with serious size data set
Data Set: https://umkc.app.box.com/s/0qu7tc3jb88at2h53l1dpcuqkt9pn
7ww 417 subjects, 6650 image face data set, pre-processed to
20x20 pel images, intensity normalized to [0, 1] Add your own face images, 10~15, frontal
Tasks: Compute Eigenface, Fisherface and Laplacianface models ROC plot on verification performance mAP for retrieval/identification performance
Z. Li: ECE 5582 Computer Vision, 2019. p.27
HW-3 test run
Laplacian face is powerful.
Z. Li: ECE 5582 Computer Vision, 2019. p.28
Graph Fourier Transform David I. Shuman, Sunil K. Narang, Pascal Frossard, Antonio Ortega, Pierre Vandergheynst:
The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains. IEEE Signal Process. Mag. 30(3): 83-98 (2013)
Z. Li: ECE 5582 Computer Vision, 2019. p.29
Signal on Graph
non-uniformly sampled
p.30Z. Li: ECE 5582 Computer Vision, 2019.
Graph Fourier Transform
GFT is different from Laplacian Embedding:
p.31Z. Li: ECE 5582 Computer Vision, 2019.
GFT Example
Graph Laplacian
p.32Z. Li: ECE 5582 Computer Vision, 2019.
Normalized Graph Laplacian
Normalize by edge pair degree
p.33Z. Li: ECE 5582 Computer Vision, 2019.
Graph Frourier Transform
Analogous to FT
p.34Z. Li: ECE 5582 Computer Vision, 2019.
Graph Spectrum
p.35Z. Li: ECE 5582 Computer Vision, 2019.
Graph Signal Smoothness
Quadratic form on L:
p.36Z. Li: ECE 5582 Computer Vision, 2019.
Summary
Graph Laplacian Embedding is an unifying theory for feature space dimension reduction PCA is a special case of graph embedding
o Fully connected affinity map, equal importance LDA is a special case of graph embedding
o Fully connected intra classo Zero affinity inter class
LPP: preserves pair wise affinity. GFT: eigen vectors of graph Laplacian, has Fourier Transform
like characteristics. Many applications in Face recognition Pose estimation Facial expression modeling Compression of Graph signals.
p.37Z. Li: ECE 5582 Computer Vision, 2019.