mining discriminative components with low-rank and sparsity constraints for face recognition qiang...
TRANSCRIPT
![Page 1: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/1.jpg)
Mining Discriminative Components With Low-Rank and
Sparsity Constraints for Face RecognitionQiang Zhang, Baoxin Li
Computer Science and EngineeringArizona State University
Tempe, AZ, 85281qzhang53, [email protected]
![Page 2: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/2.jpg)
Problem Description
• In many applications, we may acquire multiple copies of signals from the same source (an ensemble of signals);
• Signals in ensemble may be very similar (sharing a common source), but may also have very distinctive differences (e.g., very different acquisition conditions) plus other unique but small variations (e.g., sensor noise).
![Page 3: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/3.jpg)
ExamplesCommon sources
![Page 4: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/4.jpg)
ExamplesDifferent acquisition conditions
![Page 5: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/5.jpg)
ExamplesSensor noises
![Page 6: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/6.jpg)
ExamplesSignals in ensemble
![Page 7: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/7.jpg)
Signals in Ensemble
• The decomposition of the signal has several benefits:– Obtaining better compression rate. E.g.,
distributed compressed sensing [Duarte], joint sparsity model [Duarte 2005];
– Extract more relevant features. E.g., A compressive sensing approach for expression-invariant face recognition [Nagesh 2009];
![Page 8: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/8.jpg)
Example: Face Images
• Given face images of same subjects
![Page 9: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/9.jpg)
Example: Face Images
• Can we identify a “clean” image for them?
![Page 10: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/10.jpg)
Example: Face Images
• And their illumination conditions?
![Page 11: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/11.jpg)
Proposed Model
• We represent each image of an ensemble as
– the image of subject;– the image ensemble;– the common part for Subject ;– a low rank matrix;– a sparse matrix;
![Page 12: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/12.jpg)
Solving the Decomposition
• The proposed model can be formulated as:
Subject to
![Page 13: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/13.jpg)
Comparison with Other Models
• Distributed Compressive Sensing;– , s.t., ;– The proposed method is related to DCS. However,
in DCS, the innovation components are typically assumed to be sparse, which limits its application;
– Instead, the proposed model allows the innovation components have more complex structure, e.g., low rank.
• Robust Principle Component Analysis;
![Page 14: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/14.jpg)
Comparison with Other Models
• Distributed Compressive Sensing;• Robust Principle Component Analysis;– , subject to ;– RPCA decompose a set of images into a low rank
matrix and sparse matrix;– The core differences of RPCA from the proposed
method is that, RPCA represents each image as a vector of a big matrix. Those vectors are linear dependent, in addition to some sparse noise;
![Page 15: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/15.jpg)
Decomposition Algorithms
• We apply augmented Lagrange Multiplier to the proposed formulations:
• and are parameters.
![Page 16: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/16.jpg)
Decomposition Algorithms Cont’d
• We use augmented Lagrange Multiplier and block coordinate descent to solve the proposed problem:– Solve for each , while and are fixed;– Solve for each , while and are fixed;– Solve for each , while and are fixed;– Update and as ;– Check the convergence. If not converged, repeat
the previous steps.
![Page 17: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/17.jpg)
Experiment
• We use three experiments to evaluate the proposed model and algorithm:– Decomposing the synthetic images;– Decomposing the images from extended YaleB
dataset;– Applying decomposed component to classification
tasks;
![Page 18: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/18.jpg)
Decomposing the synthetic images
We create training images by mixing the images with low-rank background images. In addition, we add some sparse-supported noise.
![Page 19: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/19.jpg)
Decomposing the synthetic images
• The decomposition result. From top to bottom: common components, low-rank components and sparse components.
![Page 20: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/20.jpg)
Decomposition: Robustness over Missing Training Instances
• We randomly remove 20% training images and test the robustness of decomposition algorithm.
• From top to bottom: training images and common component, low-rank component.
![Page 21: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/21.jpg)
Decomposing Extended YaleB Dataset
• We use all 2432 images of extended YaleB dataset, which includes 38 subjects and 64 illumination conditions;– Common components capture information unique
to certain subjects;– Low rank components capture the illumination
conditions of the images;– Sparse components capture the sparse-supported
noises and shadows;
![Page 22: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/22.jpg)
Decomposing Extended YaleB Dataset
Left: common components; Right: the low-rank components.
![Page 23: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/23.jpg)
Applying Decomposed Component to Face Recognition
• According to previous discussions, common components captures the information essential to the subject and low rank components captures the variation of the images sets;
• Ideally a image from Subject should lie in a subspace spanned by its common component and the low-rank components .
![Page 24: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/24.jpg)
Face Recognition: SubspacesReconstruct image with common component of Subject 1 and low-rank components: (a) coefficient of the reconstruction, (b) the input image, and (c) the reconstructed image.
Subject 1 Subject 2
Reconstruction with incorrect
common components
![Page 25: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/25.jpg)
Face Recognition: Measuring the Distance of Subspaces
• We use principle angles to measure the distance between two subspaces:– Subspace of Subject – Subspace of test image
• Principle angle provides information of relative position of subspaces in Euclidean space.
![Page 26: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/26.jpg)
Face Recognition: Experiment
• We test the algorithm on extended YaleB dataset and Multi-PIE dataset;– Randomly split the set into training sets and
testing sets;– To test its robustness over missing training
instances, we randomly remove some of the training instances and keep “# train per subject” training instances for each subject;
– The performance is compared with SRC, Volterraface and SUN.
![Page 27: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/27.jpg)
Face Recognition: Robustness over Poses Variations
• we use all the images from 5 near frontal poses (C05, C07, C09, C27, C29), which includes153 conditions for each subject. We randomly pick M=40 illumination conditions for training and the remaining for testing.
#train per subject 20 15 10 5Proposed 99.98±0.03% 99.92±0.06% 99.24±0.06% 90.95±0.70%
SRC 99.98±0.03% 99.45±0.03% 96.79±0.28% 86.98±0.16%Volterrafaces 99.60±0.22% 98.37±0.47% 97.63±0.28% 89.72±1.45%
SUN 99.93±0.05% 99.38±0.14% 97.89±0.30% 88.29±0.02%
![Page 28: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/28.jpg)
Variation Recognition
• The proposed method can also be used to identify the variations (e.g., illumination conditions) of the images;
• For this purpose, we construct two subspaces and measure the distances with principle angle:– Subspace of variation – Subspace of test image
![Page 29: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/29.jpg)
Variation Recognition
We test the proposed algorithm on AR dataset, which contains 100 subjects and 2 sessions, where each session 13 variations. We use the first session for training and second session for testing.We show the confusion matrix, which presents the result in percentages.
variations
![Page 30: Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona](https://reader031.vdocuments.us/reader031/viewer/2022032604/56649e685503460f94b64202/html5/thumbnails/30.jpg)
Conclusions
• We proposed a novel decomposition of a set of face images of multiple subjects, each with multiple images;
• It facilitates explicit modeling of typical challenges in face recognition, such as illumination conditions and large occlusion;
• For future work, we plan to expand the current algorithm by incorporating another step that attempts to estimate a mapping matrix for assigning a condition label to each image, during the optimization iteration.