Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona State University Tempe, AZ, qzhang53,

Problem Description In many applications, we may acquire multiple copies of signals from the same source (an ensemble of signals); Signals in ensemble may be very similar (sharing a common source), but may also have very distinctive differences (e.g., very different acquisition conditions) plus other unique but small variations (e.g., sensor noise).

Examples Common sources

Examples Different acquisition conditions

Examples Sensor noises

Examples Signals in ensemble

Signals in Ensemble The decomposition of the signal has several benefits: – Obtaining better compression rate. E.g., distributed compressed sensing [Duarte], joint sparsity model [Duarte 2005]; – Extract more relevant features. E.g., A compressive sensing approach for expression- invariant face recognition [Nagesh 2009];

Example: Face Images Given face images of same subjects

Example: Face Images Can we identify a “clean” image for them?

Example: Face Images And their illumination conditions?

Example: Face Image Set Face image can be represented as a variable parameterized by subject and imaging condition, e.g., illumination, expression etc.; The images of same subject would share a common component; The innovation components should contribute to the imaging conditions. – Innovation component may not be necessary sparse, e.g., the one due to illumination variations.

Example: Decomposing Face Images AR dataset: expression variation as a sparse component. X C E

Example: Decomposing Face Images X C A

Proposed Model

Proposed Model Cont’d

Examples X = C + A + E

Solving the Decomposition

Comparison with Other Models

Decomposition Algorithms

Decomposition Algorithms Cont’d

Parameters Selection

Convergence Property With proper selection of parameters, the proposed algorithm will converge to the optimal solution. In experiment, the algorithm converges within 100 iterations.

Examples: Decomposition Process Show examples for how the algorithm converges.

Experiment We use three experiments to evaluate the proposed model and algorithm: – Decomposing the synthetic images; – Decomposing the images from extended YaleB dataset; – Applying decomposed component to classification tasks;

Decomposing the synthetic images We create training images by mixing the images with low-rank background images. In addition, we add some sparse- supported noise.

Decomposing the synthetic images The decomposition result. From top to bottom: common components, low- rank components and sparse components.

Decomposition: Robustness over Missing Training Instances We randomly remove 20% training images and test the robustness of decomposition algorithm. From top to bottom: training images and common component, low-rank component.

Decomposing Extended YaleB Dataset We use all 2432 images of extended YaleB dataset, which includes 38 subjects and 64 illumination conditions; – Common components capture information unique to certain subjects; – Low rank components capture the illumination conditions of the images; – Sparse components capture the sparse-supported noises and shadows;

Decomposing Extended YaleB Dataset Left: common components; Right: the low-rank components.

Applying Decomposed Component to Face Recognition

Face Recognition: Subspaces Reconstruct image with common component of Subject 1 and low-rank components: (a) coefficient of the reconstruction, (b) the input image, and (c) the reconstructed image. Subject 1Subject 2 Reconstruction with incorrect common components

Face Recognition: Measuring the Distance of Subspaces

Face Recognition: Experiment We test the algorithm on extended YaleB dataset and Multi-PIE dataset; – Randomly split the set into training sets and testing sets; – To test its robustness over missing training instances, we randomly remove some of the training instances and keep “# train per subject” training instances for each subject; – The performance is compared with SRC, Volterraface and SUN.

Face Recognition: Extended YaleB Dataset #train per subject Proposed 99.78±0.24%99.54±0.04%99.18±0.14%95.15±1.03% SRC 96.48±0.44%95.29±0.52%91.90±0.94%78.65±1.81% Volterrafaces 99.95±0.06%99.80±0.26%99.48±0.49%90.22±11.84% SUN 89.61±1.85%87.64±2.80%76.91±3.71%60.17±2.09% #train per subject Proposed 99.56±0.00%99.33±0.23%98.32±0.03%80.03±2.17% SRC 89.14±0.00%87.88±0.44%81.02±0.13%58.54±1.26% Volterrafaces 99.25±0.34%99.17±0.39%96.27±4.03%91.03±2.43% SUN 79.22±0.00%76.75±0.00%68.86±0.00%51.60±0.00% Extended YaleB dataset includes 64 illumination conditions and 38 subjects.

Face Recognition: Multi-PIE Dataset #train per image Proposed 100±0.00% 99.65±0.37%97.49±0.21% SRC 99.88±0.07% 99.73±0.14%97.73±0.54% Volterrafaces 100±0.00% 95.83±4.16% SUN 100%99.84±0.11%99.45±0.43%95.75±0.49% #train per image Proposed 100±0.00% %99.17±0.15%94.70±0.20% SRC 99.91±0.16%98.89±1.74%96.90±3.73%87.18±1.78% Volterrafaces 100±0.00% 99.54±0.31%94.30±4.72% SUN 100±0.00%99.84±0.05%98.53±0.29%88.75±4.72% We test the performances on face images of frontal poses (P27), which include 68 subjects and 45 illumination variations.

Face Recognition: Robustness over Poses Variations we use all the images from 5 near frontal poses (C05, C07, C09, C27, C29), which includes153 conditions for each subject. We randomly pick M=40 illumination conditions for training and the remaining for testing. #train per subject Proposed 99.98±0.03%99.92±0.06%99.24±0.06%90.95±0.70% SRC 99.98±0.03%99.45±0.03%96.79±0.28%86.98±0.16% Volterrafaces 99.60±0.22%98.37±0.47%97.63±0.28%89.72±1.45% SUN 99.93±0.05%99.38±0.14%97.89±0.30%88.29±0.02%

Variation Recognition

We test the proposed algorithm on AR dataset, which contains 100 subjects and 2 sessions, where each session 13 variations. We use the first session for training and second session for testing. We show the confusion matrix, which presents the result in percentages. variations

Conclusions We proposed a novel decomposition of a set of face images of multiple subjects, each with multiple images; It facilitates explicit modeling of typical challenges in face recognition, such as illumination conditions and large occlusion; For future work, we plan to expand the current algorithm by incorporating another step that attempts to estimate a mapping matrix for assigning a condition label to each image, during the optimization iteration.