Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology 1 Recognizing Partially Occluded, Expression Variant Faces from Single Training Image per Person with SOM and soft kNN Ensemble Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author :X Tan, S Chen, ZH Zhou, F Zhang

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 2  Motivation  Object  Architecture  Introduction  The propose method  Experiments  Conclusion  Opinion Guide

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 3 Motivation  In many real-world applications only one training image per person is available.  The test images may be partially occluded or may vary in expressions.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 4 Object  This paper using the SOM to learn the subspace that represented each individual.  And then it uses a soft k nearest neighbor (soft k-NN) ensemble method to identify the unlabelled subjects.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 5 Architecture  Although template-based methods have become one of the main techniques, a large training data set is not always possible in many real world tasks.  Beside above problem, there exist other problems, such as occlusion and expression.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 6 Architecture (cont.)  This paper extends Martinez’s work using SOM and soft kNN and then it achieves high performance.  The procedure is as follows: ─ Localization ─ The use of SOM The Single SOM-face Strategy The Multiple SOM-face Strategy ─ Identification

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 7 Architecture (cont.)  Finally, this paper have conducted various experiments to verify the performance of the proposed method.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 8 Introduction  Face Recognition Technology (FRT) has a variety of potential applications in many aspect.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 9 Introduction (cont.)  However, the general face recognition problem is still unsolved due to its inherent complexity.  To overcome this problem is to Search one or more face subspaces of the face to lower the influence of the variations.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 10 Introduction (cont.)  Most template-based FRT assume that multiple images per person are available for training.  But a large training data set is not always possible in many real world tasks.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 11 The Proposed Method  A.Localizing the face image: ─ the original image is divided into M(=l/d) sub-blocks with equal size, where l and d are the dimensionalities of the whole image and each sub-block. Image Localization Images

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 12 The Proposed Method (cont.)  B. The use of SOM ─ The SOM is chosen for several reasons as follows: It is efficient and suitable for high dimensional process Its algorithm is more robust to initialization than any other The trained SOM map are similar to input sub-blocks. SOM Projection Image Localization Soft kNN Ensemble Decision Images Results

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 13 The Proposed Method (cont.) ─ The Single SOM-face Strategy Step1:according to: Partition all the sub-blocks into Voronoi regions Setp2: average : Setp3: Smooth : ─ The multiple SOM-face Strategy new image be presented to the system, denoted as Then a separate small SOM map for the face will be trained using the above SOM algorithm.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 14 The Proposed Method (cont.)  C. Identification Given C classes, to decide which class the test face x belongs to, we first divide the test face into M sub-blocks. and then project those sub-blocks onto the trained SOM maps. Arranging it in increasing order : normalization : Finally, the label can be obtained :

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 15 Experiments  On the AR database (variations in Facial Expressions) ─ the neutral expressions images of the 100 individuals were used for training, while the smile, anger and scream images were used for testing.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 16 Experiments (cont.)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 17 Experiments (cont.)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 18 Experiments (cont.)  On the AR database (variations in partially occluded) ─ Simulated occlusion The number of the training data is same, while the smiling, angry and screaming images with simulated partial occlusions were used for testing.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 19

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 20 Experiments (cont.)  We can find that half face occlusion does not harm the performance except the occlusion of upper face (see Fig.8b).  Because the lower half, included the mouth and cheeks, which can be easily affected by most facial expression variation.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 21 Experiments (cont.) ─ Real occlusion the neutral expression images of the 100 individuals were used for training, while the occluded images were used for testing.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 22 Experiments (cont.)  It is interesting to note that the occlusion of the eyes area led to better recognition results because the scarf occluded each face irregularly.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 23 Experiments (cont.)  To simulate the occlusion, we randomly localized a square of size pxp (5<p<50) pixels in each of the four testing image.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 24 Experiments (cont.)  On the FERET database ─ Experiment 1 the performance of the two SOM-face based algorithms on the subset was evaluated and was compared with other two method’s.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 25 Experiments (cont.) ─ Experiment 2 choosing an appreciate k-value for the soft k-NN classifier.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 26 Experiments (cont.) ─ Experiment 3 The effect of different sub-block sizes is studied.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 27 Experiments (cont.) ─ Experiment 4 To investigate the incremental learning capability of the MSOM strategy, experiment was conducted using different gallery sizes.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 28 Experiments (cont.) ─ Experiment 5 we repeated one of the simulated occlusion experiments done on the AR dataset.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 29 Conclusion  This paper introduce the “SOM-face” to address the problem of face recognition with one training image per person and has several advantages over some of the previous methods.  It attributes these advantages to the seamless connection between the three parts of the method. SOMImage

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 30 Conclusion (cont.)  But the proposed method assumes that occluded is known in advance.  This paper shows that this paradigm works well in the scenario of face recognition with one training image per person.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 31 Opinion  Advantage