CONTENT BASED FACE RECOGNITION Ankur Jain 01D05007 Pranshu Sharma Prashant Baronia 01D05005 Swapnil Zarekar 01D05001 Under the guidance of Prof. Pushpak Bhattacharya

Introduction Problem Statement :  Given an image, to identify it as a face and/or extract face images from it.  To retrieve the similar images (based on a heuristic) from the given database of face images.

Why face recognition ? Various potential applications, such as  person identification.  human-computer interaction.  security systems.

 Faces are complex, multidimensional and meaningful visual stimuli.  Face Recognition is difficult.  Face Images are similar in overall configuration. Difference From Image Recognition

Approach  Similar to Content Based Image Retrieval (CBIR).  Neural Networks and Self Organizing Maps (SOMs).  Principal Component Analysis (PCA).  Relevance feed back.

Stages of Face Recognition (1) face location detection (2) feature extraction (3) facial image classification Approaches of Feature Extraction (1) local feature : eyes, nose, mouth information easily affected by irrelevant information. easily affected by irrelevant information. (2) global feature : extract feature from whole image. extract feature from whole image.

Face Recognition Using Eigenfaces

 Face Images are projected into a feature space (“Face Space”) that best encodes the variation among known face images.  The face space is defined by the “eigenfaces”, which are the eigenvectors of the set of faces. Eigen Space and Eigen Faces

 Initialization :  Acquire the training set and calculate eigenfaces (using PCA projections) which define eigenspace.  When a new face is encountered, calculate its weight.  Determine if the image is face.  If yes, classify the weight pattern as known or unknown.  (Learning) If the same unknown face is seen several times incorporate it into known faces. Steps In Face Recognition

PCA Main assumption of PCA approach: Main assumption of PCA approach:  Face space forms a cluster in image space.  PCA gives suitable representation.

Eigenfaces (1)  Calculation of Eigenfaces (1) Calculate average face : v. (2) Collect difference between training images and average face in matrix A (M by N), where M is the number of pixels and N is the number of images. (3) The eigenvectors of covariance matrix C (M by M) give the eigenfaces.  M is usually big, so this process would be time consuming. What to do?

Eigenfaces (2)  Calculation of Eigenvectors of C If the number of data points is smaller than the dimension (N<M), then there will be only N-1 meaningful eigenvectors. Instead of directly calculating the eigenvectors of C, we can calculate the eigenvalues and the corresponding eigenvectors of a much smaller matrix L (N by N). if λ i are the eigenvectors of L then A λ i are the eigenvectors for C.  The eigenvectors are in the descent order of the corresponding eigenvalues.

Eigenfaces (3)  Representation of Face Images using Eigenfaces  The training face images and new face images can be represented as linear combination of the eigenfaces.  When we have a face image u : Since the eigenvectors are orthogonal :

Eigenfaces (4)  Experiment and Results Data used here are from the ORL database of faces. Facial images of 16 persons each with 10 views are used. - Training set contains 16×7 images. - Test set contains 16×3 images. First three eigenfaces :

Classification Using Nearest Neighbor  Save average coefficients for each person. Classify new face as the person with the closest average.  Recognition accuracy increases with number of eigenfaces till 15. Later eigenfaces do not help much with recognition. Later eigenfaces do not help much with recognition. Best recognition rates Training set 99% Test set 89% Test set 89%

Neural Networks and TS-SOM

What are Neural Networks ?  Individual units to simulate Neurons  Parallel Processing  Many inputs and single output  Organization/structure of the TLU’s is important

What is SOM ?  TS-SOM :- Tree structure self-organizing maps  Competitive learning ANN  Each unit of map receives identical inputs  Units compete for selection  Modification of selected node and its neighbors

Training of SOM  Randomly initialized  Selection based on some query parameter  On selection a node and its neighbors are modified  Degree of modification reduces with each iteration

Example of a two-dimensional TS-SOM structure of 3 levels

Algorithm  Calculate weight vector for first level.  Initialize weight vectors of other levels.  Calculate centroid associated to each node as mean of closest training samples.  Iterate to the next level.

Relevance Feedback  System content based retrieval.  Point of human intervention  User analysis of system output.  User selects most relevant  Query iterated if output not satisfactory

Interaction Between User & System 1.A random set of faces is presented to the user. 2.User interactive selection of faces. 3.System content-based face retrieval. 4.User analysis of retrieved faces.  Requested face was found -> Exit  Similar faces were found. -> Go to 2  No similar faces were found.  User tired -> Exit  User not tired (re initialization -> Go to 1

Comparison of the Two Approaches  Training time Nearest neighbor is much faster.  Storage About the same.  Classification time Nearest neighbor is slightly slower.  Accuracy Neural network is able to achieve the same accuracy using 5 eigenfaces with nearest neighbor using 15, and a higher accuracy when using 15. Neural network models the problem better, but takes more training time. Neural network models the problem better, but takes more training time.

Future Work  Face Detection in motion pictures.  Detailed study of the proposed system assuming PCA assumptions not to be true.  Investigate whether eigenfaces is a good solution for this problem by comparing with other feature extraction techniques such as DCT

References  Navarrete P. and Ruiz-del-Solar J. (2002), “Interactive Face Retrieval using Self-Organizing Maps”, 2002 Int. Joint Conf. on Neural Networks – IJCNN 2002, May 12-17, Honolulu, USA.  “A tutorial on Principal Components Analysis”, By Lindsay I Smith.  “Eigenfaces for Recognition”, Turk, M. and Pentland A., (1991) Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp  Ruiz-del-Solar, J., and Navarrete, P. (2002). “Towards a Generalized Eigenspace-based Face Recognition Framework”, 4th Int. Workshop on Statistical Techniques in Pattern Recognition, August 6-9, Windsor, Canada.  Simulating Neural Networks by James A. Freeman.  Artificial Intelligence by Neil J. Nielsson.