Face Recognition

Introduction Why we are interested in face recognition? Why we are interested in face recognition? Passport control at terminals in airports Passport control at terminals in airports Participant identification in meetings Participant identification in meetings System access control System access control Scanning for criminal persons Scanning for criminal persons

Face Recognition Face is the most common biometric used by humans Applications range from static, mug-shot verification to a dynamic, uncontrolled face identification in a cluttered background Challenges: automatically locate the face recognize the face from a general view point under different illumination conditions, facial expressions, and aging effects

Authentication vs Identification Face Authentication/Verification (1:1 matching) Face Identification/recognition(1:n matching)

Applications It is not fool proof – many have been fooled by identical twins It is not fool proof – many have been fooled by identical twins Because of these, use of facial biometrics for identification is often questioned. Because of these, use of facial biometrics for identification is often questioned.

Application Video Surveillance (On-line or off-line) locates and extracts images from video footage for identification and verification

Why is Face Recognition Hard? Many faces of Madonna

Why is Face Recognition Hard?

Face Recognition Difficulties Identify similar faces (inter-class similarity) Accommodate intra-class variability due to: head pose illumination conditions expressions facial accessories aging effects Cartoon faces

Inter-class Similarity Different persons may have very similar appearance TwinsFather and son

Intra-class Variability Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

Sketch of a Pattern Recognition Architecture

Example: Face Detection Scan window over image. Classify window as either: Face Non-face

Profile views Schneiderman’s Test set as an example

Example: Finding skin Non-parametric Representation of CCD Skin has a very small range of (intensity independent) colors, and little texture Compute an intensity-independent color measure, check if color is in this range, check if there is little texture (median filter) See this as a classifier - we can set up the tests by hand, or learn them. get class conditional densities (histograms), priors from data (counting) Classifier is

Face Detection Algorithm

Face Recognition

Face Recognition: 2-D and 3-D

Consider an n-pixel image to be a point in an n- dimensional space, Each pixel value is a coordinate of x. Image as a Feature Vector

Nearest Neighbor Classifier Rj is the training dataset Rj is the training dataset The match for I is R1, who is closer than R2 The match for I is R1, who is closer than R2

Comments Sometimes called “Template Matching” Variations on distance function Multiple templates per class- perhaps many training images per class. Expensive to compute k distances, especially when each image is big (N dimensional). May not generalize well to unseen examples of class. Some solutions: Bayesian classification Dimensionality reduction

Holistic or Appearance-based Face recognition Holistic or Appearance-based Face recognition EigenFace EigenFace LDA LDA Feature-based Feature-based Face Recognition Solutions

EigenFace

EigenFace Use Principle Component Analysis (PCA) to determine the most discriminating features between images of faces. The principal component analysis or Karhunen- Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes. The principal component analysis or Karhunen- Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes.

PCA torials/principal_components.pdf torials/principal_components.pdf torials/principal_components.pdf torials/principal_components.pdf

More New Techniques in Face Biometrics Facial geometry, 3D face recognition Facial geometry, 3D face recognition users.cs.york.ac.uk/~nep/research/3Dface /tomh/3DFaceDatabase.html 3D reconstruction

Skin pattern recognition using the details of the skin for authentication using the details of the skin for authentication orange.fr/fingerchip/biometrics/types/face.htm orange.fr/fingerchip/biometrics/types/face.htm

Facial thermogram Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in Now disappeared. Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in Now disappeared. orange.fr/fingerchip/biometrics/types/face.htm orange.fr/fingerchip/biometrics/types/face.htm

Side effect of Facial thermogram can detect lies can detect lies The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. orange.fr/fingerchip/biometrics/types/face.htm orange.fr/fingerchip/biometrics/types/face.htm

Smile recognition Probing the characteristic pattern of muscles beneath the skin of the face. Probing the characteristic pattern of muscles beneath the skin of the face. Analyzing how the skin around the subject's mouth moves between the two smiles. Analyzing how the skin around the subject's mouth moves between the two smiles. Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. orange.fr/fingerchip/biometrics/types/face.htm orange.fr/fingerchip/biometrics/types/face.htm

Dynamic facial features They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. orange.fr/fingerchip/biometrics/types/face.htm orange.fr/fingerchip/biometrics/types/face.htm