1. Face Recognition

2. Introduction Why we are interested in face recognition? Why we are interested in face recognition? Passport control at terminals in airports Passport control at terminals in airports Participant identification in meetings Participant identification in meetings System access control System access control Scanning for criminal persons Scanning for criminal persons

3. Face Recognition Face is the most common biometric used by humans Applications range from static, mug-shot verification to a dynamic, uncontrolled face identification in a cluttered background Challenges: automatically locate the face recognize the face from a general view point under different illumination conditions, facial expressions, and aging effects

4. Authentication vs Identification Face Authentication/Verification (1:1 matching) Face Identification/recognition(1:n matching)

5. Applications It is not fool proof – many have been fooled by identical twins It is not fool proof – many have been fooled by identical twins Because of these, use of facial biometrics for identification is often questioned. Because of these, use of facial biometrics for identification is often questioned.

6. Application Video Surveillance (On-line or off-line) http://www.crossmatch.com/facesnap-fotoshot.php http://www.crossmatch.com/facesnap-fotoshot.php locates and extracts images from video footage for identification and verification

7. Why is Face Recognition Hard? Many faces of Madonna

8. Why is Face Recognition Hard?

9. Face Recognition Difficulties Identify similar faces (inter-class similarity) Accommodate intra-class variability due to: head pose illumination conditions expressions facial accessories aging effects Cartoon faces

10. Inter-class Similarity Different persons may have very similar appearance TwinsFather and son

11. Intra-class Variability Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

12. Sketch of a Pattern Recognition Architecture

13. Example: Face Detection Scan window over image. Classify window as either: Face Non-face

14. Profile views Schneiderman’s Test set as an example

15. Example: Finding skin Non-parametric Representation of CCD Skin has a very small range of (intensity independent) colors, and little texture Compute an intensity-independent color measure, check if color is in this range, check if there is little texture (median filter) See this as a classifier - we can set up the tests by hand, or learn them. get class conditional densities (histograms), priors from data (counting) Classifier is

17. Face Detection Algorithm

18. Face Recognition

19. Face Recognition: 2-D and 3-D

20. Consider an n-pixel image to be a point in an n- dimensional space, Each pixel value is a coordinate of x. Image as a Feature Vector

21. Nearest Neighbor Classifier Rj is the training dataset Rj is the training dataset The match for I is R1, who is closer than R2 The match for I is R1, who is closer than R2

22. Comments Sometimes called “Template Matching” Variations on distance function Multiple templates per class- perhaps many training images per class. Expensive to compute k distances, especially when each image is big (N dimensional). May not generalize well to unseen examples of class. Some solutions: Bayesian classification Dimensionality reduction

23. Holistic or Appearance-based Face recognition Holistic or Appearance-based Face recognition EigenFace EigenFace LDA LDA Feature-based Feature-based Face Recognition Solutions

24. EigenFace

25. EigenFace Use Principle Component Analysis (PCA) to determine the most discriminating features between images of faces. The principal component analysis or Karhunen- Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes. The principal component analysis or Karhunen- Loeve transform is a mathematical way of determining that linear transformation of a sample of points in L-dimensional space which exhibits the properties of the sample most clearly along the coordinate axes.

26. PCA http://www.cs.otago.ac.nz/cosc453/student_tu torials/principal_components.pdf http://www.cs.otago.ac.nz/cosc453/student_tu torials/principal_components.pdf http://www.cs.otago.ac.nz/cosc453/student_tu torials/principal_components.pdf http://www.cs.otago.ac.nz/cosc453/student_tu torials/principal_components.pdf

27. More New Techniques in Face Biometrics Facial geometry, 3D face recognition Facial geometry, 3D face recognition http://www- users.cs.york.ac.uk/~nep/research/3Dface /tomh/3DFaceDatabase.html 3D reconstruction

28. Skin pattern recognition using the details of the skin for authentication using the details of the skin for authentication http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm

29. Facial thermogram Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in 1996-1999. Now disappeared. Facial thermogram requires an (expensive) infrared camera to detect the facial heat patterns that are unique to every human being. Technology Recognition Systems worked on that subject in 1996-1999. Now disappeared. http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm

30. Side effect of Facial thermogram can detect lies can detect lies The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. The image on the left shows his normal facial thermogram, and the image on the right shows the temperature changes when he lied. http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm

31. Smile recognition Probing the characteristic pattern of muscles beneath the skin of the face. Probing the characteristic pattern of muscles beneath the skin of the face. Analyzing how the skin around the subject's mouth moves between the two smiles. Analyzing how the skin around the subject's mouth moves between the two smiles. Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. Tracking changes in the position of tiny wrinkles in the skin, each just a fraction of a millimetre wide. The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. The data is used to produce an image of the face overlaid with tiny arrows that indicate how different areas of skin move during a smile. http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm

32. Dynamic facial features They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. They track the motion of certain features on the face during a facial expression (e.g., smile) and obtain a vector field that characterizes the deformation of the face. http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm http://pagesperso- orange.fr/fingerchip/biometrics/types/face.htm