1. Face Recognition and Biometric Systems
Eigenfaces (3)
Face Recognition and Biometric Systems

2. Face Recognition and Biometric Systems
Plan of the lecture
Eigenfaces-based methods
Fisherfaces
Bayesian Matching
Local PCA
Face relevance maps
Error function minimisation
Eigenfaces – feature extraction
definition of recognition error
optimal masks and weights
Face Recognition and Biometric Systems

3. Eigenfaces – drawbacks
Main drawbacks:
holistic method
face topology not taken into account
statistical analysis of differences between images in the training set
character of differences not taken into account
Face Recognition and Biometric Systems

4. Face Recognition and Biometric Systems
Example
Face Recognition and Biometric Systems

5. Face Recognition and Biometric Systems
Example: PCA
Face Recognition and Biometric Systems

6. Example: PCA not helpful
Face Recognition and Biometric Systems

7. Example: Linear Discriminant Analysis
Face Recognition and Biometric Systems

8. Face Recognition and Biometric Systems
Fisherfaces
PCA finds main directions of variance
class identity not utilised
Methods based on PCA which utilise class identity:
Linear Discriminant Analysis (LDA)
Fisherfaces
Face Recognition and Biometric Systems

9. Face Recognition and Biometric Systems
Fisherfaces
Principal Component Analysis:
training set  covariance matrix
Linear Discriminant Analysis:
classified training set  two covar. matrices
within-class covariance matrix
between-class covariance matrix
orthogonal basis from two matrices
Face Recognition and Biometric Systems

10. Face Recognition and Biometric Systems
Fisherfaces
Between-class matrix
CB – between-class covariance matrix
C – number of classes
Mi – number of images in i-th class
 – average image
i – average image of i-th class
Face Recognition and Biometric Systems

11. Face Recognition and Biometric Systems
Fisherfaces
Within-class covariance matrix
CW – within-class covariance matrix
C – number of classes
Xi – set of images which belong to i-th class
xk – k-th image which belongs to i-th class
i – average image of i-th class
Face Recognition and Biometric Systems

12. Face Recognition and Biometric Systems
Fisherfaces
PCA:
 - eigenvectors matrix (vectors in columns)
LDA:
Face Recognition and Biometric Systems

13. Face Recognition and Biometric Systems
Fisherfaces
LDA – hard to find inverse matrix
Fisherfaces – improved approach:
PCA for dimensionality reduction
LDA for finding optimal orthogonal basis
Face Recognition and Biometric Systems

14. Face Recognition and Biometric Systems
Fisherfaces
Feature extraction in the Fisherfaces:
Feature vector calculated by PCA
normalised image as an input
dimensionality reduction
Feature vector calculated by LDA
PCA feature vector as an input
rotation of feature vector
no dimensionality reduction
Face Recognition and Biometric Systems

15. Face Recognition and Biometric Systems
Bayesian Matching
Vectors similarity based on probability of their difference classification
I – set of intra-personal pairs
E – set of extra-personal pairs
Face Recognition and Biometric Systems

16. Face Recognition and Biometric Systems
Bayesian Matching
P(|) – probability of observing a given difference in a defined set of differences
function of PCA back projection error – ()
Face Recognition and Biometric Systems

17. Face Recognition and Biometric Systems
Bayesian Matching
Two classes of image pairs
intra- and extra-personal
Differences generated from pairs
two classes of pairs
PCA used for both classes separately
two image spaces
Face Recognition and Biometric Systems

18. Face Recognition and Biometric Systems

19. Face Recognition and Biometric Systems
Bayesian Matching
Image difference recognition
Dual Eigenfaces
Difference distance from two image spaces
Bayesian Matching – a slow method
image difference calculated for every comparison
possibility of applying other method for selecting candidates (n most similar images)
Face Recognition and Biometric Systems

20. Face Recognition and Biometric Systems
Local PCA
Based on detected features
eyes, nose, mouth
PCA for features
small part of face image
analysis of small images (eigeneyes, eigennoses, etc.)
Less dimensional spaces
Lower effectiveness, but supports the Eigenfaces
Face Recognition and Biometric Systems

21. Face Recognition and Biometric Systems
Local PCA K1 K2 K3 K4
Face Recognition and Biometric Systems

22. Face Recognition and Biometric Systems
Other methods
Local Feature Analysis
2D PCA, 2D LDA
Independent Component Analysis
Face Recognition and Biometric Systems

23. Face Recognition and Biometric Systems
Face relevance map
Face topology
eyes & nose – extra-personal differences
mouth & cheeks – intra-personal differences
Nature of features concerned with location
Face Recognition and Biometric Systems

24. Face Recognition and Biometric Systems
Face relevance map
Face relevance map
enhance influence of extra-personal features
decrease influence of intra-personal features
Feature extraction with a map (m)
Face Recognition and Biometric Systems

25. Face Recognition and Biometric Systems
Face relevance map
„T” map
artificial map for eyes and nose
binary values
Results:
FeretA: 423 -> 445 (3,7%)
Conclusion: good approach, need for better map generation methods
Face Recognition and Biometric Systems

26. Face Recognition and Biometric Systems
Face relevance map
Difference map – statistical analysis
Pairs of images:
intra-personal
extra-personal
Average differences between images:
average intra-personal difference
average extra-personal difference
Map obtained by subtracting intra-personal difference from extra-personal one
Results for FeretA: 423 -> 462 (6,4%)
Face Recognition and Biometric Systems

27. Face Recognition and Biometric Systems
Face relevance map
Colour data
information lost during conversion to GS
low distinctiveness
can be used for map generation
Colour used for detection
eye and mouth map
masks based on detection maps
Face Recognition and Biometric Systems

28. Face Recognition and Biometric Systems
Face relevance map
Desired effect:
higher values around eyes and nose
lower values in the area of mouth
Maps deliver information about features location
Two possible approaches:
image -> feature maps -> face relevance map
image -> feature maps -> features -> f.r.m.
Face Recognition and Biometric Systems

29. Face Recognition and Biometric Systems
Face relevance map
Maps from points
Nose location derived from eye & mouth
weighted mean
eye(R): (15, 24)
eye(L): (49, 24)
mouth: (32, 58)
Face Recognition and Biometric Systems

30. Face Recognition and Biometric Systems
Face relevance map
Single point influence
r – radius, mmax – maximal map value
Map – summed influence of the points
eye, nose – positive weights
mouth – negative weights
Face Recognition and Biometric Systems

31. Face Recognition and Biometric Systems
Face relevance map
Maps from colour
improvement comparable to difference maps
colour data carry information concerning nature of face areas
generated for every image
Map may be imposed during normalisation
Face Recognition and Biometric Systems

32. Face Recognition and Biometric Systems
Face relevance map
Maps from colour - examples
Face Recognition and Biometric Systems

33. Face Recognition and Biometric Systems
Face relevance map
Maps from colour - examples
Face Recognition and Biometric Systems

34. Face Recognition and Biometric Systems
Face relevance map
Back-projection based dynamic map
dynamic – created for every image
Back projection:
map of local projection error
higher error = lower importance
map should be smoothed
Good for occluded images
Face Recognition and Biometric Systems

35. Face Recognition and Biometric Systems
Face relevance map
Back-projection based dynamic map
examples of occluded face images
Face Recognition and Biometric Systems

36. Face Recognition and Biometric Systems
Recognition error
Maps take into account difference nature basing on face topology
Differences not concerned with location
lighting
Eigenfaces – appearance interpretation
various types of information
some responsible for lighting
Weights assigned to eigenvectors:
Face Recognition and Biometric Systems

37. Face Recognition and Biometric Systems
Recognition error
Eigenvectors weights
lower values for intra-personal directions of variance
How to obtain the weights?
visual assessment – may be incorrect
the same procedure as in the case of difference masks
Face Recognition and Biometric Systems

38. Face Recognition and Biometric Systems
Recognition error
A better method for obtaining maps and eigenvector weights:
error function minimisation
Face Recognition and Biometric Systems

39. Face Recognition and Biometric Systems
Recognition error
Definition of recognition problem:
M vectors, C classes and C base vectors (ui1)
Mi vectors in i-th class (uij)
classification of non-base vectors (j > 1)
Single comparison
similarity to home class and foreign class
classes represented by base vectors
Face Recognition and Biometric Systems

40. Face Recognition and Biometric Systems
Recognition error
Single comparison error:
uij – a vector which is being recognised
ui1 – home class base vector
uk1 – foreign class base vector
S – similarity between vectors
Face Recognition and Biometric Systems

41. Face Recognition and Biometric Systems
Recognition error
Single comparison:
correct if
incorrect if
Face Recognition and Biometric Systems

42. Face Recognition and Biometric Systems
Recognition error
Error for comparison with all classes:
Error for the whole set:
Face Recognition and Biometric Systems

43. Eigenfaces: feature extractionK1 K2 K3
...
...
Scalar products between
normalised image and
eigenvectors
Feature vector
Face Recognition and Biometric Systems

44. Eigenfaces: feature extraction
Feature vector element ( ):
- dimensionality of feature vector
- normalised face image
- i-th eigenvector
Improvements to the Eigenfaces
face relevance masks
eigenvector weights
Face Recognition and Biometric Systems

45. Eigenfaces: feature extraction
Improved feature extraction:
- i-th eigenvector weight
- j-th element (pixel) of the mask
- j-th element of the i-th eigenvector
- i-th element of the feature vector
Face Recognition and Biometric Systems

46. Eigenfaces: feature extraction
Similarity based on Euclidean distance:
Face Recognition and Biometric Systems

47. Face Recognition and Biometric Systems
Error minimisation
Recognition error is a function of mask and eigenvector weights
The function may be minimised
optimal mask
optimal eigenvector weights
Example of mask optimisation...
Face Recognition and Biometric Systems

49. Face Recognition and Biometric Systems
Error minimisation
Optimised dataset
problem of overfitting
How to avoid overfitting?
large datasets
optimisation can be stopped
Advantages of overfitting
overfitting to a group of people
Face Recognition and Biometric Systems

52. Face Recognition and Biometric Systems
Summary
There are many methods derived from the Eigenfaces
Error is a function of masks and eigenvectors weights
Classification parameters can be optimised
Improvement aims at:
including face topology
feature analysis
difference classification
Face Recognition and Biometric Systems

53. Thank you for your attention!
Next time:
Face detection
Face Recognition and Biometric Systems