Face Recognition and Biometric Systems Eigenfaces (3) Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Plan of the lecture Eigenfaces-based methods Fisherfaces Bayesian Matching Local PCA Face relevance maps Error function minimisation Eigenfaces – feature extraction definition of recognition error optimal masks and weights Face Recognition and Biometric Systems

Eigenfaces – drawbacks Main drawbacks: holistic method face topology not taken into account statistical analysis of differences between images in the training set character of differences not taken into account Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Example Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Example: PCA Face Recognition and Biometric Systems

Example: PCA not helpful Face Recognition and Biometric Systems

Example: Linear Discriminant Analysis Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces PCA finds main directions of variance class identity not utilised Methods based on PCA which utilise class identity: Linear Discriminant Analysis (LDA) Fisherfaces Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces Principal Component Analysis: training set  covariance matrix Linear Discriminant Analysis: classified training set  two covar. matrices within-class covariance matrix between-class covariance matrix orthogonal basis from two matrices Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces Between-class matrix CB – between-class covariance matrix C – number of classes Mi – number of images in i-th class  – average image i – average image of i-th class Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces Within-class covariance matrix CW – within-class covariance matrix C – number of classes Xi – set of images which belong to i-th class xk – k-th image which belongs to i-th class i – average image of i-th class Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces PCA:  - eigenvectors matrix (vectors in columns) LDA: Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces LDA – hard to find inverse matrix Fisherfaces – improved approach: PCA for dimensionality reduction LDA for finding optimal orthogonal basis Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Fisherfaces Feature extraction in the Fisherfaces: Feature vector calculated by PCA normalised image as an input dimensionality reduction Feature vector calculated by LDA PCA feature vector as an input rotation of feature vector no dimensionality reduction Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Bayesian Matching Vectors similarity based on probability of their difference classification I – set of intra-personal pairs E – set of extra-personal pairs Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Bayesian Matching P(|) – probability of observing a given difference in a defined set of differences function of PCA back projection error – () Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Bayesian Matching Two classes of image pairs intra- and extra-personal Differences generated from pairs two classes of pairs PCA used for both classes separately two image spaces Face Recognition and Biometric Systems

Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Bayesian Matching Image difference recognition Dual Eigenfaces Difference distance from two image spaces Bayesian Matching – a slow method image difference calculated for every comparison possibility of applying other method for selecting candidates (n most similar images) Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Local PCA Based on detected features eyes, nose, mouth PCA for features small part of face image analysis of small images (eigeneyes, eigennoses, etc.) Less dimensional spaces Lower effectiveness, but supports the Eigenfaces Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Local PCA K1 K2 K3 K4 Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Other methods Local Feature Analysis 2D PCA, 2D LDA Independent Component Analysis Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Face topology eyes & nose – extra-personal differences mouth & cheeks – intra-personal differences Nature of features concerned with location Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Face relevance map enhance influence of extra-personal features decrease influence of intra-personal features Feature extraction with a map (m) Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map „T” map artificial map for eyes and nose binary values Results: FeretA: 423 -> 445 (3,7%) Conclusion: good approach, need for better map generation methods Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Difference map – statistical analysis Pairs of images: intra-personal extra-personal Average differences between images: average intra-personal difference average extra-personal difference Map obtained by subtracting intra-personal difference from extra-personal one Results for FeretA: 423 -> 462 (6,4%) Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Colour data information lost during conversion to GS low distinctiveness can be used for map generation Colour used for detection eye and mouth map masks based on detection maps Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Desired effect: higher values around eyes and nose lower values in the area of mouth Maps deliver information about features location Two possible approaches: image -> feature maps -> face relevance map image -> feature maps -> features -> f.r.m. Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Maps from points Nose location derived from eye & mouth weighted mean eye(R): (15, 24) eye(L): (49, 24) mouth: (32, 58) Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Single point influence r – radius, mmax – maximal map value Map – summed influence of the points eye, nose – positive weights mouth – negative weights Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Maps from colour improvement comparable to difference maps colour data carry information concerning nature of face areas generated for every image Map may be imposed during normalisation Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Maps from colour - examples Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Maps from colour - examples Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Back-projection based dynamic map dynamic – created for every image Back projection: map of local projection error higher error = lower importance map should be smoothed Good for occluded images Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Face relevance map Back-projection based dynamic map examples of occluded face images Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Maps take into account difference nature basing on face topology Differences not concerned with location lighting Eigenfaces – appearance interpretation various types of information some responsible for lighting Weights assigned to eigenvectors: Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Eigenvectors weights lower values for intra-personal directions of variance How to obtain the weights? visual assessment – may be incorrect the same procedure as in the case of difference masks Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error A better method for obtaining maps and eigenvector weights: error function minimisation Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Definition of recognition problem: M vectors, C classes and C base vectors (ui1) Mi vectors in i-th class (uij) classification of non-base vectors (j > 1) Single comparison similarity to home class and foreign class classes represented by base vectors Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Single comparison error: uij – a vector which is being recognised ui1 – home class base vector uk1 – foreign class base vector S – similarity between vectors Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Single comparison: correct if incorrect if Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Recognition error Error for comparison with all classes: Error for the whole set: Face Recognition and Biometric Systems

Eigenfaces: feature extraction K1 K2 K3 ... ... Scalar products between normalised image and eigenvectors Feature vector Face Recognition and Biometric Systems

Eigenfaces: feature extraction Feature vector element ( ): - dimensionality of feature vector - normalised face image - i-th eigenvector Improvements to the Eigenfaces face relevance masks eigenvector weights Face Recognition and Biometric Systems

Eigenfaces: feature extraction Improved feature extraction: - i-th eigenvector weight - j-th element (pixel) of the mask - j-th element of the i-th eigenvector - i-th element of the feature vector Face Recognition and Biometric Systems

Eigenfaces: feature extraction Similarity based on Euclidean distance: Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Error minimisation Recognition error is a function of mask and eigenvector weights The function may be minimised optimal mask optimal eigenvector weights Example of mask optimisation... Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Error minimisation Optimised dataset problem of overfitting How to avoid overfitting? large datasets optimisation can be stopped Advantages of overfitting overfitting to a group of people Face Recognition and Biometric Systems

Face Recognition and Biometric Systems Summary There are many methods derived from the Eigenfaces Error is a function of masks and eigenvectors weights Classification parameters can be optimised Improvement aims at: including face topology feature analysis difference classification Face Recognition and Biometric Systems

Thank you for your attention! Next time: Face detection Face Recognition and Biometric Systems