1. Principal Component Analysis of Face Properties Samarasena Buchala 1, Neil Davey 1, Tim Gale 1,2, Ray Frank 1 1 School of Computer Science, University of Hertfordshire, College Lane, Hatfield, AL10 9AB, UK 2 Department of Psychiatry, QEII Hospital, Welwyn Garden City, AL7 4HQ, UK

2. Overview Principal Component Analysis – Face Recognition Principal Component Analysis Analysis of components of PCA –Linear Discriminant Analysis of PCA components –Does PCA efficiently encodes information in face images –Analysis of gender, ethnicity, age, and identity –Do components encode information related to multiple properties

3. Face Properties Human face is considered to be special in terms of biological and social roles Has multiple properties from which they can be categorised at different levels of specificity – gender, ethnicity, age, identity, expression, degree of attractiveness, typicality, attractiveness, so on. Widely researched in the fields of Computer Science and Psychology

4. Principal Component Analysis (PCA) on Face Images Dimensionality reduction – Sirovich and Kirby (1987) Face recognition – Turk and Pentland (1991) Benchmark for face recognition algorithms - (Moon & Phillips, 2001). Distinctiveness effects of faces - (Hancock, 1996) other-race effect - (O'Toole et al., 1991b) Dimensional-based model of facial expression - (Calder et al., 2001)

5. Research Questions Does PCA encode information related to gender, ethnicity, age, and identity efficiently? What information do PCA encode? Are there components (features) of PCA that encode multiples properties?

6. PCA The aim of the PCA is a linear reduction of D dimensional data to d dimensional data (d<D), while preserving as much information, in the data, as possible. Linear functions y 1 = w 1 X y 2 = w 2 X * y d = w d X Y= W X X – inputs; Y – outputs, components; W – eigenvectors, eigenfaces, basis vectors

7. How many components? Usual choice consider the first d PCs which account for some percentage, usually above 90 %, of the cumulative variance of the data. This is disadvantageous if the last components are interesting W2W2 W1W1 x1x1 x2x2

8. Dataset A subset of FERET dataset 2670 grey scale frontal face images Rich in variety: face images vary in pose, background lighting, presence or absence of glasses, slight change in expression PropertyNo. Categorie s No. Face s Gender2 Male1603 Female1067 Ethnicity3 Caucasian1758 African320 East Asian363 Age5 20 – 29665 30 – 391264 40 – 49429 50 – 59206 60+106 Identity358 Individuals with 3 or more examples 1161

9. Dataset Each image is pre-processed to a 65 X 75 resolution. Aligned based on eye locations Cropped such that little or no hair information is available Histogram equalisation is applied to reduce lighting effects

10. Does PCA efficiently represents information in face images? Images of 65 × 75 resolution leads to a dimensionality of 4875. The first 350 components accounted for 90% variance of the data. Each face is thus represented using 350 components instead of 4875 dimensions Classification employing 5-fold cross validation, with 80% of faces in each category for training and 20% of faces in each category for testing for identity recognition leave-one-out method is used. LDA is performed on the PCA data Euclidean measure is used for classification Property Classification Gender86.4% Ethnicity81.6% Age91.5% Identity90%

11. What information does PCA encode? – Gender Gender encoding power estimated using the LDA 3 rd component carries highest gender encoding power followed by the 4 th components All important components are among the first 50 components

12. What information does PCA encode? – Gender Reconstructed images from the altered components (a) third and (b) fourth components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D. Third component encodes information related to the complexion, length of the nose, presence or absence of hair on the forehead, and texture around the mouth region. Fourth component encodes information related to the eyebrow thickness, presence or absence of smiling expression -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

13. Gender (a) Face examples with the first two being female and the next two being male faces. (b) Reconstructed faces of (a) using the top 20 gender important components. (c) Reconstructed faces of (a) using all components, except the top 20 gender important components.

14. What information does PCA encode? – Ethnicity 6 th component carries highest ethnicity encoding power followed by the 15 th components All ethnicity important components are among the first 50 components

15. Ethnicity Reconstructed images from the altered components (a) 6 th and (b) 4 th components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D. 6 th component encodes information related to complexion, broadness and length of the nose 15 th component encodes information related to length of the nose, complexion, and presence or absence of smiling expression -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

16. What information does PCA encode? – Age Age – 20-39 and 50-60+ age groups termed as young and old) 10 th component is found to be the most important for age Reconstructed images from the altered tenth component. The component is progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D -6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

17. What information does PCA encode? – Identity Many components are found to be important for identity. However, their importance magnitude is small. These components are widely distributed and not restricted to the first 50 components

18. Can a single component encode multiple properties? A grey beard informs that the person is a male and also, most probably, old. As all important components of gender, ethnicity, and age are among the first 50 components there are overlapping components. One example is the 3 rd component which is found to be the most important for gender and second most important for age

19. Can a single component encode multiple properties? Normal distribution plots of the (a) third (b) and fourth components for male and female classes of young and old age groups.

20. Summary PCA encodes face image properties such as gender, ethnicity, age, and identity efficiently. Very few components are required to encode properties such as gender, ethnicity and age and these components are amongst the first few components which capture large part of the variance of the data. Large number of components are required to encode identity and these components are widely distributed. There may be components which encode multiple properties.