1. Automatic Face Feature Localization for Face Recognition Christopher I. Fallin honors thesis defense: May 1, 2009 advisor: Dr. Patrick J. Flynn May 1, 20091Chris Fallin - thesis defense

2. Outline Face recognition – Methods of evaluation Elastic Bunch Graph Matching – Gabor Jets – Bunch Graphs and Feature Localization My Contributions: Automatic Fiducial Points – Information Content model – Fiducial point placement – results

3. Face Recognition May 1, 20093Chris Fallin - thesis defense Subfield of biometrics: – life (bio) – measure (metric) – Extract identifying information from measures of human traits Face recognition: digital images of face 2D, 3D, infrared, multimodal, … http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaces.jpg

4. May 1, 2009Chris Fallin - thesis defense4 Image Set Face Recognition Evaluation

5. May 1, 2009Chris Fallin - thesis defense5 http://en.wikipedia.org/wiki/File:Roc-general.pnghttp://en.wikipedia.org/wiki/File:Roc-general.png – used under terms of GNU FDL System Decision ROC curves YNYN Actual Y N

6. May 1, 2009Chris Fallin - thesis defense6 EER = 11.1%

7. Rank-one Score May 1, 2009Chris Fallin - thesis defense7 Gallery ABCDEFG A0.890.700.100.520.340.480.37 B0.700.730.450.820.120.430.44 ProbeC0.100.450.920.890.230.820.13 D0.520.820.890.560.200.380.14 E0.340.120.230.200.820.520.23 F0.480.430.820.380.520.840.11 G0.370.440.130.140.230.110.99 Rank-one:5/7 = 71.4%

8. Elastic Bunch Graph Matching (EBGM) Wiskott et al., USC/Bochum, mid-90s Basis of ZN-Face, successful commercial system We use Face Identity Evaluation System, from Colorado State Face features represented by Gabor filter responses Features are localized – Fit an elastic graph onto the features by localization: local optimization problems May 1, 20098Chris Fallin - thesis defense

9. A Face Graph May 1, 2009Chris Fallin - thesis defense9 [Wiskott99]

10. Gabor Jets Vector of filter responses to 40 Gabor kernels – 5 wavelengths – 8 orientations – Each is complex-valued Gabor jets capture information well: Gokberk et al. get 91% rank-one with fixed grid – On FERET: 78.5% max, with 12 grid points May 1, 2009Chris Fallin - thesis defense10

11. Bunch Graphs Each feature has a bunch of canonical jets Represents typical features Best-match at each feature point for novel images May 1, 2009 Chris Fallin - thesis defense 11 [Wiskott99]

12. Feature Localization Initial alignment: eye locations known a-priori Overlay bunch graph with average edge lengths Take Gabor jets; pick best match in each bunch Localize based on displacement estimation (local optimization problem) May 1, 200912Chris Fallin - thesis defense

13. The Idea: Automatic Fiducial Point Placement Bunch graph training requires manual fiducial point placement – 70 images, 25 points Why not statistically determine optimal features to match on? We can align/normalize all faces and take some statistical measure at each point in face space to determine goodness Replaces training step; back-end algorithm is identical May 1, 200913Chris Fallin - thesis defense

14. Related Work Gokberk et al.: choosing fiducial points with genetic algorithms – But their chosen points are global – Same goal as our system, excluding prelocalization Salient Points – Wavelet-based approach to image retrieval – Choras et al., 2006: similar approach with goodness function, but no EBGM May 1, 200914Chris Fallin - thesis defense

15. Information Content: Variance Model Compute goodness function over face- space Inter-subject variance over intra-subject variance Self-normalizing unitless measure Requires multiple images per subject May 1, 2009Chris Fallin - thesis defense15

16. Computing the goodness function FRGC: 5404 images – 700 MB, 128x128 grayscale (7 GB before normalization) Each pixel: 12 seconds, on fast Athlon 64 Split into 128 Condor jobs – Each pixel is independent: easy Pre-normalize image set, dump to fast-loading binary format (single file) Run Condor jobs: three hours Post-processing to reassemble results May 1, 2009Chris Fallin - thesis defense16

17. Fiducial Point Placement Random placement with probability density Compute gradient of goodness function Probability is product of gradient and goodness Place points sequentially, decay probability around points exponentially Mirror-point constraint: mirror placements across centerline, or snap to center May 1, 2009Chris Fallin - thesis defense17

18. Prelocalization: Pseudo-Bunches Displacement estimation requires canonical feature jet from bunch We cant provide this if we have no knowledge of feature Solution: fake a jet bunch – Make educated guess with K-means clustering on jets from all images at given point Then, run displacement estimation to prelocalize points on each image May 1, 200918Chris Fallin - thesis defense

19. Results Competitive with original, manual points – In both cases, automatic training points yield only ~1% performance drop – With no human training! Prelocalization did not work as intended Success without this suggested by Gokberks results May 1, 2009Chris Fallin - thesis defense19 FERET FRGC

20. ROC curves FERETFRGC May 1, 2009Chris Fallin - thesis defense20 (EER = 11.4%) (orig: 11.1%) (EER = 31.4%) (orig: 34.8%)

21. Prelocalization: causes for failure Poor pseudo-bunch clustering: K-means often found optimal clustering at self-imposed cap of N/10 clusters – Likely because initial jets are too far off Naïve localization: single-step – Bolme thesis compares several optimization algorithms Average displacement of 2.628 pixels: larger than 2.021 pixels in manual points May 1, 200921Chris Fallin - thesis defense

22. Future Work More sophisticated prelocalization Look at pseudo-bunch statistics to determine failure mode in more detail Look at per-fiducial point statistics to determine where performance is weak Investigate: are manual pts a theoretical limit, or can we exceed them? Try new image classes – test claim of genericism May 1, 200922Chris Fallin - thesis defense

23. Questions? Email cfallin@c1f.netcfallin@c1f.net Full thesis and source code will be posted online: http://c1f.net/research/mark5/http://c1f.net/research/mark5/ May 1, 200923Chris Fallin - thesis defense

24. Distance Metrics on Jets Phase-insensitive: magnitude only – Selects best jet in bunch Phase-sensitive – Can solve for displacement vector: basis of localization Displacement estimation May 1, 2009Chris Fallin - thesis defense24