1. Joint and implicit registration for face recognition Dr. Peng Li and Dr. Simon J.D. Prince Department of Computer Science University College London {p.li,s.prince}@cs.ucl.ac.uk 14:00-15:00 Tuesday, 23 June 2009

2. The face recognition pipeline Matching Probe Gallery Keypoint registration Result Detected face Global approaches Eigenfaces [Turk 1991] Fisherfaces [Belhumeur 1997] Local approaches AAM [Cootes 2001] ASM [Mahoor 2006] EBGM [Wiskott 1997] Distance-based approaches Fisherfaces [Belhumeur1997] Laplacianfaces [He2005] KLDA [Yang2005] Probabilistic approaches Bayesian [Moghaddam 2000] PLDA [Ioffe 2006, Prince 2007] Feature extraction Face recognition Face detection Original Image

3. The face recognition pipeline Extract Gabor jet around each keypoint Generative probabilistic model Independent term for each keypoint …… Matching Probe Gallery Keypoint registration Original Image Result Detected face Feature extraction Face recognition Face detection

4. Hypothesis 1 H1: We can use the same probabilistic model for registration and recognition. Probabilistic model Result Keypoint registration Feature extraction Face recognition Face detection Keypoint registration Feature extraction …… Matching Probe Gallery Detected face Original Image

5. Hypothesis 2: Joint Registration Gallery Probe + + + Generic eyeParticular eye + x H2: We can use the gallery image to help find keypoints in the probe image.

6. Hypothesis 3: Implicit Registration Probe Posterior distribution + * Hidden variable H3: We do not need to make hard estimates of keypoint positions. t p – keypoint position

7. Outline Background Hypotheses Probabilistic face recognition Frontal face recognition H1: Same model for registration and recognition H2: Joint registration H3: Implicit registration Cross-pose face recognition Conclusion

8. Probabilistic linear discriminant analysis (Prince & Elder,ICCV 2007) mean m Signal Noise + + + x ij = μ + ++ Fh i Gw ij  ij = G(:,1) G(:,2) G(:,3) w 1j w 2j w 3j Within-individual variation Between-individual variation F(:,1) F(:,2) h1h1 h2h2 h3h3 F(:,3) i - # of identity j - # of image Image x ij Independent per-pixel Gaussian noise, 

9. Face recognition by model selection xpxp xgxg hghg hphp MdMd wgwg wpwp –Match xpxp xgxg hghg wgwg MsMs wpwp –No-Match Observed Variables Choose MAP model Pr(x p, x g |M d ) Pr(x p, x g |M s ) Observed Variables Hidden Variables X p - Probe image X g - Gallery image Hidden Variables

10. Methodology Gallery Probe + + tptp 1: Find keypoint in probe image alone by MAP2: Joint registration by MAP3: Implicit registration using probe image alone4: Joint and Implicit registration Posterior over keypoint position t p – keypoint position xpxp xgxg hghg hphp wgwg wpwp xpxp xgxg hghg wgwg wpwp

11. Experimental Setting: XM2VTS Database Dataset –Training: First 195 identities –Test: Last 100 identities Gallery data: 1st image of 1st session Probe data: 1st image of 4th session Feature Extraction: Gabor filter at all possible locations of 13 keypoints

12. Experiment 1: finding keypoints using recognition model in probe alone Recognition First match identification rate Higher is better Registration Average error of all keypoints Lower is better

13. Gallery image helps find keypoints in probe image Localization errors are close to human labelling Experiment 2: joint registration

14. Experiment 3: implicit registration Marginalizing over keypoint position is better than using MAP keypoint position

15. Experiment 4: joint and implicit registration Joint and implicit registration performs best. Comparable to using manually labeled keypoints.

16. Cross-pose face recognition using tied PLDA model (Prince & Elder, 2007) Key idea: separate within-individual and between- individual variance at each pose Data: XM2VTS database: with 90° pose difference. Gallery (frontal face) ↔ Probe (profile face) Feature extraction: Gabor feature for 6 keypoints FRONTAL IMAGE PROFILE IMAGE x ijk = μkμk + ++ FkhiFkhi G k w ijk  ijk K = 1 K = 2 K – Pose Index

17. Experiment 5: Cross-pose face recognition and registration Similar results to frontal face recognition & registration Comparable to using manually labeled keypoints.

18. Concluding Remarks Three hypotheses –Same model for both face registration & recognition. –Joint registration for face recognition –Implicit registration for face recognition All work well for both frontal & cross-pose face registration & recognition