1 Numerical geometry of non-rigid shapes Expression-invariant face recognition Expression-invariant face recognition Lecture 8 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009

2 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in the age of Patriarchs “The voice is the voice of Jacob, but the hands are the hands of Esau”

3 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in fairy tales

4 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in opera

5 Numerical geometry of non-rigid shapes Expression-invariant face recognition First face recognition Galton’s mechanical face recognition machine I contrieved an apparatus […] which I call a mechanical selector. Its object is to find which set out of a standard collection of many sets of measures, resembles any one of given set within specified degree of unlikeness. “ ” Nature, 1888 Sir Francis Galton ( )

6 Numerical geometry of non-rigid shapes Expression-invariant face recognition Face recognition today = ? Is this the same person?

7 Numerical geometry of non-rigid shapes Expression-invariant face recognition Some terminology Gallery Instances of faces of a person with known identity Probe A face unseen before with unknown identity Impostor False rejection Probe deemed dissimilar to gallery of same identity False acceptance Probe of different identity deemed similar to gallery

8 Numerical geometry of non-rigid shapes Expression-invariant face recognition The coin that betrayed Louis XVI Gallery Probe

9 Numerical geometry of non-rigid shapes Expression-invariant face recognition How face recognition works Gallery Probe FACE DISTANCE > threshold? REJECTACCEPT NoYes

10 Numerical geometry of non-rigid shapes Expression-invariant face recognition Recognition accuracy False acceptance rate (FRR) False rejection rate (FAR) High threshold (loose) Low threshold (conservative) Equal Error Eate (EER) FAR = FRR

11 Numerical geometry of non-rigid shapes Expression-invariant face recognition Face distance Discriminative Large between faces of different persons Invariant Small between faces of same person in different conditions

12 Numerical geometry of non-rigid shapes Expression-invariant face recognition + GEOMETRIC (3D) PHOTOMETRIC (2D) What is a face? =

13 Numerical geometry of non-rigid shapes Expression-invariant face recognition What is more important: 2D or 3D? + =

14 Numerical geometry of non-rigid shapes Expression-invariant face recognition What is more important: 2D or 3D? + =

15 Numerical geometry of non-rigid shapes Expression-invariant face recognition Acco Turkish bath museum, Israel Live statue

16 Numerical geometry of non-rigid shapes Expression-invariant face recognition Conclusion 1 3D data encodes valuable information about person’s identity Less sensitive to external factors (light, pose, makeup) More difficult to forge

17 Numerical geometry of non-rigid shapes Expression-invariant face recognition The curse of expressions

18 Numerical geometry of non-rigid shapes Expression-invariant face recognition Is geometry sensitive to expressions? x x’ y y’ Euclidean distances

19 Numerical geometry of non-rigid shapes Expression-invariant face recognition Is geometry sensitive to expressions? x x’ y y’ Geodesic distances

20 Numerical geometry of non-rigid shapes Expression-invariant face recognition Conclusion 2 Distance distortion distribution Extrinsic geometry sensitive to expressions Intrinsic geometry insensitive to expressions

21 Numerical geometry of non-rigid shapes Expression-invariant face recognition Isometric model of expressions Facial expressions are approximate isometries of the facial surface Identity = intrinsic geometry Expression = extrinsic geometry A. M. Bronstein et al., IJCV, 2005

22 Numerical geometry of non-rigid shapes Expression-invariant face recognition

23 Numerical geometry of non-rigid shapes Expression-invariant face recognition How to canonize a person? 3D scan Smooting Canonization Cropping

24 Numerical geometry of non-rigid shapes Expression-invariant face recognition Canonical forms of faces

25 Numerical geometry of non-rigid shapes Expression-invariant face recognition SCANNED FACE CANONICAL FORM DISTANCES

26 Numerical geometry of non-rigid shapes Expression-invariant face recognition Telling identical twins apart Extrinsic similarity Intrinsic similarity MichaelAlex

27 Numerical geometry of non-rigid shapes Expression-invariant face recognition Telling identical twins apart MichaelAlex

28 Numerical geometry of non-rigid shapes Expression-invariant face recognition

29 Numerical geometry of non-rigid shapes Expression-invariant face recognition

30 Numerical geometry of non-rigid shapes Expression-invariant face recognition Comparing photometric properties Facial surface with texture 3D canonical form 2D canonical form Two-dimensional canonical form can be used as a common parametrization of the facial textures

31 Numerical geometry of non-rigid shapes Expression-invariant face recognition Spherical embedding R = 80 mmR = 100 mmR = 150 mm

32 Numerical geometry of non-rigid shapes Expression-invariant face recognition Partial similarity