1. Fırat Onur Alsaran, Neşe Alyüz, Melike Esma İlter, Mürsel Taşgın, Ahmet Burak Yoldemir CmpE 58Z Term Project

2. Ahmet Burak Yoldemir

3. Motivation  Ear biometrics has several advantages over complete face  Facial biometrics may fail due to:  Expressions  Cosmetics  Hair styles  Growth of facial hair  Ears are affected very little from such changes

4. Ear database  448 ear images are manually cropped from profile images of CMU Multi-PIE Database  Only left ears are used  There are 4 ear images of 112 people  Illumination conditions of these 4 images are all different

5. Samples from the database  Person 1:  Person 2:  High illumination variance!

6. First attempts  Filter bank approaches are applied first Angular radial transform Gabor filters Leung-Malik filters Schmid filters

7. First attempts  Filter bank approaches are applied first Angular radial transform Gabor filters Leung-Malik filters Schmid filters

8. First attempts  Filter bank approaches are applied first Angular radial transform Gabor filters Leung-Malik filters Schmid filters

9. First attempts  Filter bank approaches are applied first Angular radial transform Gabor filters Leung-Malik filters Schmid filters

10. Illumination tolerance  None of the filter bank approaches is able to tolerate illumination changes, as they have fixed bases  A grayscale invariant texture measure: Local Binary Patterns

11. Local binary patterns - Advantages  Tolerance against illumination changes  Computational simplicity  A compact description of the image

12. Local binary patterns - Example

13. Local binary patterns  After obtaining LBP codes, a histogram of these codes is obtained using 256 bins  This histogram is actually a histogram of micro-patterns  The result is a 256 dimensional feature vector of an ear image

14. Local binary patterns  LBP method is very sensitive to high frequency components  A slight noise can change the ordering of the pixel values in a neighborhood, which results in a different micro-pattern  To prevent this, images are filtered with a Gaussian kernel of 5x5 before finding micro- patterns

15. Recognition step  Euclidean distance between these feature vectors is used as the (dis)similarity measure  A similarity matrix is formed using these distances

16. Multi-presentation approach  To increase recognition performance, multi- presentation approach is adopted  Each ear is represented using 2 images, verification is accomplished by taking 2 ear images of the user  Mean and max rules are applied to fuse the scores

17. Results – Without Gaussian filtering MethodEER (%) Original32.19 MP (max)14.73 MP(mean)1.77

18. Results – With Gaussian filtering MethodEER (%) Original13.18 MP (max)5.43 MP(mean)1.14

19. Mürsel Taşgın

20. Facial Profile recognition Motivation  Facial profile images can be collected from side cameras  Computation complexity is lower  Complementary solution for face recognition

21. Profile Database  448 profile photos from Multi-PIE database  112 subjects, each having 4 photos  Facial profiles are extracted manually in the first place

22. Facial Profile Registration 12 Rotate 90º CW 3 Extract profileEdge detection 4 5 6 Scale and move to top (nose at the center) Chin & nose detection using gradient of image Nose at the center and touching top Histogram representation (image to function) gradient

23. Facial Profile Registration (cont.)  Edge detection(Sobel) is used to convert black-white profile image to a histogram function  Profile line is decreased to a single pixel white line  Nose is the highest point in the histogram  Chin point is detected using gradient of histogram and image-filling function of Matlab:  If gradient of the image changes sharply at chin area, it is marked as chin point  If image-fill function fills in the chin area then the end point is marked as chin lips Image-filling detects lips, so use gradient to find chin

24. Facial Profile Matching (Histogram Matching)  Facial profiles are represented as histogram functions.  After registration, pointwise distance is measured:  Difference between points are summed over all points  Other metrics are available as well: Bhattacharyya distance White line is profile-1 Red line is profile-2 Green vertical lines are distances

25. Neşe Alyüz

26. Motivation  Multiple biometric sources can provide better performance  Ear and Facial Profile biometrics can be acquired simultaneously  Instead of using a single modality of ear or profile, apply fusion  Most common fusion level: score level  Heterogeneous Scores –> score normalization is important

27. Score Normalization Techniques  Min-max normalization  Z-Score normalization  Median Absolute Deviation (MAD) normalization  Tanh normalization

28. Min-max Normalization  Best suited for the case where bounds are known  Shift scores into range [0 1]  Given a set of matching scores: {s k }  Normalized scores:  Original distribution is kept, only scaling When bounds are estimated, not robust to outliers

29. Z-score Normalization  Performs well if prior knowledge is available  Mean and standard deviation are used  Given a set of matching scores: {s k }  Normalized scores: Original distribution is not retained Does not guarantee a common numerical range When mean and std are estimated, very sensitive to outliers

30. Median Absolute Deviation (MAD) Normalization  Median and MAD are insensitive to outliers and to points in the extreme tails of the distribution  MAD normalization benefits from this fact  Normalized scores: where MAD = median(|s k - median|) Median and MAD have low efficiencies When score distribution is not Gaussian, poor estimates Input distribution is not retained Normalized scores are not in a common range

31. Tanh Normalization  Robust to outliers  Highly efficient  Normalized scores:  Tanh distribution: normalized genuine scores has a mean of 0.05 and std of ~o.o1. Determines the spread of genuine scores

32. Score Fusion Techniques  MAX rule  MEAN rule  SUM rule  PRODUCT rule Evaluated on scores that are normalized with different approaches

33. Experimental Results  Initial Results on Similarity matrices of Assignment #3: Face and Fingerprint biometrics  40 subjects with 8 sample/subject  SMs: 320x320 similarity matrices  Enrollment: 1 sample/subject for each bimetric

34. Experimental Results - EERs FusionMAXMEANSUMPRODUCT No Norm.8.588.27 14.01 Min-max14.878.64 8.32 Z-score8.157.89 20.42 MAD7.887.86 18.58 Tanh7.847.61 7.57 Individual ModalitiesEERs Face12.09 Fingerprint21.76

35. Experimental Results - TODO FusionMAXMEANSUMPRODUCT No Norm. Min-max Z-score MAD Tanh Individual ModalitiesEERs Face Profile #1 Face Profile #2 Ear