1. Image-Based Biometric Person Authentication
Professor Heikki Kälviäinen
Machine Vision and Pattern Recognition Laboratory (MVPR)
Department of Information Technology
Faculty of Technology Management
Lappeenranta University of Technology (LUT)
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

2. Content Machine vision and pattern recognition in LUT.
Biometric person authentication.
Face detection.
Why is detection/localization difficult.
Existing approaches.
Proposed algorithm.
Results and evaluation.
New solutions.
Conclusions.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

3. Machine Vision and Pattern Recognition Laboratory (MVPR)
Leader: Prof. Heikki Kälviäinen.
2nd largest computer vision research group in Finland.
Center of Excellence in Research in LUT.
24 members:
3 Professors + 3 Post docs + 2 Visiting doctors + 11 PhD students
+ undergraduate students + industry coordinator.
Co-operation with 14 international universities and research institutes.
Results: 18 Ph.D. degrees (and 3 externally produced), over 400 scientific publications, 40 research projects, and spin-off companies.
Objectives: 2 PhDs/year.
Annual external project funding EUR, basic funding EUR, total 1.0 million EUR.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

4. MVPR Laboratory: Research Profile
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

5. Machine Vision System
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

6. Biometric Person Authentication
Hand geometry
Face recognition
Fingerprints
Iris
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

7. Infrared (face, body parts) Odor Behavioral: Speech Handwriting
Biometric: any measurement of a person’s physiological traits or behavior
Physiological:
Face
Fingerprint
Iris
Retinal scan
Ear shape
Hand geometry
Infrared (face, body parts)
Odor
Behavioral:
Speech
Handwriting
Signature
Lip movements
Keystroke dynamics
Gait
Genetic:
Tissue sample
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

8. FACEDETECT Image-Based Biometric Person Authentication
Docent, Dr. Joni Kamäräinen, Docent, Dr.Ville Kyrki,
Mr. Pekka Paalanen, Mr. Jarmo Ilonen,
Prof. Heikki Kälviäinen
Machine Vision and Pattern Recognition Research Group
Lappeenranta University of Technology
FINLAND
Dr. Miroslav Hamouz, Prof. Josef Kittler,
Prof. Jiri Matas
Centre for Vision, Speech, and Signal Processing (CVSSP)
University of Surrey
UNITED KINGDOM
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

9. Why is Face Detection Difficult?
Object-class recognition (an object to be recognized is not a single entity rather a a group of similar objects).
Faces exhibit significant variability in shape, colour, and texture, and may appear in arbitrary poses:
Appearance variations over the whole population.
Capture effects.
Background.
Illumination.
Video versus still image.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

10. State of the Art Image-based methods: Scanning window.
Face modeled as manifolds in some high dimensional space.
Moghaddam, Pentland – probabilistic PCA
Sung and Poggio, Rowley et al.- neural networks
Osuna et al. – SVM
Viola and Jones – Adaboost on
Haar features
Jesorsky et al – Haussdorf
distance on edge images
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

11. State of the Art (cont.) Feature-based methods:
Face modelled as a viable configuration of local features.
Needs higher resolution than image-based methods.
False alarms. Vogelhuber, Schmid, Gaussian derivatives + angles and length ratios Weber er al., interest operator + statistical model on positions Cristinacce and Cootes, Adaboost + shape model
Warping methods: Variability decomposed into a shape model and the model of local appearance or texture which is iteratively deformed to fit. Cootes et al., Active Shape and Appearance models Lades et al., Wiskott et al. Dynamic link architectures
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

12. Introduction
Face verification (authentication) Validating a claimed identity based on the image of a face: are you Mr./Ms. X?
Face recognition (identification) Identifying a person based on an image of his/her face: who are you?
Face detection/localization Location of human faces in images at different positions, scales, orientations, and lighting conditions.
The topic of my research belongs to the field of computer face recognition. This field has attracted a lot of attention over the past years, you may for example have heard about the government’s proposals on the biometric Ids.
The work presented today was a part of EU funded project BANCA which dealt with the use of face and voice for person authentication.
Let me explain some terminology first.
Face verification deals with situations where a person claims an identity and the system
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

13. Avoiding a scanning window. Using feature detectors.
Proposed Algorithm
Avoiding a scanning window.
Using feature detectors.
Shape-free texture model for the final decision.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

14. Feature Detector: 2-D Gabor Filter
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

15. Maximal joint localization in the spatial and frequency domain.
Gabor Features
Maximal joint localization in the spatial and frequency domain.
Smooth and noise tolerant.
Parameters for invariance manipulation:
Frequency
Envelope sharpness
Orientation
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

16. Constructing Response Matrix
Filter response r(x,y; f,) can be calculated for various
frequencies f and orientations  to construct a response matrix.
image scaling appears as a
shift of the rows (high
frequencies may vanish)
image rotation appears as a
circular shift of the columns
columns represent orientations
rows represent frequencies
A SCALE AND ROTATION INVARIANT
TREATMENT OF THE RESPONSE MATRIX
CAN BE ESTABLISHED, AND THUS, WE
CAN CONCENTRATE ONLY HOW TO
CLASSIFY THEM IN THE STANDARD POSE
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

17. 2-D Gabor Features What do they ”see”?
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

18. Evidence Extraction Requirements Preferred Scale invariant extraction.
Rotation invariant extraction.
Provides sufficiently small amount of correct candidate points.
(n best points from each class; needs confidence measure).
Preferred
Estimation of evidence scale and orientation.
Fast extraction (scalability).
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

19. Classifier Construction
Stability property guarantees approximately the Gaussian form of classes in the feature space.
One class may still consist of several sub-clusters (open eye, closed eye, etc.).
nostril
eye
Gaussian mixture
model densities
(EM estimation)
eye
eye
Bayesian
classification
of features
nostril
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

20. Affine Learned Correspondences
Aligned images of objects and
manually selected features
Variability and correspondences 1 2 3 4 5 6
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

21. Affine Hypothesis Search
False alarms occur and hypothesis
verification is needed
Evidence
extraction. 2 3 1 4 5
Instance
approved
2. Affine search and
match to correspon-
dence model.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

22. Face Space
Normalization of space where shape variations and capture effects are removed from patterns.
Based on three points on the face -> affine registration.
Optimal with regard to the photometric variance over a big set of faces.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

23. Features & Feature Detectors
Features = salient parts of face.
Small localization variance and frequent occurrence over population.
Illumination, scale, rotation, and translation invariance.
Automatic analysis using the face space desirable.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

24. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

25. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

26. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

27. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

28. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

29. Confidence Regions Exhaustive search over triplets O(n)n3.
Not all triplets have to verified, regions supporting highly likely transformations can be learned.
Speed-up up to times.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

30. Performance Measure
Strict measure using the location of eye centres, not only an upright bounding box.
deye<=0.05 in order to succeed in verification.
deye<=0.25 corresponds to the definition of successful detection in the majority of state-of-the-art algorithms.
C = ground truth eye center coordinates
d = distances between the detected
and ground truth ones
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

31. deye = 0.05 Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

32. Recognition System Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

33. BANCA Database
Large realistic face and voice database collected (BANCA database):
4 languages, each language 6540 images of 52 people.
Three scenarios simulating controlled access, office environment and outdoor scenes.
Publicly available including a rigorous evaluation protocol.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

34. Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

35. Both eye centres detected (%)
XM2VTS Database
LABEL
Rate (%) 1
56.1 2
84.2 3
70.9 4
50.9 5
84.9 6
64.2 7
70.4 8
75.5 9
54.2 10
45.8
1 triplet detected (%)
88.3
Both eye centres detected (%)
74.5
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

36. Both eye centres detected (%)
BioID Database
LABEL
Rate (%) 1
55.6 2
67.6 3
51.2 4
39.1 5
61.1 6
54.8 7
29.5 8
34.5 9
40.0 10
48.7
1 triplet detected (%)
73.4
Both eye centres detected (%)
48.6
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

37. Both eye centres detected (%)
BANCA Database
LABEL
Rate (%) 1
41.4 2
60.3 3
44.5 4
44.0 5
67.4 6
34.3 7
54.8 8
63.6 9
49.6 10
61.8
1 triplet detected (%)
81.4
Both eye centres detected (%)
44.0
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

38. 3-dimensional Face Recognition
3-D images.
3-D algorithms.
Accurate!
Images?
Reference
databases?
Speed?
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

39. FACEDETECT - Publications
Hamouz, M., Kittler, J., Kamarainen, J.-K., Paalanen, P., Kälviäinen, H., Matas, J., Feature-Based Affine-Invariant Localization of Faces, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 9, 2005, pp (Impact factor: 3.810)
Kamarainen, Joni-Kristian, Ville Kyrki, and Heikki Kälviäinen. Invariance properties of Gabor filter based features - Overview and applications. IEEE Transactions on Image Processing, Vol. 15, No. 5, 2006, pp (Impact factor: 2.428)
Kyrki, Ville, Joni-Kristian Kamarainen, and Heikki Kälviäinen. Simple Gabor feature space for invariant object recognition. Pattern Recognition Letters, Vol. 25. No , pp (Impact factor: 1.138)
Paalanen, P., Kamarainen, J.-K., Ilonen, J., Kälviäinen, H., Feature Representation and Discrimination Based on Gaussian Mixture Model Probability Densities - Practices and Algorithms, Pattern Recognition, Vol. 39, No. 7, pp (Impact factor: 2.153)
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen

40. Conclusions and Future Work
Algorithm successfully tested on a large face authentication data set.
Combination of features brings a significant performance boost.
Gabor jets proved as a suitable local representation of a signal.
Adequate resolution necessary for feature detectors to succeed.
3-D face recognition much more accurate than 2-D recognition.
Methods for non-frontal poses (more 3-D face research needed).
Speed: real-time solutions (3-D image acquisition and analysis).
Applications:
Security applications: biometric passports, access, cash dispensers, etc.
Surveillance applications.
Machine Vision and Pattern Recognition Laboratory
Professor Heikki Kälviäinen