1. Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor
Ryan Connaughton
Kevin W. Bowyer
Patrick Flynn
April 16, 2011
Computer Vision Research Lab
Department of Computer Science & Engineering
University of Notre Dame

2. Biometrics and Multi-Biometrics
Trait
Biometric
Sample
Output
Sensor
Matcher
Multi-Modal
Multi-Sensor
Multi-Sample
Multi-Algorithm
Redundancy at any stage is referred to as multi-biometrics 2

3. Fusion in Multi-Biometrics
Fusion: Combining information from multiple sources
Types of fusion:
Signal Level
Feature Level
Score Level
Rank Level
Decision Level 3 3

4. Advantages and Disadvantages
Potential advantages of multi-biometrics:
Increased recognition accuracy
Wider population coverage & lower failure-to-acquire rates
More difficult to spoof
Potential disadvantages:
Increased computation time
Increased acquisition time
Increased sensor cost 4

5. Project Goal
Investigate the feasibility of multi-biometrics based on a single sensor
Specifically, combine multi-sample and multi-modal elements to create a system based on face and iris biometrics
Compare performance of multi-biometric approach to single biometric approach 5

6. Sensors – Iris on the Move (IOM)
Developed by Sarnoff Corp. [1]
Designed for Iris recognition
Stand-off and on-the-move
Array of 3 frontal video cameras
Each frame is 2048 x 2048 px
Average iris diameter is ~120 px
Synchronized NIR illumination
Image from K. W. Bowyer, K. Hollingsworth, and P. J. Flynn. Image understanding for iris biometrics: A survey. In Computer Vision and Image Understanding, volume 110, pages 6

7. IOM Frame Example 7

8. Sensors – LG IrisAccess 4000 (LG-4000)
Developed by LG Iris [2]
High-quality iris sensor
Short-range, stationary subjects
Average iris diameter is ~250 px
Image from LG Iris Products and Solutions, URL 8

9. LG-4000 Image Example 9

10. Diagram of Approach 10

11. Preprocessing
Stitch and perform histogram matching between corresponding frames
Use template matching to determine translation required to align frames 11

12. Face Detection Performed on stitched frames
OpenCV version Viola-Jones face detector used [3],[4]
Trained on whole faces
Faces are cropped according to face detector's estimation of size and location 12

13. Eye Detection
Used for iris biometrics and for alignment during face matching
Performed in two phases
Phase 1: Detect eyes in upper quadrants of previously detected faces
Phase 2: Detect eyes in frames where no faces were found
Both phases use template matching approach to search for specular highlights 13

14. Face and Iris Matcher Face Matcher
Colorado State University's implementation of eigenface [5],[6]
Mahalanobis Cosine: -1 to 1, -1 is perfect match
Iris Matcher
Modified version of Daugman's algorithm [7]
Normalized Hamming Distance: 0 to 1.0, 0 is perfect match 14

15. Fusion Summary Multi-modal and multi-sample scenario
Test and compare multiple fusion approaches
Score-level
Rank-level
Three approaches:
Min rule
Borda count
Sum rule 15

16. Min Fusion Multi-sample, uni-modal, score-level fusion
MinIris = Min{ Ii,j | i=1...n, j=1...G }
MinFace = Min { Fi,j | i=1...m, j=1...G }
Ii,j = HD between i-th probe iris and j-th gallery iris
Fi,j = Mahalanobis distance between i-th probe face and j-th gallery face
n,m = number of irises and faces detected
G = number of gallery subjects 16

17. Borda Fusion Multi-sample, multi-modal or uni-modal, rank-level fusion
For each probe biometric sample
Sort gallery subjects by match score (best to worst)
Cast votes for the top v-ranked gallery subjects
BordaLinear: VoteWeightn = v + 2 – n
BordaExp: VoteWeightn = 2v-n
Gallery subject with the most votes is the best match for that probe video
Three variations: BordaIris, BordaFace, and BordaBoth 17

18. Sum Fusion Multi-sample, multi-modal or uni-modal, score-level fusion
Ii,k = HD between i-th probe iris and k-th gallery iris
FNormi,k = Normalized Mahalanobis distance between i-th probe face and k-th gallery face
n,m = number of irises and faces detected
α,β = weights assigned to face and iris modalities 18

19. Dataset Collected 1,886 IOM video sets, spanning 363 subjects
Ranged from 1 to 15 probe videos per subject
Iris gallery consisted of one left eye and one right eye for each subject
Acquired with an LG-4000
Face gallery consisted of one full face image for each subject
Manually selected and annotated from stitched IOM frames
Earliest IOM video with full face available was used to generate gallery image
Videos used to generate gallery images were not included in probe set 19

20. Detection Results 20

21. Independent rank-one:
Face Matching Results
Mean match score:
(σ = 0.213)
Mean non-match score:
0.000 (σ = 0.676)
Independent rank-one:
51.6% (5073/9833) 21

22. Independent rank-one:
Iris Matching Results
Mean match score:
0.398 (σ = 0.053)
Mean non-match score:
0.449 (σ = 0.013)
Independent rank-one:
46.6% (13556/29112) 22

23. Rank-One Recognition Rates23

24. Comparison Summary 24

25. Conclusions
Investigated fusion of face and iris biometrics from a single sensor
Conducted multi-modal experiments on a genuine dataset of videos of 363 subjects
Combined multi-modal and multi-sample biometrics, as well as score-level and rank-level fusion
Implemented the proposed multi-biometric workflow on a stand-off and on-the-move sensor
Thus far, the best tested multi-modal approach yielded an increase of 5.4% in rank-one recognition over uni-modal approach 25

26. Acknowledgments & Questions
[1] J. Matey, O. Naroditsky, K. Hanna, R. Kolczynski, D. LoIacono, S. Mangru, M. Tinker, T. Zappia, and W. Zhao. Iris on the Move: Acquisition of Images for Iris Recognition in Less Constrained Environments. In Proceedings of the IEEE, volume 94, pages November 2006.
[2] LG Iris. LG Iris Products and Solutions, URL
[3] G. Bradski and A. Kaehler. Learning OpenCV. O'Reilly Media, Inc., 2008.
[4] P. Viola and M. Jones. Rapid Object Detection Using a Boosted Cascade of Simple Features. In 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), volume 1, pages , 2001.
[5] Colorado State University. Evaluation of Face and Recognition Algorithms, URL
[6] M. Turk and A. Pentland. Face Recognition Using Eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1991), volume 1, pages , June 1991.
[7] J. Daugman. How Iris Recognition Works. In 2002 International Conference on Image Processing, volume 1, pages , 2002.
Datasets used in this work were acquired under funding from the National Science Foundation under grant CNS , by the Central Intelligence Agency, and by the Technical Support Working Group under US Army Contract W91CRB-08-C-0093.
Current funding is provided by a grant from the Intelligence Advanced Research Projects Activity. 26