1. Match Score Fusion of Fingerprint and Face Biometrics for Verification
chiung ching ho, mufaddal ali hussin, hu-ng
Data Science SIG
Faculty of Computing and Informatics
Multimedia University
IIC’2015 | International Conference on Intelligent and Interactive Computing

2. Presentation Outline Introduction Research question Key literature
Method
Results
Discussion
Conclusion
References
Contact Information
IIC’2015 | International Conference on Intelligent and Interactive Computing

3. Introduction
IIC’2015 | International Conference on Intelligent and Interactive Computing

4. Millions of passwords and usernames compromised
“Although researchers typically only release passwords, I am releasing usernames with the passwords. Analysis of usernames with passwords is an area that has been greatly neglected and can provide as much insight as studying passwords alone. Most researchers are afraid to publish usernames and passwords together because combined they become an authentication feature”
Mark Burnett, a security consultant who released 10 million usernames and passwords
IIC’2015 | International Conference on Intelligent and Interactive Computing

5. Face and fingerprint biometric as an alternative to passwords
CHALLENGES :
Lack of standardization in
Biometric sensors
Biometric data interchange
Biometric sensor resolution
Non-cancelable Biometric features
IIC’2015 | International Conference on Intelligent and Interactive Computing

6. Research question
How does the fingerprint-face match-scores feature vectors perform as compared to rule-based match- score for verification purposes ?
IIC’2015 | International Conference on Intelligent and Interactive Computing

7. Key literature
IIC’2015 | International Conference on Intelligent and Interactive Computing

8. Biometric verification
Biometric verification is the process whereby a user of the verification system claims to be a particular person, and provides the biometric sample. The role of the verification system is to check if the user is indeed who he or she claims to be Haffeman et al. (2015). The biometric traits can be either physical or physiological in nature, and is unique for individuals. Biometric verification can be treated as a two-class problem.
IIC’2015 | International Conference on Intelligent and Interactive Computing

9. Multimodal biometric for person verification
The use of multimodal biometric for person verification was pioneered by Ben-Yacoub et al. (1999) , who fused face and speech data for verification. Kumar et al. (2003) attempted person verification by fusing palm- print and hand geometries. A recent work by Talgad et al. (2014) fused fingerprint and face biometrics at the match-score level for person identification.
IIC’2015 | International Conference on Intelligent and Interactive Computing

10. Score-level fusion
Previous works on multimodal biometric fusion suggests a hierarchy of effectiveness among the different levels of fusion. Feature-level fusion has been proven to outperform match-score level fusion, and in turn match-score level fusion outperforms decision-level fusion as shown by Sree and Radha (2015) and Meva and Kumbharana (2013). ? 1 ?
1) Sensor Level. Multisensorial biometric systems sample the same instance of a biometric trait with two or more distinctly different sensors [14]. Processing of the multiple samples can be done with one algorithm or combination of algorithms. Example face recognition application could use both a visible light camera and an infrared camera coupled with specific frequency.
(2) Feature Level. The feature level fusion is useful in classification [14]. Different feature vectors are combined, obtained either with different sensors or by applying different feature extraction algorithms to the same raw data [21].
(3) Decision Level. With this approach, each biometric subsystem completes autonomously the processes of feature extraction, matching, and recognition. Decision strategies are usually of Boolean functions, where the recognition yields the majority decision among all present subsystems [9].
(4) Rank Level. Instead of using the entire template, partitions of the template are used. Ranks from template partitions are consolidated to estimate the fusion rank for the classification [18]. Rank level fusion involves combining identification ranks obtained from multiple unimodal biometrics. It consolidates a rank that is used for making final decision [19].
(5) Score Level. It refers to the combination of matching scores provided by the different systems. The score level fusion techniques are divided into two main sets: fixed rules (AND, OR, majority, maximum, minimum, sum, product and arithmetic rules) and trained rules (weighted sum, weighted product, fisher linear discriminate, quadratic discriminate, logistic regression, support vector machine, multilayer perceptrons, and Bayesian classifier ) [22]. Figure 2 shows the five levels of biometric fusion. 3 2
IIC’2015 | International Conference on Intelligent and Interactive Computing

11. Method
IIC’2015 | International Conference on Intelligent and Interactive Computing

12. The Dataset : BSSR1
“Set 1 is comprised of face and fingerprint scores from the same set of 517 individuals. For each individual, the set contains one score from the comparison of two right index fingerprints, one score from the comparison of two left index fingerprints, and two scores (from two separate matchers) from the comparison of two frontal faces. The fingerprint images and the face images from which these scores were computed are from the same person (i.e. the chimera assumption is not in use) at the same time (and the dates are provided). The non-matching scores from the full cross-comparison are also included.”
IIC’2015 | International Conference on Intelligent and Interactive Computing

13. Methodology 1
The methodology used in our work is shown in the diagram on the right.
Pre-processing of the data includes score normalization as the scores in the BSSR1 data set are set to different scales.
Subsequently, experiments were conducted in two-research directions:
Match score as a concatenated feature vector
Sum and product rules for match- score level fusion 2 2 3 3
1) Sensor Level. Multisensorial biometric systems sample the same instance of a biometric trait with two or more distinctly different sensors [14]. Processing of the multiple samples can be done with one algorithm or combination of algorithms. Example face recognition application could use both a visible light camera and an infrared camera coupled with specific frequency.
(2) Feature Level. The feature level fusion is useful in classification [14]. Different feature vectors are combined, obtained either with different sensors or by applying different feature extraction algorithms to the same raw data [21].
(3) Decision Level. With this approach, each biometric subsystem completes autonomously the processes of feature extraction, matching, and recognition. Decision strategies are usually of Boolean functions, where the recognition yields the majority decision among all present subsystems [9].
(4) Rank Level. Instead of using the entire template, partitions of the template are used. Ranks from template partitions are consolidated to estimate the fusion rank for the classification [18]. Rank level fusion involves combining identification ranks obtained from multiple unimodal biometrics. It consolidates a rank that is used for making final decision [19].
(5) Score Level. It refers to the combination of matching scores provided by the different systems. The score level fusion techniques are divided into two main sets: fixed rules (AND, OR, majority, maximum, minimum, sum, product and arithmetic rules) and trained rules (weighted sum, weighted product, fisher linear discriminate, quadratic discriminate, logistic regression, support vector machine, multilayer perceptrons, and Bayesian classifier ) [22]. Figure 2 shows the five levels of biometric fusion. 4 4 5
IIC’2015 | International Conference on Intelligent and Interactive Computing

14. Methodology
1) Sensor Level. Multisensorial biometric systems sample the same instance of a biometric trait with two or more distinctly different sensors [14]. Processing of the multiple samples can be done with one algorithm or combination of algorithms. Example face recognition application could use both a visible light camera and an infrared camera coupled with specific frequency.
(2) Feature Level. The feature level fusion is useful in classification [14]. Different feature vectors are combined, obtained either with different sensors or by applying different feature extraction algorithms to the same raw data [21].
(3) Decision Level. With this approach, each biometric subsystem completes autonomously the processes of feature extraction, matching, and recognition. Decision strategies are usually of Boolean functions, where the recognition yields the majority decision among all present subsystems [9].
(4) Rank Level. Instead of using the entire template, partitions of the template are used. Ranks from template partitions are consolidated to estimate the fusion rank for the classification [18]. Rank level fusion involves combining identification ranks obtained from multiple unimodal biometrics. It consolidates a rank that is used for making final decision [19].
(5) Score Level. It refers to the combination of matching scores provided by the different systems. The score level fusion techniques are divided into two main sets: fixed rules (AND, OR, majority, maximum, minimum, sum, product and arithmetic rules) and trained rules (weighted sum, weighted product, fisher linear discriminate, quadratic discriminate, logistic regression, support vector machine, multilayer perceptrons, and Bayesian classifier ) [22]. Figure 2 shows the five levels of biometric fusion.
Multimodal biometric datasets were created by combining the match-scores from the face recognition systems C & G, with the scores from the fingerprint recognition systems
Li and Ri.
IIC’2015 | International Conference on Intelligent and Interactive Computing

15. Match-score as a feature vector
A 1034-length feature vector was concatenated from the face biometric modality (517) and the fingerprint biometric modality (517).
To shorten the feature vector, feature selection was performed using the gain ratio feature selection technique.
224 individuals were placed in the Enrollee class, while 293 individuals were placed in the User class.
Subsequently, 10-fold cross validation was performed using three classifiers:
Bayes Network
K-Nearest Neighbour
Support Vector Machines
IIC’2015 | International Conference on Intelligent and Interactive Computing

16. Classification results performed post Min-max normalization
IIC’2015 | International Conference on Intelligent and Interactive Computing

17. Simple Sum and Product Rules
For the Simple Sum rule, the match score for each biometric modality is summed up, while the Product rule multiplied the match score for each biometric modality.
Subsequently, a threshold value was chosen to evaluate whether to a score belonged to a Enrollee or to a User.
The TPR and FPR rates were calculated, and subsequently the Area Under ROC curve were calculated as the effective measure.
IIC’2015 | International Conference on Intelligent and Interactive Computing

18. Classification results using Simple Sum and Product Rules post Min-max Normalization
IIC’2015 | International Conference on Intelligent and Interactive Computing

19. Discussion
IIC’2015 | International Conference on Intelligent and Interactive Computing

20. Match-score feature vector vs Simple Rules
It is clear that using match-scores as a feature vector is produces superior results
as compared to performing verification at the match-score level itself using simple
sum and product rules
However, the usage of simple sum and product rules are comparable to the results
obtained using certain normalization techniques (Tan H and MAD), and as such
can be a viable alternative when classification cost becomes prohibitive
IIC’2015 | International Conference on Intelligent and Interactive Computing

21. Conclusion
IIC’2015 | International Conference on Intelligent and Interactive Computing

22. The results obtained in this paper demonstrates that it is a viable choice to perform verification at a match score level, especially when the match-score is used as a feature vector.
This approach will go some ways in solving the problems caused by heterogeneous biometric sensors and the lack of standards for biometric data interchange.
In future, the transmission of match-score as opposed to the transmission of raw biometric data may well pave the way for incorporating IoT biometric devices for future biometric applications.
IIC’2015 | International Conference on Intelligent and Interactive Computing

23. Thank you! My contact details: Dr. Ho Chiung Ching FCI, MMU
IIC’2015 | International Conference on Intelligent and Interactive Computing

24. References
A-survey-on-fusion-techniques-formultimodal-biometric-identification.pdf. (n.d.). Retrieved from biometric-identification.pdf
Ben-Yacoub, S., Abdeljaoued, Y., & Mayoraz, E. (1999). Fusion of face and speech data for person identity verification. IEEE Transactions on Neural Networks, 10(5), 1065–
Hafemann, L. G., Sabourin, R., & Oliveira, L. S. (2015). Offline Handwritten Signature Verification - Literature Review. arXiv: [cs, Stat]. Retrieved from
Jain, A., Nandakumar, K., & Ross, A. (2005). Score normalization in multimodal biometric systems. Pattern Recognition, 38(12), 2270–
Kumar, A., Wong, D. C. M., Shen, H. C., & Jain, A. K. (2003). Personal Verification Using Palmprint and Hand Geometry Biometric. In J. Kittler & M. S. Nixon (Eds.), Audio- and Video- Based Biometric Person Authentication (Vol. 2688, pp. 668–678). Berlin, Heidelberg: Springer Berlin Heidelberg. Retrieved from
Meva, D. T., & Kumbharana, C. K. (2013). Comparative study of different fusion techniques in multimodal biometric authentication. International Journal of Computer Applications, 66(19), 16–19.
Ross, A. A., & Govindarajan, R. (2005). Feature level fusion of hand and face biometrics. In A. K. Jain & N. K. Ratha (Eds.), (pp. 196–204).
Ross, A., & Jain, A. (2003). Information fusion in biometrics. Pattern Recognition Letters, 24(13), 2115–
Sree, S. R. S., & Radha, N. (2015). A Survey on Fusion Techniques for Multimodal Biometric Identification. International Journal of Innovative Research in Computer and Communication Engineering, 02(12), 7493–
Telgad, R. L., Deshmukh, P. D., & Siddiqui, A. M. N. (2014). Combination approach to score level fusion for Multimodal Biometric system by using face and fingerprint. In Recent Advances and Innovations in Engineering (ICRAIE), 2014 (pp. 1–8).
IIC’2015 | International Conference on Intelligent and Interactive Computing