1. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 1 Handwritten Signatures Authentication Using Artificial Neural Networks Committee Machines Milton Roberto Heinen - Applied Computing / Unisinos and II / UFRGS Prof. Dr. Fernando S. Osório - Applied Computing / Unisinos Prof. Dr. Paulo M. Engel - Informatics Institute / UFRGS Applied Computing - PIPCA / Unisinos Informatics Institute - II / UFRGS UNISINOS University and UFRGS University - Brazil Web: http://inf.unisinos.br/~osorio/NeuralSignX CLEI 2007 San José - Costa Rica - October 2007

2. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 2 Handwritten Signatures Authentication Using Artificial Neural Networks Committee Machines Milton Roberto Heinen - Applied Computing / Unisinos and II / UFRGS Prof. Dr. Fernando S. Osório - Applied Computing / Unisinos Prof. Dr. Paulo M. Engel - Informatics Institute / UFRGS Applied Computing - PIPCA / Unisinos Informatics Institute - II / UFRGS UNISINOS University and UFRGS University - Brazil Web: http://inf.unisinos.br/~osorio/NeuralSignX CLEI 2007 San José - Costa Rica - October 2007 Presented by Cássia Nino - Applied Computing / Unisinos

3. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 3 Presentation Topics Agenda: 1. Motivation and Context 2. Handwritten Signature Authentication 3. NeuralSignX System a) Modules: Acquisition, Pre-Processing, ANN Classification b) Specialists Committee 4.Description of Experiments 5.Signature Authentication Results 6.Conclusions

4. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 4 1. Motivation and Context Motivation: Security How to guarantee a legitimate access to resources: Data access, Bank access, Passport, Credit Card, etc. Do you really are who you claim to be ?? Context: Biometric SystemsSignature Identify: who you are Off-Line Verify: authenticate one user Scanned Signature (static) Physical features: On-Line Fingerprints, hand, eyes,... Digital Pen (dynamic) Behavioral features: Signature, keyboard typing, voice,...

5. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 5 1. Motivation and Context Motivation: Security How to guarantee a legitimate access to resources: Data access, Bank access, Passport, Credit Card, etc. Do you really are who you claim to be ?? Context: Biometric SystemsSignature Identify: who you are Off-Line Verify: authenticate one user Scanned Signature (static) Physical features: On-Line Fingerprints, hand, eyes,... Digital Pen (dynamic) Behavioral features: Signature, keyboard typing, voice,...

6. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 6 1. Motivation and Context Motivation: Security How to guarantee a legitimate access to resources: Data access, Bank access, Passport, Credit Card, etc. Do you really are who you claim to be ?? Context: Biometric SystemsSignature Identify: who you are Off-Line Verify: authenticate one user Scanned Signature (static) Physical features: On-Line Fingerprints, hand, eyes,... Digital Pen (dynamic) Behavioral features: Signature, keyboard typing, voice,... This Work

7. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 7 2. Handwritten Signature Authentication Signature Authentication Binary Classification: - True: Authentic [ not exactly the same as a previously known ] - False: Random, Traced, Skilled On-Line Signatures - Dynamical and Temporal Information - Easy to access input devices... tablet + pen Becoming more usual nowadays: Tablet PCs Palm Tops Cell phones Portable Games Low Cost Digitizing Tables

8. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 8 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification

9. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 9 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification LOGIN=MILTON 116 478 0 21:42:23:821 116 478 1 21:42:23:831 125 467 1 21:42:23:842 134 456 1 21:42:23:852 145 441 1 21:42:23:873 168 444 0 21:42:23:883 198 373 1 21:42:23:894 Position X Position Y Pen Up/Down Time Stamp

10. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 10 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module Collected Data... 2. Pre-Processing Module 3. ANN Module - Learning - Classification Signatures Database TOTAL: 2440 Signatures 1800 authentic signatures (30 per user, 60 users) 320 false (traced forgeries) 320 false (skilled forgeries) Select 1 user to authenticate All other signatures can be used as non-authentic The user signatures are divided into: training set / validation set Typical authentic user training data set is composed by 10 to 15 signatures

11. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 11 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification Position and Scale Adjust

12. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 12 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification Features Extraction - Signature elapsed time (1 value) - Number of pen lifts (1 value) - Signature length (1 value) - Medium and maximum pen velocity (2 values) - Number of pen X, Y direction changes (2 values) - Cardinal points measure (8 values) - Pseudo-vectors total length (8 values) - Signature density grid (48 values) - Vertical and horizontal intersections (26 values) - Sequential sampling (16 values) - Symmetry measure (2 values) Total: 117 values! (or even more)

13. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 13 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification Features Extraction - Signature elapsed time (1 value) - Number of pen lifts (1 value) - Signature length (1 value) - Medium and maximum pen velocity (2 values) - Number of pen X, Y direction changes (2 values) - Cardinal points measure (8 values) - Pseudo-vectors total length (8 values) - Signature density grid (48 values) - Vertical and horizontal intersections (26 values) - Sequential sampling (16 values) - Symmetry measure (2 values) Total: 117 values! (or even more) Large Input Space: Reduced using PCA and other techniques presented in some of our previous works IEEE/IJCNN 2006, SBIA/SBRN/WCI 2006 and SBC/ENIA 2005

14. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 14 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning - Classification Features Extraction - Signature elapsed time (1 value) - Number of pen lifts (1 value) - Medium and maximum pen velocity (2 values) - Cardinal points measure (8 values) - Signature density grid (48 values) - Vertical and horizontal intersections (26 values) Total: 86 input values In this work: we selected these 6 features represented by 86 values NeuralsignX pre-processing module is available at: http://inf.unisinos.br/~osorio/NeuralSignX/nsx.html

15. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 15 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module ANN - Artificial Neural Networks Neural Learning: Select 1 user as authentic Learn how to classify as - Authentic Signature - Non Authentic Signature Classification: Classification: Committee of three specialists (5 Neural Nets each) Inputs: 86 values Output: 1 binary (Authentic or Not)

16. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 16 3. NeuralSignX System NeuralSignX Modules: 1. Acquisition Module 2. Pre-Processing Module 3. ANN Module - Learning: > MLP with RProp - Classification: Main goals > Improve generalization > Reduce FP (False Positive) FP: False Positive - False signature classified as authentic (accepted) FN: False Negative - Authentic signature classified as false (refused) Inputs: 86 values are divided into 3 Esp s Output: 1 combined binary output Simulation Tool: SNNS Simulator Committee of three specialists

17. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 17 4. Description of Experiments Experiments: Signatures Database 1800 Authentic 320 Traced 320 Skilled ================= 2440 Signatures (usually from 10 to 30 different signatures per user) 10 Users were authenticated 10 Folds Cross-Validation 10 Runs each folder (≠ initializations) 1000 experiments Results: HIT = % of correct answer FPR = False Positive Rate (accepted) FNR = False Negative Rate (refused) Single ANN: 86 inputs, 5 hidden, 1 output versus Committee: 3 specialists (5 ANN in each one) Experiment Results:

18. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 18 5. Signature Authentication Results Results: One Single ANN MSE = Mean Squared Error HIT = % of correct answer FPR = False Positive Rate (accepted) FNR = False Negative Rate (refused) Top: FPR = 0.14 % FNR = 23.00 % Global mean HIT = 99.86 % FPR = 0.05 % FNR = 8.03 % Single ANN High FNR error rate

19. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 19 5. Signature Authentication Results Results: Specialists committee MSE = Mean Squared Error HIT = % of correct answer FPR = False Positive Rate (accepted) FNR = False Negative Rate (refused) Top: FPR = 0.00 % FNR = 13.33 % Global mean HIT = 99.95 % FPR = 0.00 % FNR = 4.00 % Committee Lower FNR error rate

20. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 20 5. Signature Authentication Results Comparing Results: Single x Specialists committee MSE = Mean Squared Error HIT = % of correct answer FPR = False Positive Rate (accepted) FNR = False Negative Rate (refused) Top: FPR = 0.00 % FNR = 13.33 % Global mean HIT = 99.95 % FPR = 0.00 % FNR = 4.00 % Committee Top: FPR = 0.14 % FNR = 23.00 % Global mean HIT = 99.86 % FPR = 0.05 % FNR = 8.03 % Single ANN Specialists committee performs better!

21. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 21 6. Conclusions Conclusions and Future Work We proposed: Authentication using ANN Committee Machines We obtained: a more robust and secure authentication system - Pre-Processing: Good Features = Good Results - Classification: Specialist Committee are able to... Improve Generalization (HIT) ! Reduce Errors (FPR and FNR) ! Future Work: How to Improve Security ? Hybrid User Authentication System Combine Specialistscombine different biometric techniques" "Combine Specialists but also combine different biometric techniques" Composed authentication: Signature, Eye scanning, Fingerprints, etc.

22. Handwritten Signatures Authentication using ANNs Committee Machines M.Heinen, F. Osório and P. Engel October 2007 22 CONTACT INFORMATION UNISINOS University - Brazil Applied Computing Research Post-grad Program - PIPCA Artificial Intelligence Research Group - GIA UFRGS University - Brazil Informatics Institute - II NeuralSignX Project Web Page: http://inf.unisinos.br/~osorio/NeuralSignX/nsx.html Contact: Milton Heinen, Fernando Osório and Paulo Engel Web: http://inf.unisinos.br/~osorio/ E-Mail: fosorio@unisinos.br