Presentation is loading. Please wait.

Presentation is loading. Please wait.

Multimodal User Authentication: From Theory to Practice

Similar presentations


Presentation on theme: "Multimodal User Authentication: From Theory to Practice"— Presentation transcript:

1 Multimodal User Authentication: From Theory to Practice
TUTORIAL Conference IEEE ICME 2003 Speakers Jean-Luc DUGELAY Jean-Claude JUNQUA Location Baltimore Date & Time Sunday, 6 July 2003, 13: :00

2 Who we are… Jean-Luc DUGELAY Jean-Claude JUNQUA Ph.D. 92
Professor at Eurécom Sophia Antipolis, France Security Imaging Watermarking Biometrics Jean-Claude JUNQUA Ph.D. 89 Director of PSTL (Panasonic Speech Technology Laboratory) Santa Barbara, California, U.S.A. Speech Recognition Synthesis Multimodal Dialogue Speaker Verification

3 Outline (1/3) Multimodal user authentication: Background Introduction
Is there a universal biometric identifier? What are the factors influencing the reliability of biometric systems? Why are there still very few biometric systems in use today? Physiological versus behavioral biometrics Why multimodal biometrics? Can multimodal biometrics improve performance? Tradeoffs between robustness (security) and convenience

4 Outline (2/3) Main individual modalities
Signature Voice Hand Fingerprint Face Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, teeth) [Specificities, Pros & cons, Open problems, Sensing technologies, Major algorithms, Database examples, …] Towards Multimodal Biometric Systems Sequence Fusion

5 Outline (3/3) Applications, Standards and Evaluation
Main application areas Biometrics and privacy Important criteria to deploy multimodal authentication systems Biometric standards Multimodal databases Best practices in testing biometric systems Examples of multimodal user authentication systems Perspectives and future challenges Demonstrations Forthcoming events Bibliography

6 Outline (1/3) Multimodal user authentication: Background Introduction
Is there a universal biometric identifier? What are the factors influencing the reliability of biometric systems? Why are there still very few biometric systems in use today? Physiological versus behavioral biometrics Why multimodal biometrics? Can multimodal biometrics improve performance? Tradeoffs between robustness (security) and convenience

7 Introduction Drawbacks of traditional identification
(knowledge- or token- based): PIN may be forgotten or guessed by imposter Physical keys may be misplaced or lost It is not possible to differentiate between an authorized person and an imposter Biometric system Pattern recognition system which establishes the authenticity of a specific physiological or behavioral user’s characteristic Relies on “who you are or what you do” to make a positive personal identification Comprises an enrollment stage and an identification/verification stage Identification (1:N matching, who am I?) & Verification (1:1 matching, Am I who I claim I am?) → Well-known example: login + passwd The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

8 Seven types of authentication:
Something you know (1) e.g. PIN code, mother’s maiden name, birthday Something you have (2) e.g. Card, key Something you know + something you have (3) e.g. ATM card + PIN Something you are – Biometrics (4) no PIN to remember, no PIN to forget Something you have + something you are (5) Smart Card Something you know + something you have (6) Something you know + something you have + something you are (7) Security level 7 5,6 4 3 1, 2 Types

9 A Generic Biometric System
Sensor (e.g. Microphone, Camera) Feature Extraction Enrollment and Template Storage Enrollment Template Adaptation Identification/Verification Sensor Feature Extraction Identification/ Verification Now let us have closer look at the biometric system During the enrolling phase, the chosen biometric is captured. Next, the biometric is processed and the template is extract and enrolled. The template is then stored in a local, central repository or a portable token. In the identification phase, live-scan the chosen biometric. Process the biometric and extract template. The extracted template is then compared against the stored templates. It then provide a matching score for the business applications And record a secure audit trail with respect to system use. Action

10 How to measure performance
Biometric systems are not perfect. They make errors in identifying or true claimants and in rejecting imposters The probability of committing these two types of errors are called False Rejection Rate (FRR) False Acceptance Rate (FAR) ROC: Receiver Operating Characteristic Now let us take a closer look at each of these terms that affect the biometric system performance – firstly, accuracy Typically, biometric identification systems are (will be) not perfect and make errors in correctly recognizing genuine identities (false negative) and in correctly rejecting imposters (false positive). The probability of committing these two types of errors are termed as false reject rate (FRR) false acceptance rate (FAR) The magnitudes of these errors depend upon how liberally / conservatively a biometric system determines whether the two measurements originate from the same individual. A smaller FRR will tend to lead to a larger FAR, while a smaller FAR will usually implies a larger FRR. FRR is user-dependent

11 Performance & Evaluation

12 Performance & Evaluation
Martin et al., « The DET curve in assessment of detection task performance » Proc. EuroSpeech 1997. 100% 10% 1% 0.1% 0.0001% 0.001% 0.01% FAR FRR Better performance Detection error trade-off (DET) curves (uniform treatment of both types of error)

13 State-of-the-art error rates
Table adapted from State-of-the-art error rates Test data FRR FAR Fingerprint 20 years (average age) 0.2% Face 11 to 13 months spaced 10-20% 0.1-20% Text-dependent Speaker verification Text-dependent (entrance door, 3 months period ) 1-3% Text-independent speaker verification Text-Independent (NIST 2000) 2-5%

14 Some other performance criteria
Failure and difficulties* to enroll (e.g. amount of data) Failure and difficulties to acquire Performance False rejection rate False acceptance rate * 4% of fingerprints are of poor quality

15 Is there a universal biometric identifier?
There are many biometric identifiers: Fingerprint Voice Image Hand geometry Retina Iris Signature Keystroke dynamics Gait DNA (requires physical sample) Wrist/hand veins Brain activity etc. In theory many of these biometric identifiers should be universal. However, in practice this is not the case. Ideally, a biometric identifier should be universal, unique, permanent and measurable However, in practice each biometric identifier depends on factors such as users’ attitudes, Personality, operational environment, etc.

16 Each biometric identifier has its strengths and weaknesses
Adapted from Source: * Also, (Vol. 1, Issue 01) & ** (Dr. J. Wayman) Each biometric identifier has its strengths and weaknesses

17 What are the factors influencing the reliability of biometric systems
The factors influencing the reliability of biometric systems depend on the biometric identifier used. Understanding the requirements, the users and the environment is the key However, some general factors can be identified User behavior/cooperativeness Stability (time and environment) of the biometric identifier How easy is it to use the system? Is the user accustomed to the use of the biometric sensor? Quality of the enrollment Population demographics User interface The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

18 Influence of the user and the environment
Behavior Consistency Physiology Appearance Familiarity with the equipment The environment Lighting Background noise Weather (e.g. humidity) The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

19 Influence of time As the time between enrollment and testing increases, the biometric features enrolled are generally becoming less reliable e.g. - 50% decrease in performance after a period of 1 year for a biometric system based on faces. However, as the user keeps using the biometric system he/she tends to adapt to the biometric system Supervised or unsupervised adaptation helps dealing with the mismatch between enrollment and testing The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

20 Why are there still very few biometric systems in use today?
Main reasons Accuracy User acceptance & social factors (e.g. lack of familiarity, privacy) Standards Every biometric has its limitations Cost of deployment Ease of use Ease of development (e.g. standards) Lack of understanding on how to combine biometric identifiers Difficulties to enroll a large set of individuals Lack of large scale deployments The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

21 Why are there still very few biometric systems in use today?
Social factors Informational privacy (collection, storage and use of the user information) Personal privacy (how invasive or intrusive is the biometric identifier used?) Political will/cultural climate User acceptance/familiarity of the technology The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

22 Physiological versus behavioral biometrics
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Physiological versus behavioral biometrics Physiological: what we Are or Have Behavioral: what we Do Thus Physiological is static, Behavioral is dynamic Combinations provide potential for robustness

23 Physiological: Behavioral:
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Examples of physiological and behavioral biometrics Physiological: Behavioral: what we Are/Have what we Do Eye scans are Fingerprints are Face are Handwriting do Gait do Speech (audio) do Speech (visual) are do

24 Behavioral & Physiological Biometrics
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Behavioral & Physiological Biometrics Audio Behavioral Instantaneous ‘snap-shots’ inherently Behavioral (information signal is a function of time) Visual Physiological Instantaneous ‘snap-shots’ inherently Physiological (information signal is not a function of time, though with speech it can become so)

25 Dynamics in Biometrics
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Dynamics in Biometrics Physiological: what we Are or Have Possible / maybe detrimental Slow/small/nil One-off possible Behavioral: what we Do Essential Inherent/ unavoidable (nuisance) Multi-session / adaptive to capture inherent variation Implications Movement Signature Variation (undesirable) System Enrollment/ Training Of course, ‘signatures’ must always relate to physical properties, some less than others, e.g. gait – highly so, speech or handwriting – perhaps less so

26 Speech as a Biometric Speech    Physiological: Behavioral:
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Speech as a Biometric Speech Visual (lips) Acoustic Physiological: Are / Have Behavioral: Do (Dynamics) Error rates Quantity/quality of data

27 Visual Speech Biometric
From: The Role of Dynamics in Visual Speech Biometrics Presentation by John Mason and Jason Brand, ICASSP 2002 Visual Speech Biometric Instantaneous lip contours from a series of frames in speaking mode

28 Why Multimodal Biometrics?
Introduction No single biometric is generally considered sufficiently accurate and user-acceptable for any given application Authentication systems that are robust in natural environments (e.g. in the presence of noise and illumination changes) cannot rely on a single modality Multimodal user authentication can provide a more balanced solution to the security and convenience requirements of many applications There is a clear requirement for the system to be able to adapt to the user needs and conditions and, especially, to be able to determine and maintain an acceptable balance between confidence and convenience for its users. Each individual biometric can operate in either a verification mode or an identification mode The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

29 Multimodal Biometrics
Generic architecture Biometric sensor 1 Biometric feature 1 Decision fusion Claimed identity accepted or rejected Biometric feature N Biometric sensor N The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach Biometric database

30 Can multimodal biometrics improve performance?
Introduction Multimodal user authentication provides a practically viable approach for overcoming the performance and acceptability barriers to the widespread adoption of authentication systems Integration of multimodal biometric modalities is strongly based on a thorough understanding of each of the modalities and the different sensing technologies A fully successful multimodal fusion can only be obtained through a careful investigation of these technologies and their interaction Multimodal biometrics can improve accuracy or speed (e.g. face recognition, can be used to index a template database and fingerprint verification can be used to ensure the overall accuracy) There is the perception that if a a strong test is combined with a weaker test, the resulting decision is averaged. However, the performance improvement comes from a well-designed fusion algorithm which should take advantage of additional information The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

31 Can multimodal biometrics improve performance?
Sequential A B The FAR is determined by the FAR of both systems Fusion A B Fusion Parallel A B The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach A and B provides separate scores The fusion algorithm decides Can produce a very low FAR as well as a very low FRR If A rejects then B is used The FRR is determined by both systems

32 Pros and Cons of Multimodal Biometrics
Can overcome weaknesses of individual biometric identifiers Can extend the operation range to a larger target user population Can increase the reliability of the decision made by a single biometric system Is generally more robust to fraudulent technologies (it is more difficult to forge multiple biometric characteristics) If well-designed can improve performance and speed Cons Can make the interaction longer Cost of deployment is generally higher Integration of multiple biometrics is more complex (score normalization, etc.) The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach

33 Trade-off between robustness (security) and convenience
A very secure system will have a higher rejection rate Or it will have different passes to increase security at the expense of user convenience For some modalities (e.g. voice) the amount of enrollment data is directly related to this tradeoff Robustness (Security) The drawbacks of these approaches are that tokens may stolen, lost, forgotten, or misplaced; PIN may be forgotten, or guessed by imposter But most importantly, these traditional approaches unable us to differentiate between an authorized person and an imposter Hence, there is a need to another approach in personal identification – the biometric approach Convenience

34 Outline (2/3) Main individual modalities
Signature Voice Hand Fingerprint Face Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears) [Specificities, Pros & cons, Open problems, Sensing technologies, Major algorithms, Database examples, …] Towards Multimodal Biometric Systems Sequence Fusion

35 Modality by modality Signature Voice Hand Fingerprint Face

36 Signatures: introduction
M. EL Yassa et al., « ETAT DE L'ART SUR LA VÉRIFICATION OFF-LINE DE SIGNATURES MANUSCRITES» SETIT 2003, Tunis. Signatures: introduction Several types of signatures American Close to recursive handwritten European Includes a graphical component Arabic etc. Is a signature authentic or not? Main difficulty – Intra-class variations (a signature of one individual) Easier to forge than other biometric attributes European American

37 Signatures: static vs. dynamic
Off-line versus on-line signature Off-line Only spatial information is available Static: shape of the signature On-line Add. features: velocity, pressure, etc. Dynamic: the dynamics of how you sign (such as speed, pressure, and timing) + time

38 Signatures: acquisitions
Off-line Scans On-line Special hardware: Digitizing tablet Pressure sensitive pen “e-pad”

39 Signatures: Forgeries
Random forgeries The forger has either no knowledge about the original signature, or does not try to imitate the shape of the signature. Zero-effort forgeries Skilled forgeries Others: Counter-drawing Disguise

40 Signatures: Classical criteria
Alignments: baseline & envelope Drawing characteristics: upward / downward drawing Speed Proportions Pressure

41 Signatures: Example of features
Features: centroid, baseline, top/down envelops

42 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, teeth)

43 Types of Speaker Recognition Systems
Three types of speech-based authentication: Text-dependent systems. User enrolls & authenticates with same password (template approach) Advantage: low memory, low processing power, low cost. Disadvantage: impostor can record client’s voice Prompted phrases/passwords system (HMM phoneme models) Advantage: combination of recognition and adaptation can improve performance Disadvantage: less natural Text-independent systems. Enrollment speech and authentication speech are different (single state HMM, GMM) Advantage: very secure Disadvantage: lower accuracy, need more resources, more enrollment and authentication speech

44 Text-Dependent Authentication
Speaker Enrollment Enrollment speech “Santa Barbara, California” Simple Model for P Rightful user P Speaker Verification Impostor Model Simple Model for P Test speech S “Santa Barbara, California” Compare scores Claimant C Reject C Accept C

45 Text-Independent Authentication
Speaker Enrollment Enrollment speech “April is the cruellest month” Model for P (GMM) Rightful user P Speaker Verification Model for P Impostor model I Test speech S “Do I dare to eat a peach?” log p(S|P)-log p(S|I) > T? yes no Accept C Reject C Claimant C

46 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, teeth)

47 Hand geometry: acquisition
“Hand punch” Fairly simple and accurate But human hand is not unique; only used for verification (not descriptive enough for identification) Usage: Some people are reluctant to put their hands on the same support used previously by others Special hardware; the hand is properly aligned by the pegs fingers

48 Hand geometry 14 axes along which features values are computed.
Reference: A prototype Hand Geometry-based Verification System, Arun Ross Hand geometry 14 axes along which features values are computed. (5 pegs serve as control points and assist in choosing these axes). Ps and Pe refer to the end points (using gray scale profile) → feature vector

49 Hand/Finger geometry Measurements (4 categories)
Reference: Biometric Identification through Hand Geometry Measurements R. Sanchez-Reillonet al. IEEE PAMI Vol. 22 no. 10, 2000. Hand/Finger geometry Measurements (4 categories) Widths: Palm, plus each of the four fingers is measured in different heights Heights: middle, little and palm. Deviations: distance between a middle point of the finger and the middle point of the straight line between the interfinger point and the last height where the finger width is measured. Angles: between the interfinger points and the horizontal. Classification and Verification Euclidean, Hamming, Gaussian Mixture Models (GMMs), Radial Basis Function Neural Networks (RBF).

50 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, teeth)

51 Fingerprints: acquisition
Inked fingerprints: finger is rolled or dabbed on a sheet of paper Live-scan fingerprints: no need of an intermediate medium like paper; systems are optical, thermal, electromagnetic or ultrasound based Quality fingerprint acquisition is extremely challenging: elastic distortion of the finger on the acquisition surface dry/wet skin scars, cuts, presence of dirt/ grease, etc. Exact position of the finger on the scanner machine (i.e. slight rotation are possible) Pressure of the finger on the surface of the acquisition machine Degree of finger moisture at the contact area Source:

52 Fingerprints: image examples
Subset of ST Microelectronics’ Fingerprint Image Database Fingerprints: image examples

53 Fingerprints: global classification
Global patterns of ridges and furrows form special configurations in the central region of fingerprints Arch Left Loop Right Loop Whorl 6% 34% 32% 28% Class information is not sufficient to carry out recognition Can be used for clustering: once a fingerprint is classified, it can be matched only with a subset of the database (Courtesy of ST Microelectronics)

54 Fingerprints: local classification
Local ridge characteristics determine the uniqueness of a fingerprint bifurcation ending bridge lake island Ridge endings and bifurcations are usually used for their robustness and stability Most automatic fingerprint matching algorithms mimic the process used by forensic experts to perform recognition: minutiae extraction template matching (Courtesy of ST Microelectronics)

55 Fingerprints: Typical algorithm for Minutiae extraction
Reference: On-Line Fingerprint Verification Anil Jain & Ruud Bolle IEEE T-PAMI, Vol. 19, No. 4, April 1997 Fingerprints: Typical algorithm for Minutiae extraction Minutiae Extraction Smoothing Filter Oriented Field Estimation Fingerprint Region Localization Ridge Extraction Thinning Minutiae extraction (Courtesy of ST Microelectronics)

56 Fingerprints: Matching
Alignment stage (global) Adjustment (local) (Courtesy of ST Microelectronics)

57 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Reference: Human and Machine Recognition of Faces: A Survey R. Chellappa et al. Proc. of the IEEE Vol. 83., No. 5, May 1995 Modality by modality Signature Voice Hand Fingerprint Face Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears)

58 Face: frontal face recognition
Face detection - Is there a face? Face segmentation - Where? Face tracking (if video) Face size and position. In practice, it is very difficult to control the position of the subject with respect to the camera → “Normalize” inter-ocular distance… Changes in illumination. If a spotlight is not used, lighting variations occur. For example, close to a window, the lighting depends strongly on the time of the day and the weather → “Normalize” gray-scale histogram… Facial expressions. In practice, it is almost impossible to control the mood of the subject. The smile causes probably the largest variation of facial expressions Others. Glasses, Hats, Facial hair, etc.

59 Face (frontal): Image Examples database: ORL, FERET, M2VTS & XM2VTS
Subset of AT&T’s Face Image Database Face (frontal): Image Examples database: ORL, FERET, M2VTS & XM2VTS

60 Face detection & segmentation
Reference: Detecting Faces in Images: A Survey M.-H. Yang at al. IEEE t-PAMI, Vol. 21, No. 1, Jan. 2002 Face detection & segmentation

61 Face: frontal face recognition
Two successful classes of algorithms According to the last round of NIST evaluations, current best solutions are derived either from Eigenfaces or Elastic Graph Matching approaches. Projection-based approaches: Eigenfaces → Fisherfaces. Deformable models: Elastic Graph Matching (EGM) → Elastic Bunch Graph Matching.

62 Eigenfaces (eigeneyes, eigenmouths, eigenvoices, eigenears, etc.)
Reference: Face Recognition Using Eigenfaces M. Turk & A. Pentland IEEE 1991 Eigenfaces (eigeneyes, eigenmouths, eigenvoices, eigenears, etc.)

63 Reference: Face Recognition Using Eigenfaces
M. Turk & A. Pentland IEEE 1991 Eigenfaces (Originally designed for compression, not recognition) Eigenspace can be built using the clients (higher performances but less flexible) or not I(x,y) can be considered as a two-dimensional NxN array of pixels (if N=256; can be seen as a point in a 65,536 dimensional space) “face space” The space of variation between photographs of human faces with the same orientation and scale lit in the same way can be described by a relatively low dimensional subspace. Individual face image ≈ linear combination of a small number of face components I1, I2 … IN: set of reference or training faces (eigenface 0)

64 Reference: Face Recognition Using Eigenfaces
M. Turk & A. Pentland IEEE 1991 Eigenfaces Each face differs from the other faces by the vector Di = Ii – E0. Covariance matrix C: Eigenvectors of C (variation between face images) → Eigenfaces Ek Dimensionality Reduction Technique (DRT) Principal Component Analysis (PCA) → eigenvectors ordered by the magnitude of their contribution to the variation between the training images Extract the R eigenvalues; Order them from largest to smallest, 1, 2, …r Order corresponding eigenvectors E1, E2, …Er → « principal components » Weighted sums of a small collection of characteristic images

65 Eigenfaces PCA in a 2-D space Euclidean distances between
Reference: Face Recognition Using Eigenfaces M. Turk & A. Pentland IEEE 1991 Eigenfaces PCA in a 2-D space Euclidean distances between the K coordinates representing the new face and each of the K-dimensional vectors representing the stored faces, → the stored image yielding the smallest distance

66 Eigenfaces vs. Fisherfaces
Eigenfaces [Kirby & Sirovich, Turk & Pentland]: no distinction between inter- and intra-class variabilities Average face (eigenface 0) and first four eigenfaces Fisherfaces [Belhumeur, Hespanha & Kriegman]: - discriminative approach: find a sub-space which maximizes the ratio of inter-class and intra-class variability same intra-class variability for all classes Average face (Fisherface 0) and first four Fisherfaces

67 EGM Elastic Graph Matching
Distortion Invariant Object Recognition In the Dynamic Link Architecture M. Lades et al. IEEE Trans. On Computers, Vol. 42, no Courtesy of Thessaloniki Univ. EGM Elastic Graph Matching (a) (b) (c) (a) Model grid for person A (1 feature vector / node) (b) Best grid for test person A after elastic graph matching with the model grid. (c) Best grid for test person B after elastic graph matching with the model grid for person A. Vertex labels (local mappings costs) Edge labels (local distortions costs) λ controls the rigidity of the image graph

68 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, dental)

69 Profile distance and angle between fiducial points Source:

70 Dynamic Video Biometrics
Higher potential of video w.r.t. still images More clues (Abundant data) Face / Facial feature tracking New opportunities Visual speech data → correlation between speech and lip motion) Dynamic facial Expression → behavior (not physical only) Shape/Structure from motion Useful for covert surveillance (but non-cooperative with low resolution)

71 3-D Faces, range data Advantages:
- Face Recognition using range images B. Achermann et al., VSMM 97. - Face Identification by Fitting a 3D Morphable Model using Linear Shape and Texture Error Functions, S. Romdhani et al., ECCV 2002 - Face Recognition based on depth maps and surface curvature G. Gordon, SPIE Proc., vol. 1570, 1991. 3-D Faces, range data Advantages: Access to the depth information (i.e. shape) Pose and light conditions may be compensated Higher robustness (e.g. the system cannot be trapped by an impostor using a 2D picture of someone else) Disadvantages: Acquisition process is slow and highly expensive, e.g. 3-D Scanner, 2 calibrated video cameras Cooperation of the users is required Little literature available on the topic (novel facial biometrics) Some published works… Extension of existing algorithms from 2-D to 3-D (e.g. eigenfaces) Adaptation of a generic 3-D Deformable Model to 2-D images of users to provide a set of parameters associated with a person Segmentation of range data into 4 surface regions; Normalization based on the location of eyes, nose and mouth; Distance computed from the volume between surfaces

72 Facial Thermogram (IR imaging)
- Thermal pattern recoginition systems faces security challenges head on M. Lawlor, Signal Magazine Nov. 97 - Comparison of visible and infra-red imagery for face recognition J. Wilder et al., Int. Conf. On Automatic Face and Gesture Recognition, Oct. 96 Facial Thermogram (IR imaging) The facial heat emission patterns can used to characterize a person Patterns depend on 9 factors including: Location of major blood vessels Skeleton thickness Amount of tissue Muscle Fat Advantages: Unique (even for identical twins) Stable over time Cannot be altered through plastic surgery Independent of the lighting conditions Disadvantages: IR imagery depend on the temperature Opaque to glass Source: Other Biometric Techniques Chapter 10. D. Baik and I. Kim

73 Modality by modality Signature Voice Hand Fingerprint Face Frontal
Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, dental)

74 Iris How Iris Recognition Works
J. Daugmann Iris

75 Iris 4 steps Acquisition ( < 1 meter)
How Iris Recognition Works J. Daugmann Iris 4 steps Acquisition ( < 1 meter) Find iris in the image (edge detection) 3. Features extraction: - Local regions of an iris are projected onto quadrature 2D Gabor wavelets, generating complex-valued coefficients whose real and imaginary parts specify the coordinates of a phasor in the complex plane - The angle of each phasor is quantized to one of the four quadrants, setting two bits of phase information - This process is repeated all across the iris with many wavelet sizes, frequencies & orientations → the Iris-Code (1024 phase bits are computed) 4. Verification

76 Coutesy of J. Leroux des Jardins
Retina Unique Number/Pattern of blood vessels, that emanate from the optic nerve and disperse throughout the retina Relative angle w.r.t. optical nerve Bifurcations No two retinas are the same even in identical twins Vascular pattern does not change over the course of life Glasses, contact lenses, existing medical conditions (e.g. cataracts) do not interfere At the moment, identification based on retina is used for animals (bovines) ‘Uncomfortable’ acquisition Eye has to fix a lighting point Projected lighting source on the center of the optical nerve Light is absorbed by red vessels but reflected by retina tissues

77 Retina Extracting Intensity Profiles Performing Scan
Source: Retina Extracting Intensity Profiles Performing Scan Locating Blood Vessels Generating Circular Bar Code

78 Ear Algo. Eigenears Advantages of ears over faces Feature points
Sources On the use of Outer Ear Images for Personal Identification in Security Applications B. Moreno et al., IEEE 1999. Ear Biometrics for Machine Vision M. Burge and W. Burger Ear Advantages of ears over faces Uniform distribution of colors Reduced surface Less variability vs. pose/expressions, Shape and appearance fixes. Passive identification (≠ fingerprints) Algo. Eigenears Feature points

79 Iannarelli’s Ear Biometrics
Source: Ear Biometrics for Machine Vision M. Burge and W. Burger Iannarelli’s Ear Biometrics Iannarelli System (1949) is based on 12 measurements. (External anatomy) 1 Helix Rim, 2 Lobule, 3 Antihelix, 4 Concha, 5 Tragus, 6 Antigrus 7 Crus of Helix 8 Triangular Fossa 9 Incisure Intertragica Distance between each of the numbered areas Segmented outer ear Segmented inner ear

80 Eigenears Sources: - On the use of Outer Ear Images
For Personal Identification in Security Applications B. Moreno et al., IEEE 1999. Eigenears

81 « Liveness » and countermeasure
Impostors may use a fake biometric, Photography of a face Recorded voice Plaster hand etc. Countermeasure: To use a « liveness » test to check the presence of a “real” biometric, e.g. cardiac activity, heart rate

82 Diverse facets of Multimodal
Source: Ph. D., Fingerprint classification and matching using a filterbank, S. Prabhakar Diverse facets of Multimodal

83 Multimodality & fusion e.g. Some possible scenarios in Faces
By default By default Visible Iris If needed only less comfortable but more accurate IR If case of darkness profile By default Ear frontal profile Fusion If needed only less comfortable but more accurate In case of different shots available

84 Fusion At 3 possible levels: Abstract level
Person Identification Using Multiple Cues R. Brunelli & D. Falavigna IEEE T-PAMI, Vol. 17, no. 10, pp , Oct. 95 Fusion At 3 possible levels: Abstract level → Output of each module is a list of labels without any confidence information, Identification: ID of the person Verification: binary response Rank level Output of each module is a set of possible labels ranked by decreasing confidence values Measurement level A measure of confidence is associated with each label

85 Fusion {wi} i=1..N the set of possible classes {xj} j=1..M
Person Identification Using Multiple Cues J. Kittler et al. IEEE T-PAMI, Vol. 20, no. 3, pp , 98. Fusion {wi} i=1..N the set of possible classes Identification: Number of persons present inside the database; Verification: Authentic and Impostor. {xj} j=1..M the set of biometrics Abstract: Vote based on majority Ranks: Maximum, minimum and median Scores: Averaging & Weighed averaging

86 (hard/soft) Fusion: AND/OR
A Tutorial on Support Vector Machines for Pattern Recognition C. Burges (hard/soft) Fusion: AND/OR Operator AND Operator OR Arithmetic Operator (mean) Score #2 Score #1 Accepted User Advanced fusion: SVM (Support Vector Machine)

87 Example: Face + Fingerprint
Integrating Faces and Fingerprints for Personal Identification L. Hong & A. Jain IEEE T-PAMI, Vol. 20, No.12, pp , 1998. Example: Face + Fingerprint Goal: To overcome the limitations of both systems i- Pre-selection of N persons using Face Recognition (top 5 matches) ii- Fingerprint Verification only performed on pre-selected persons Reminder Face recognition is fast but not reliable while fingerprint verification is reliable but inefficient in database retrieval Reported results FAR FRR face fingerprint integration 1% 15.8% 3.9% 1.8% 0.01% 61.2% % (0.9 sec.) (3.2 sec.) (4.1 sec.)

88 Example: Frontal + Profile + Speech
KITTLER et al. « On Combining Classifiers » IEEE T-PAMI, Vol. 20, No. 3, March 98 Example: Frontal + Profile + Speech Method EER (%) Frontal 12.2 Profile 8.5 Speech 1.4 Sum 0.7 Product Maximum Median 1.2 Minimum 4.5

89 Example: Face + Speech Supervisor FA(%) FR(%) TE(%) Face 3.6 7.4 11.0
- B. Duc et al., « Fusion of audio and video information for multimodal person authentication » Pattern recognition Letters 18 (1997) - S. Ben-Yacoub, « Multi-Modal Data Fusion for Person Authentication using SVM » IDIAP RR 98-07 Example: Face + Speech Supervisor FA(%) FR(%) TE(%) Face (EGM) 3.6 7.4 11.0 Speech (Text-dependent) 6.7 0.0 Arithmetic mean 1.2 2.1 3.3 Bayesian conciliation 0.54 Linear-SVM 0.07 Polynomial-SVM 0.21 RBF-SVM 0.12 MLP-SVM 0.15

90 Example: Faces N. Experts DT BKS 2 97.02 97.92 3 97.51 98.15 4 96.46
J. Kittler et al., « Enhancing the performance of personal identity authentication systems by fusion of face verification experts” Example: Faces Expert Evaluation Test Test Set UniS-gdm 97.83 97.15 UniS-noc 96.46 96.90 Unis-eucl 88.80 91.15 UCL-lda 96.11 96.68 UCL-pm1 94.43 95.34 UCL-pm2 95.29 96.14 N. Experts DT BKS 2 97.02 97.92 3 97.51 98.15 4 96.46 98.21 5 96.12 98.34 6 95.81 98.43 DATABASE: XM2VTS DT: decision Templates BKS: Behavior Knowledge Space ASR: Average Success Rate of client acceptance and impostor rejection on the Evaluation set (top), on the Test set (down). BKS >> DT By adding experts, the performance of the multimodal system will not be degraded. For a sufficient number of experts, optimal configuration selected on the evaluation set, also a posteriori optimal on the test set. N. Experts DT BKS 2 A posteriori 87.79 97.39 3 97.7 97.78 4 97.28 97.8 97.68 98.05 5 96.6 98.08 98.19 6 96.24 96.33 98.49

91 Note on Biometrics vs. Data Hiding
- Information Hiding, S. Katzenbeisser and F. Petitcolas, Eds Artech House - Hide a Face in a Fingerprint Image Jain et al. Note on Biometrics vs. Data Hiding Goal: To combine watermarking and biometrics, for example by hiding the minutia of a passport’s owner inside his/her id picture present on the document → In order to enforce the security of documents (harder to falsify thanks to cross-security) Embedding of eigenface data (associated with a face image) in a fingerprint image (cover) of a given person Problem: Relevant characteristics of host image must remain unchanged (e.g. location and nature of minutia), i.e. the same map of minutia must be obtained either from the original fingerprint image or from its watermarked version Basic recall on watermarking The aim of digital watermarking is to include a subliminal information (i.e. imperceptible) in a multimedia document for security purpose (e.g. copyright) It would then be possible to recover the embedded message using a secret key, at any time, even if the document was altered Trade-off: capacity, visibility and robustness

92 Outline (3/3) Applications, Standards and Evaluation
Main application areas Biometrics and privacy Important criteria to deploy multimodal authentication systems Biometric standards Multimodal databases Best practices in testing biometric systems Examples of multimodal user authentication systems Perspectives and future challenges Demonstrations Forthcoming events Bibliography

93 Main application areas
S. Nanavati, M. Thieme, and R. Nanavati: Biometrics, Identity Verification in a Networked World, Wiley Computer Publishing, 2002 Main application areas Biometric applications can be classified as follows: Forensics: criminal investigation and prison security Retail/ATM/point of sale Civilian applications: electronic commerce and electronic banking (e.g. Visa cards) Information system/ computer network security: user authentication, remote access to databases Physical access/time and attendance (e.g. cellular phone, workstations, door entrance, automobile) Citizen identification (e.g. interaction with government agencies) Surveillance (identify or verify the identity of individuals present in a given space/area, e.g. airport) Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database. Different Markets require different biometric levels of security

94 Biometrics in Airports (USA)
First Installed, Trial Population Staff, travellers Supplier Charlotte/Douglas Iris US Airways employees entering secure ares Chicago Finger Cargo truck drivers who deliver to the airport Identix Fresno Face Idaho Hand Trial Reco. Systems Inc. JFK Lincoln Logan Nov. 02 Trier Technologies LAX Manchester Miami Mineta San Jose Portland Salt Lake City SF Springfield St. Petersburg St. Petersburg- Clearwater

95 Biometrics in Airports (EU, Others)
First Installed, Trial Population Staff, passengers Supplier Country Charles de Gaulle Trial France Orly Berlin 1-1 Face reco. Jan. 03 ZN Vision Techn. AG Germany Frankfurt Iris Amsterdam Schiphol Oct. 01 Joh. Enschede BV Netherlands London Heathrow Aug. 02 UK Keflavik 1-n Face Jun. 01 Crowd Identix Inc., Visionics FaceIt Software Iceland Ben Gurion 1-1 Hand 1998 Passenger moving through custom Reco. Syst. Inc. Israel Narita Face and Iris NTT DoCoMo King Abdul Aziz Feb. 02 Saudi Arabia Singapore 1-1 Finger Thunder Bay Face NExus AcSys Canada Toronto Hand Vancouver Finger and Hand

96 In practice… INPASS program Enrollment procedure about 30 min.
Inpass benefits: insignificant (?) Palm Beach Airport Airport Face scanner failed… Error rate of 53% (455 success out of 958 attempts) Vendor argues system was not used properly (i.e. incorrect lighting)

97 Vertical Markets Law enforcement Government sector Financial sector
S. Nanavati, M. Thieme, and R. Nanavati: Biometrics, Identity Verification in a Networked World, Wiley Computer Publishing, 2002 Vertical Markets Law enforcement Government sector Financial sector Healthcare Travel and immigration Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database. However, biometric deployments in these markets are not always very different

98 Fingerprint technology is the only biometric that has been implemented
Published September 2001, International Biometrics Group Fingerprint technology is the only biometric that has been implemented within a large scale (IAFIS)

99 Smart Cards and Biometrics
A Smart Card is a portable secure storage (can contain computer chip) Smart Cards are excellent support for privacy Smart Cards can verify the biometric identity Smart cards can update the biometric template Smart Cards prevent the need for a big centralized database (support privacy) Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database.

100 Privacy Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database. “your privacy is important to us. How much would you pay to preserve it?” The Wall Street Journal, November 14th, 2001

101 Privacy Definition by Alan Westin
“Privacy is the claim of individuals, groups, or institutions to determine for themselves, when, how and to what extent information about them is communicated to others” Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database.

102 Privacy General requirements Technical requirements
Use biometric data in accordance with privacy needs Technical requirements Do not store biometric raw data in a database Do not use the biometric data outside the specified purpose Do not collect unnecessary personal data Use adequate algorithms for the calculation of biometric signatures Being a rapidly evolving technology, Biometric has been widely used in forensics. It is also seriously consider for adoption in a spectrum of civilian applications Such as e-commerce and e-banking And biometric system can also play an important role in information system/ computer network sercurity such as user authentication and remote access to database.

103 Privacy Concerns Factors affecting privacy High Very High
Amount of data Low High Sensitivity of the data Privacy is becoming an increasingly important issue especially in large systems

104 Important criteria to deploy multimodal user authentication systems
Enrollment User acceptance Privacy/Civil liberties ID management/ID theft Database management/Integrity Political and cultural environment System complexity Cost Lastly, to summarize my presentation, With advancement in technology, we definitely see an increasing involvement of biometrics in the automatic identification system. However, when implementing biometric-based identification system, the various factors like, accuracy, speed and storage requirement must be taken into consideration. And the applicability of each biometric technique depends heavily on the application domain.

105 Important criteria to deploy multimodal user authentication systems
Before introducing this technology to customers, a number of fundamental questions about consumer understanding, expectations and concerns need to be answered. The answers to these questions will help the development of solutions that are accepted by the consumers Understanding consumer attitudes towards this technology is essential to business managers as they study the ROI and design the user interface“ Lastly, to summarize my presentation, With advancement in technology, we definitely see an increasing involvement of biometrics in the automatic identification system. However, when implementing biometric-based identification system, the various factors like, accuracy, speed and storage requirement must be taken into consideration. And the applicability of each biometric technique depends heavily on the application domain.

106 Biometric standards What are standards and what are they good for?
Standards (a general set of rules to which all complying procedures, products or research must adhere) offer a myriad of benefits. They reduce differences between products and promote an aura of stability, maturity and quality to both consumers and potential investors (http://www.biometrics.org/html/standards.html) Who establishes them? Standard Bodies, e.g. American National Standards Institute (ANSI) International Standards Organization (ISO) National Institute of Standards and Technology (NIST) What biometric standards are available? Several already exist (http://www.biometrics.org/html/standards.html)

107 Biometric standards BioAPI
BioAPI (March 2002: BioAPI Version 1.1 was approved as ANSI/INCITS ) Biometric Consortium took the lead to merge the efforts of several vendors under BioAPI with strong support from NIST Defines a generic way of interfacing to a broad range of biometric technologies Founded in 1988 by Compaq, Microsoft, Novell, IBM, Identicator, Miros. Merged with other efforts in 1999 Purpose: Development of a standard biometric API to bring platform and device independence to application developers, integrators and end-users Benefits Easy substitution of biometric technologies Use of biometric technologies across multiple applications Easy integration of multiple biometrics using the same interface Rapid application development – increased competition (tends to lower costs) Application compatibility / interoperability

108 Biometric standards CBEFF Common Biometric Exchange File Format
CBEFF (NISTIR 6529, Jan.3, 2001) Common Biometric Exchange File Format Describes a set of data elements necessary to support biometric technologies in a common way Features Facilitate biometric data interchange between different system components or systems Promotes interoperability of biometric-based application programs and systems Provides forward compatibility for technology improvements Simplifies the hardware and software integration process

109 Multimodal databases There are very few They are costly to record
Many parameters need to be taken into account (e.g. for one modality such as voice: speaker population, environment, age, text to say, etc.) Realistic data (from real-world application is very difficult to collect and it is generally difficult to control the different factors) Nature of the imposters is an issue, etc. → One possibility could consist in building a multimodal database by artificially combining different unimodal ones. There is no correlation between most of biometrics Lastly, to summarize my presentation, With advancement in technology, we definitely see an increasing involvement of biometrics in the automatic identification system. However, when implementing biometric-based identification system, the various factors like, accuracy, speed and storage requirement must be taken into consideration. And the applicability of each biometric technique depends heavily on the application domain.

110 Multimodal databases: XM2VTS
The database was recorded within the M2VTS project (Multimodal Verification for Teleservices and Security applications), a part of the EU ACTS program, which deals with access control by the use of multimodal identification of human faces The goal of using a multimodal recognition scheme is to improve the recognition efficiency by combining single modalities, namely face and voice features The XM2VTSDB contains four recordings of 295 subjects taken over a period of four months. Each recording contains a speaking head shot and a rotating head shot. Sets of data taken from this database are available including high quality colour images, 32 KHz 16-bit sound files, video sequences and a 3D Model

111 Multimodal databases: BT-DAVID
The BT-DAVID (Digital Audio-Visual Integrated Database) audio-visual database is designed for undertaking research in speech or person recognition, as well as synthesis and communication of audio-visual signals Expected areas of application are: automatic speech/person recognition for terminal interfaces or automated transaction machines, voice control of video-conferencing resources, speech-assisted video coding, and synthesis of talking heads The BT-DAVID database contains full-motion video, showing a full-face and a profile view of talking subjects, together with the associated synchronous sound. BT-DAVID includes audio-visual material from more than 100 subjects including 30 clients recorded on 5 sessions spaced over several months The BT-DAVID database was compiled by the Speech and Image Research Group at University of Wales Swansea under a contract to BT Labs

112 Best practices in testing biometric systems
Fact: It is still very difficult to predict real-world error rates Besides performance (which includes both false positive and false negative decisions along with failure to enroll and failure to acquire rates across the test population) the following criteria should also be taken into account Reliability, availability and maintainability Vulnerability Security User acceptance Human factors Cost/benefit Privacy regulation compliance

113 Best practices in testing biometric systems
A.J. Mansfield and J.L. Wayman: Best Practices in Texting and Reporting Performance of Biometric Devices, NPL Report CMSC 14/02 P.J. Phillips, A. Martin, C.L. Wilson, and M. Przybocki: An introduction to evaluating biometric systems. Computer, (Feb. 2000), 56-63 Best practices in testing biometric systems Biometric technical performance testing can be of three types: Technology (database evaluation) Scenario (overall system performance in a prototype or simulated application. Could be a combination of offline and online testings) Operational evaluation (performance of a complete system in a specific application environment with a specific target population. In general not repeatable) Each type of test requires a different protocol and produces different results The nature of impostors is an important part of the testing of biometric systems

114 K. Jain, L. Hong, and Y. Kulkarni: A Multimodal Biometric System Using Fingerprint,
Face, and Speech, Technical Report MSU-CPS-98-32, Department of Computer Science, Michigan State University Example Acceptance rate (%) False acceptance rate (%)

115 Example BioID SDK by HumanScan (Germany)
R.W. Frischholz and U. Dieckmann: “BioID: A Multimodal Biometric Identification System”, IEEE Computer, vol. 33, no. 2, pp , February 2000 Example BioID SDK by HumanScan (Germany) BioID SDK offers multimodal biometrics in the form of a software development kit BioID SDK offers three biometrics: Face recognition Voice recognition Lip movement recognition Since BioID uses true multimodality, the preferred way of using it is by using all of the three biometrics together. But BioID can be easily configured (e.g. using the Control Panel) to use any of the above three biometrics alone or in any combination Basic features include: User enrollment wizard User recognition (verification or identification) User template and authorization management Enrollment management Template storage to database, local PC or Smart Card Support BioAPI

116 Perspectives and future challenges
Multimodal biometrics will play vital roles in the next generation of automatic identification systems Future challenges in multimodal biometric systems Accuracy is still an issue for most of existing biometrics Feature extraction Dealing with dynamic information using a small amount of training data How to combine information (fusion) and make use of the strengths of each modality Collection of a multimodal and realistic database (most of the existing databases are unimodal) Integrating higher level of information (e.g. for speech, prosodic modeling, word/phrase usage) Scalability Establishment of common standards along the lines of GSM in the mobile world Dealing with privacy concerns Ease of use and development

117 Demonstrations Iris recognition
Speaker and Fingerprint recognition for door entrance system

118 Forthcoming events IEE PROCEEDINGS VISION, IMAGE AND SIGNAL PROCESSING, Special Issue on BIOMETRICS ON THE INTERNET, Aladdin Ariyaeeinia, University of Hertfordshire, UK, Guest Editor (http://www.iee.org/Publish/Support/Auth/Authproc.cfm) Multimodal User Authentication workshop, Santa Barbara, CA, U.S.A. December 2003 (http://mmua.cs.ucsb.edu ) International Conference on Biometric Authentication, Hong Kong, January 2004 (http://www4.comp.polyu.edu.hk/~icba/) EURASIP, Applied Signal Processing, Special Issue on Biometric Signal Processing (4th quarter 2003) (http://asp.hindawi.com)

119 Acknowledgments (for inputs, fruitful discussions and help)
Institut Eurécom (Florent Perronnin) Panasonic Speech Technology Laboratory University of Thessaloniki

120 Bibliography BOOKS S. Nanavati, M. Thieme, and R. Nanavati: “Biometrics, Identity Verification in a Networked World”, Wiley Computer Publishing, 2002. J. Ashbourn: “Biometrics, Advanced Identity Verification, The Complete Guide”, Springer, 2000. L.C. Jain, U. Halici, I. Hayashi, S.B. Lee, and S. Tsutsui, editors: “Intelligent Biometric Techniques in Fingerprint and Face Recognition”, The CRC Press International Series on Computational Intelligence, 1999. A. Jain, R. Bolle, and S. Pankanti, editors: “Biometrics, Personal Identification in Networked Society”, Kluwer Academic Publishers, 1998.

121 Bibliography INTRODUCTION OVERVIEW COURSE (slides)
Y. W. Yun The ‘123’ of Biometric Technology. OVERVIEW J.-L. Dugelay, J.-C. Junqua, C. Kotropoulos, R. Kuhn, F. Perronnin, and I. Pitas: “Recent Advances in Biometric Person Authentication”, ICASSP 2002, pp. IV 4060-IV 4063. COURSE (slides) J. Wayman (San José State University) Biometrics & How they Work.

122 Bibliography MULTIMODAL
A.J. Mansfield and J.L. Wayman: “Best Practices in Texting and Reporting Performance of Biometric Devices”, NPL Report CMSC 14/02. K. Jain, L. Hong, and Y. Kulkarni: “A Multimodal Biometric System Using Fingerprint, Face, and Speech”, Technical Report MSU-CPS-98-32, Department of Computer Science, Michigan State University. L. Hong and A. Jain: “Integrating Faces and Fingerprints for Personal Identification”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp , December 1998.

123 Bibliography FUSION R. Brunelli and D. Falavigna: “Person Identification Using Multiple Cues”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 10, pp , October 1995. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas: “On Combining Classifiers”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp , March 1998. R.W. Frischholz and U. Dieckmann: “BioID: A Multimodal Biometric Identification System”, IEEE Computer, vol. 33, no. 2, pp , February 2000. V. Chatzis, A.G. Bors, and I. Pitas: “Multimodal Decision-Level Fusion for Person Authentication”, IEEE Trans. Systems, Man and Cybernetics, Part A, vol. 29, pp , November 1999. C. Burges: “A Tutorial on Support Vector Machines For Pattern Recognition”.

124 Bibliography LIPREADING – audio/video
B. Duc, E.S. Bigun, J. Bigun, G. Maitre, and S. Fischer: “Fusion of Audio and Video Information for Multi-Modal Person Authentication”, Pattern Recognition Letters, vol. 18, pp , 1997. S. Ben-Yacoub, Y. Abdeljaoued, and E. Mayoraz: “Fusion of Face and Speech Data for Person Identity Verification”, IEEE Trans. Neural Networks, vol. 10, no. 5, pp , September 1999.

125 Bibliography SPEECH D.A. Reynolds and L.P. Heck: “Speaker Verification: From Research to Reality”, Tutorial, ICASSP, Salt Lake City, Utah, May 7, 2001. G. Doddington: “Speaker Recognition Based on Idiolectal Differences between Speakers”, Eurospeech 2001, V. 4, pp , Aalborg, Denmark, Sept. 3-7, 2001. O. Thyes, R. Kuhn, P. Nguyen, and J-C. Junqua: “Speaker Identification and Verification Using Eigenvoices”, ICSLP-2000, V. 2, pp , Beijing China, Oct

126 Bibliography FACE IRIS RETINA EAR
R. Chellappa, C.L. Wilson, and S. Sirohey: “Human and Machine Recognition of Faces: A Survey”, Proceedings of the IEEE, vol. 83, no. 5, pp , May 1995. M. Turk and A. Pentland: “Eigenfaces for Recognition”, J. Cognitive Neuroscience, vol. 3, no. 1, pp , 1991. S. Pigeon and L. Vandendorpe: “Image-based Multi-Modal Face Authentication”, Signal Processing, vol. 69, pp , August 1998. IRIS J. Daugman: “Recognizing Persons by their Iris Patterns”, in Biometrics, Personal Identification in Networked Society, pp , A. Jain, R. Bolle and S. Pankanti, editors, Kluwer Academic Publishers, 1998. RETINA EAR B. Moreno et al.: « On the Use of Outer Ear Images for Personal Identification in Security Applications » 1999 IEEE. M. Burge and W. Burge: « Ear Biometrics for Machine Vision ».

127 Bibliography FINGERPRINT
R. Adhami and P. Meenen: “Fingerprinting for Security”, IEEE Potentials, Vol. 20, no. 3, pp , Aug.-Sept A. Jain and S. Pankanti: “Automated Fingerprint Identification and Imaging Systems”, Advances in Fingerprint Technology, 2nd Ed., Elsevier Science, New York, 2001.

128 Bibliography HAND GEOMETRY SIGNATURE
R. Sanchez-Reillonet et al. : “Biometric Identification through Hand Geometry Measurements”, IEEE PAMI Vol. 22, No. 10, 2000. SIGNATURE A. Jain et al. : “On-line Fingerprint Verification”, IEEE T-PAMI Vol. 19, No. 4, April 97.

129 Bibliography DATABASE EVALUATION
S. Pigeon and L.Vandendorpe: “The M2VTS multimodal Face Database”, Lecture Notes in Computer Science: Audio- and Video-Based Biometric Person Authentication (J. Bigun, G. Chollet, and G. Borgefors Eds.), vol. 1206, pp , 1997. K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre: “XM2VTSDB: The Extended M2VTS Database”, in Proc. 2nd Int. Conf. on Audio- and Video-Based Biometric Person Authentication, March 1999. EVALUATION P. J. Philips, et al.: « The Feret evaluation methodology for face-recognition algorithms », IEEE T-PAMI, Vol. 22, No. 10, Oct

130 End. jcj@research.panasonic.com (Jean-Claude JUNQUA)
(Jean-Luc DUGELAY)


Download ppt "Multimodal User Authentication: From Theory to Practice"

Similar presentations


Ads by Google