Biometrics and Sensors Venu Govindaraju CUBS, University at Buffalo govind@buffalo.edu

Organization Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Research at UB Multimodal Identification Biometrics Sensors Fingerprint Signature Hand Geometry Sensors Materials and Light Sources Analog VLSI and Optical Detectors Packaging and Reliability Engineering

Applications And Scope of Biometrics Technologies Horizontal Applications Key Vertical Markets Fingerprint Civil ID Government Sector Facial Recognition Surveillance and Screening Travel and Transportation Iris Scan PC / Network Access Financial Sector Middleware Retail / ATM / Point of Sale Health Care AFIS eCommerce / Telephony Law Enforcement Voice Scan Physical Access / Time and Attendance   Hand Geometry Criminal ID Signature Verification Keystroke Dynamics

Scope of Research In Biometrics State of the art Research Problems Fingerprint 0.15% FRR at 1% FAR (FVC 2002) Fingerprint Enhancement Partial fingerprint matching Face Recognition 10% FRR at 1% FAR (FRVT 2002) Improving accuracy Face alignment variation Handling lighting variations Hand Geometry 4% FRR at 0% FAR (Transport Security Adminstration Tests) Developing reliable models Identification problem Signature Verification 1.5% (IBM Israel) Developing offline verification systems Handling skillful forgeries Chemical Biometrics No open testing done yet Development of sensors Materials research

Biometrics Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Conventional Security Measures Token Based Smart cards Swipe cards Knowledge Based Username/password PIN Disadvantages of Conventional Measures Tokens can be lost or misused Passwords can be forgotten Multiple tokens and passwords difficult to manage Conventional security systems rely on PINs, passwords and other token or key based methods for authentication and identification of users. Though these systems are easy to use, they are insecure as the tokens can be lost, stolen or used by more than one person. With each service requiring different form and means of identification, the multiplicity of authentication schemes becomes difficult to manage.  

Biometrics Definition Examples Physical Biometrics Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits Examples Physical Biometrics Fingerprint, Hand Geometry,Iris,Face Behavioral Biometrics Handwriting, Signature, Speech, Gait Chemical Biometrics DNA, blood-glucose Biometrics offers a promising solution for reliable and uniform identification and verification of an individual. Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits. Physical biometrics rely on physiological features such as fingerprints, hand geometry, iris pattern, facial features etc. for identity verification. Behavioral biometrics depends upon behavioral features such as speech patterns, handwriting, signature, walking gait etc. for authentication. These traits are unique to an individual and hence cannot be misused, lost or stolen. Biometrics are based on established scientific principles as a basis for authentication.

Fingerprint Verification Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Fingerprint Verification Fingerprints can be classified based on the ridge flow pattern The corrugated surface of the fingerprint is made up of ridges and valleys cover that the entire palmer surface of the hand. The flow pattern of these ridges and valleys are unique to each individual. These patterns that are used for identification and authentication. The image below shows the image of a fingerprint along with the distinguishing features on the print.   The flow of the ridges and patterns has been classified into 5 broad classes. This classification is used to catalog the fingerprints and also in authenticating two prints. Henry systems follow an elaborate classification scheme of cataloging and filing forensic prints. Fig1 shows the different classes of ridge flows. This methods cannot be used to distinguish between two fingerprints. Fig 2 shows the distinguishing features on the fingerprint. These features are discontinuities or anomalies in the normal flow of ridges on the surface of the finger. These features are termed as minutiae (small details). There are eighteen different types of minutiae. Fig1 shows the most commonly encountered ones and their names. Fig 2 shows a thumbprint captured on paper. These are typically the kind of images that forensic AFIS (Automatic Fingerprint Identification Systems) have to deal with. The quality of the print not only deteriorates during capture but also during storage and hence AFIS systems are more sophisticated than their biometric counterparts. Fingerprints can be distinguished based on the ridge characteristics

Fingerprint Image Enhancement Preprocessing Enhancement Feature Extraction Matching High contrast print Typical dry print The performance of any fingerprint recognizer depends upon the quality of the fingerprint image processed. AFIS systems have achieved high performances in case of reasonably good prints. However there is not yet satisfactory methods to deal with bad quality fingerprints as often encountered in forensic and biometric applications. Effective methodologies for cleaning the valleys between the ridge contours are lacking in current fingerprint recognition systems. The figures show some prints acquired under different conditions. It can be seen that the minutiae features cannot be easily extracted from the original gray scale image. Therefore some form on enhancement is required before processing the fingerprint image further. Low contrast print Typical Wet Print

Projection Based Method Traditional Approach Preprocessing Enhancement Feature Extraction Matching Local Orientation (x,y) Gradient Method Enhancement Frequency/Spatial The fingerprint can be seen as an oriented texture. This property of the fingeprints can be used while enhancing the fingerprint image. Traditional image processing methods such as gaussian smoothening, or low pass filtering cannot be used as they tend to bridge the gaps between the ridges. The ridges have to be enhanced only in a direction parallel to their orientation. There are spatial and frequency domain methods that perform such kind of filtering. Spatial domain methods are based on anisotropic filters aligned in the ridge direction(??) or are based on Gabor filtering(Anil Jain et al). Frequency domain methods utilizing FFT (Grother et al, Monroe et al) are also present and are more successful. The enhancement depends upon the accurate estimation of the local ridge orientation and local ridge spacing within the image. Presently the local orientation and the local ridge spacing are obtained through separate algorithms. The enhancement is performed using this information. This approach requires multiple passes and is computationally expensive. We propose a unified approach to fingerprint image enhancement using FFT analysis. The proposed method extracts the local orientation and ridge spacing information in one pass and at the same time performs the enhancement. Local Ridge Spacing F(x,y) Projection Based Method

Fourier Analysis Approach Preprocessing Enhancement Feature Extraction Matching Energy Map E(x,y) FFT Analysis Orientation Map O(x,y) FFT Enhancement The proposed methods uses FFT based frequency domain analysis to estimate the ridge spacing and orientation in the image. The enhancement is also done in the frequency domain. The analysis yields the energy map, orientation map and ridge spacing map corresponding to the image. The energy map indicates the presence of ridges and their contrast within the image. The energy map can be successfully used to segment the fingerprint image from the background. The orientation map provides the orientation of the ridges in a local neighborhood. This information is used in the enhancement procedure. The ridge spacing map provides the inter ridge distance variation in the image. The ridge spacing can be used in the enhancement procedure to design a suitable band pass filter that allows ridges and eliminates noise and gray level gradients across the image. The following slides show the results of the FFT analysis algorithm. Ridge Spacing Map F(x,y)

Fourier Analysis – Applied to fingerprints Fingerprint ridges can be modeled as an oriented wave Local ridge orientation Local ridge frequency

Fourier Analysis –Energy Map Preprocessing Enhancement Feature Extraction Matching It can be seen that the energy map can be used to easily segment the fingerprint image. The foreground has very high energy (red) and the background has very low energy (blue). Original Image Energy Map Thresholded Map

Fourier Analysis – Frequency Map Preprocessing Enhancement Feature Extraction Matching The ridge spacing map indicates the ridge frequency variation across the fingerprint image. High ridge frequency (yellow) indicates closely spaced ridge. Low ridge frequency indicates (green) indicates widely spaced ridge. Original Image Local Ridge Frequency Map

Fourier Analysis-Orientation Map Preprocessing Enhancement Feature Extraction Matching The ridge orientation map describes the direction of ridge flow across the fingerprint. The orientation map is very important to successfully enhance and binarize the fingerprint image. It can be seen that the FFT analysis method gives accurate ridge flow directions, similar to gradient based approaches. Local Ridge Orientation Map Original Image

FFT Based Enhancement Preprocessing Enhancement Feature Extraction Matching The slide shows the results of the frequency domain enhancement. Presently the enhancement approach suggested by Grother et al is used to perform the enhancement. The frequency and orientation map obtained in the previous section is not included in the filter design. The results will only improve with their inclusion. Original Image Enhanced Image

Common Feature Extraction Methods Preprocessing Enhancement Feature Extraction Matching Thinning-based Method Thinning produces artifacts Shifting of Minutiae coordinates Direct Gray-Scale Extraction Method Difficult to determine location and orientation Binarized Image is noisy. Most feature extraction algorithms described in the literature extract minutiae from a thinned skeleton image that is generated from a binarized fingerprint image. Thinning is a lossy and computationally expensive operation and the accuracy of the output skeletal representation varies for different algorithms. The thinning process also introduces artifacts such as ridge break and bridges that lead to spurious minutiae. To overcome these disadvantages we use a chaincode representation for detection of fine feature points or minutiae. The bifurcation points and endpoints of ridge contours and their orientation are readily extracted using a ridge contour following procedure.

Chaincoded Ridge Following Method Preprocessing Enhancement Feature Extraction Matching The contour is then traced counterclockwise (clockwise for interior contours) and expressed as an array of contour elements (Figure (a)). Each contour elementrepresents a pixel on the contour, contains fields for the x,y coordinates of the pixel, the slope or direction of the contour into the pixel, and auxiliary information such as curvature. The slope convention used by the algorithms described is as shown in Figure (b) above. We consistently trace the ridge contours of the fingerprint images in a counter-clock-wise fashion. When we arrive at a point where we have to make a sharp left turn we mark a candidate for a ridge endingpoint. Fig(i) Similarly when we arrive at a sharp right turn, the turning location marks a bifurcation point Fig(ii). Determination of left or right turn using sign of S(Pin, Pout) = x1y2 –x2y1 Significant turns satisfies the conditions: x1y1 + x2y2 < Threshold The angle theta between vectors Pin and Pout meets the conditions: 90 <= theta <= 180

Minutiae Detection Preprocessing Enhancement Feature Extraction Matching Several points in each turn are detected as potential minutiae candidate One of each group is selected as detected minutiae. Minutiae Orientation is detected by considering the angle subtended by two extreme points on the ridge at the middle point. vector minutiae point middle point between Forward N-pixel point in the contour Backward N-pixel point in the contour

Pruning Detected Minutiae Preprocessing Enhancement Feature Extraction Matching Ending minutiae in the boundary of fingerprint images need to be removed with help of FFT Energy Map Closest minutiae with similar orientation need to be removed

Secondary Features Pure localized feature Derived from minutiae representation Orientation invariant Denote as (r0, r1, δ0, δ1, ) r0, r1: lengths of MN0 and MN1 δ0, δ1: relative minutiae orientation w.r.t. M : angle of N0MN1 Preprocessing Enhancement Feature Extraction Matching

Dynamic Tolerance Areas Preprocessing Enhancement Feature Extraction Matching Tolerance Area is dynamically decided w.r.t. the length of the leg. Longer leg: Tolerates more distortion in length than the angle. Shorter leg: tolerates less distortion in length than the angle. A B O Tolerance areas are the gray areas around the neighborhood minutiae of the querying feature point Mi. If both the neighborhood minutiae of template feature point fall into the tolerance areas of Mi, we say it is a match. Each tolerance area shape is decided by two parameters: the distance and angle thresholds. The threshold for distance is positive proportion to the length of the leg MiN0. The angle threshold is negative proportion to the length of leg. That means if the length of the leg getting shorter the tolerance area is getting wider in angle but narrower in depth. Dynamic Windows Dynamic tolerance

Feature Matching Preprocessing Enhancement Feature Extraction Matching For each triangle, generate a list of candidate matching triangles To recover the rotation between the prints. Find the most probable orientation difference Apply the results of the pruning and match the rest of the points based on the reference points established.

Validation OD=0.7865° Preprocessing Enhancement Feature Extraction Matching OD=0.7865° For each triangle, generate a list of candidate matching triangles To recover the rotation between the prints. Find the most probable orientation difference Apply the results of the pruning and match the rest of the points based on the reference points established.

Minutia Matching Preprocessing Enhancement Feature Extraction Matching For each triangle, generate a list of candidate matching triangles To recover the rotation between the prints. Find the most probable orientation difference Apply the results of the pruning and match the rest of the points based on the reference points established

Fig(b) Paired fingerprints Data Sets Fig(a) Sensors and technology used in acquisition Fig(b) Paired fingerprints Fig(c) Database sets The database consists of fours sets of live captured fingerprint images each of which are acquired using different sensors and technology. The acquisition methods are tabulated in Fig(a). Each database consists of hundred distinct fingers with 8 prints from the same finger. One of the fingerprint pair is shown in Fig(b). The representative prints from each of the database is shown in Fig(c).

Preliminary Results State of the art Min Total Error = 1.16% FRR at 0 FAR = 5.0% Threshold FAR FRR State of the art Min Total Error = 0.19% FRR at 0 FAR = 0.38%

Signature Verification Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Signature Verification Online Signature verification Off line Signature Verification

Preprocessing Preprocessing Make signature invariant to scale, translation and rotation. Preprocessing Template generation Matching (-170)- (-125) (-3.0)- (4.0) mean-std norm. Resampling -1.5-3.5 0-160 Smoothing

Template Generation- Challenges Preprocessing Template generation Matching Extracting features. Usually we can not expect more than 6 genuine signatures for training for each subject. This is unlike handwriting recognition Decide the consistent features. There are over 100 features for signature, such as Width, Height, Duration, Orientation, X positions, Y positions, Speed, Curvature, Pressure, so on.

Matching – Similarity Measure Simple Regression Model Preprocessing Template generation Matching Y = (y1 , y2 , …, yn) X = (x1 , x2 , …, xn) Similarity by R2 : 91% R2= Similarity by R2 : 31%

Traditional Regression approach Preprocessing Template generation Matching Advantages: Invariant to scale and translation. Similarity (Goodness-of-fit) makes sense. Disadvantages: One-one alignment, brittle. One-One alignment Dynamic alignment

Dynamic Regression approach(1) Preprocessing Template generation Matching ( y2 is matched x2, x3, so we extend it to be two points in Y sequence.) Similarity = R2 Where (x1i, y1i, v1i) are points in the sequence And a, b, c are the weights, e.g., 0.5, 0.5, 0.25 DTW warping path in a n-by-m matrix is the path which has min cumulative cost. The unmarked area is the constrain that path is allowed to go.

Offline Signature Verification Shapes can be described using structural or statistical features We use an analytical approach that uses the attributes of structures. Extracting structural features

Attributes of structural features Statistical analysis of the feature attributes

Hidden Markov Models and SFSA Obtaining a stochastic model Outgoing transitional probabilities The occurrence of the structural features can be modeled as a HMM The HMM can be converted to a SFSA by assigning observation and probability to the transitions instead of to the states

Hand Geometry Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Hand Geometry Used where Robustness, Low cost are the concerns. Comparatively less accurate. Combination with other Biometric techniques, increases accuracy. Sufficient for verification where finger print use may infringe on privacy.

Feature Extraction A snapshot of the top and side views of the user’s right hand gives the contours outlining the hand. Features necessary to identify the hand are extracted from these contours. Using simple image processing techniques, the contours of the set of two images of the hand are obtained. Hand-verification is done by correlating these features. Research: New features and algorithms for better discrimination between two hands.

Multimodal Biometrics Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Combination of biometric matchers Combination of the matching results of different biometric features provides higher accuracy. Fingerprint matching Hand geometry matching Signature matching Alice Bob : 26 12 0.31 0.45 5.54 7.81 0.95 0.11 Combination algorithm

Sequential combination of matchers Fingerprint matching Combination algorithm 1 No Desired confidence achieved? Yes Signature matching Combination algorithm 2 Alice Bob : 0.95 0.11 Yes No Desired confidence achieved? Hand geometry matching Combination algorithm 3

Securing Biometric Data Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Securing password information It is impossible to learn the original password given stored hash value of it.

Securing fingerprint information Wish to use similar functions for fingerprint data:

Obstacles in finding hash functions Fingerprint space Hash space f1 h(f1) h f2 h(f2) Since match algorithm will work with the values of hash functions, similar fingerprints should have similar hash values rotation and translation of original image should not have big impact on hash values partial fingerprints should be matched

Sensors and Devices Biometrics and Sensor research at UB Biometrics Fingerprint Verification Securing Biometric Data Signature Verification Hand Geometry Sensors and Devices

Sensors and Biometrics Fingerprint Optical Sensors Capacitive Sensors Thermal Sensors Ultrasound Sensors Signature Digitizer Tablet Digitizer Pen Offline scanning Face Recognition Optical Digital camera Thermal cameras Chemical Biometrics Sensor Arrays Smart Devices (Research at UB)

Stimulator and Support System Sensors Detector System CMOS CCD’s Photodiodes Image Processing Biosurfaces - Biofouling Bioinspired Pattern Recognition Biomimetics – Artificial Vision, Smell. Bioinspired Super Correlator Analyte Sensing Layer Tissues Cells Proteins DNA and RNA Organic and Inorganic Dyes Molecular Imprinting Biosurfaces – Biofouling Immobilization and Stabilization Transduction mechanism Multi-Analyte detection Photonic Bandgap (PBG) Resonators Evanescent Wave Devices (PBG) Stimulator and Support System Light Sources (OLEDs, LEDs, Lasers) Signal Generators Driver Circuits Power Supply Biosurfaces – Biofouling Nano-LEDs Bioinspired Photovoltaics, Biofuel Cells Environmental Testing Low Power Light Sources c) Device b) Enabling Technologies a) Fundamental Knowledge

Stimulator and Support System Sensor Components Stimulator and Support System Sensing Layer Detector System Blocking Filter Output Device

CMOS Integrated Sensor System

Sensor System Components 60 mm 1.2 mm thick

PIXIES Protein Imprinted Xerogels with Integrated Emission Sites Response (%) Protein * Analyte The sensors selectively respond to Ovalbumin Orders of magnitude greater than other components Each site can individually respond to different analytes *

Summary A unique collaborative initiative that enables state-of-the-art Biometric Science and Technology Creating a multi-disciplinary environment attracting faculty and students from engineering and sciences Preparing and educating future Biometric Scientists and Engineers Targeting all the aspects of Biometrics from authentication to materials and including them into a packaged device

Websites Acknowledgements www.cubs.buffalo.edu www.photonics.buffalo.edu www.cedar.buffalo.edu www.packaging.buffalo.edu Acknowledgements Financial support of: National Science Foundation (NSF) Office of Naval Research (ONR) Calspan UB Research Center (CUBRC) University at Buffalo Center for Advanced Technology (UBCAT)

Thank You