A Cross-Sensor Evaluation of Three Commercial Iris Cameras for Iris Biometrics Ryan Connaughton and Amanda Sgroi June 20, 2011 CVPR Biometrics Workshop Computer Vision Research Lab Department of Computer Science & Engineering University of Notre Dame

Objectives Compare three commercially available sensors – Does one sensor consistently out-perform the others? – What factors impact sensor performance the most? Observe performance of sensors in a cross-sensor scenario – What kind of performance can we expect from a cross- sensor system? – What is the relationship between single-sensor and cross-sensor performance? 2

Overview of Experiment Strategy Collect images for the same subjects using all 3 sensors under similar conditions Use 3 different matching algorithms to perform matching experiments In a single-sensor context In a cross-sensor context Analyze performance of sensors in each scenario 3

Sensors 4 SensorIris-to-Sensor Distance Wavelength(s) of NIR Illumination Type of Illumination (cross or direct) Acquisition Instructions S18 to 12 inches820 nmBoth (same time)Sensor Prompt S210 to 14 inches770 and 870 nmBoth (different times)Sensor Prompt S313 inches870 and 760 nmCross *Operator *Speculation

Image Examples 5 S1 S2 S3 Same Subject, Same Session Images

Data Collection Results 23,444 Iris Images acquired, spanning 510 subjects (1,020 unique irises) 6

The Matching Algorithms A1 - Similarity Score - Asymmetric Scores A2 - Distance Score - Asymmetric Scores A3 - Distance Score - Symmetric Scores 7

8 Segmentation Information

9 Note: Image information using A1 is not easily accessible and is thus not included.

Match and Non-Match Comparisons 10

The Experiments S1 vs S1 S2 vs S2 S3 vs S3 S1 vs S2 S1 vs S3 S2 vs S3 11 Single-Sensor & Cross-Session Cross-Sensor & Cross-Session These experiments were repeated for all 3 matchers

12 ROC Curves Using A1

13 ROC Curves Using A2

14 ROC Curves Using A3

15 TAR's at FAR = 0.01 A1A2A3 S1 vs S (1) (2) (1) S2 vs S (3) (1) (4) S3 vs S (6) (6) (5) S1 vs S (2) (3) (3) S1 vs S (4) (5) (2) S2 vs S (5) (4) (6) Numbers in parentheses indicate ranking within the corresponding matching algorithm

16 Sensor FAR = 0.01 A1A2A3 1S1 vs S1S2 vs S2S1 vs S1 2 S1 vs S2S1 vs S1S1 vs S3 3 S2 vs S2S1 vs S2 4 S1 vs S3S2 vs S3S2 vs S2 5 S2 vs S3S1 vs S3S3 vs S3 6 S2 vs S3 Brackets indicate that performance difference at FAR=0.01 may not be statistically significant

17 Single-Sensor Conclusions S3 consistently performed the worst for all matchers S1 was best for 2 of 3 matchers Best overall performance was achieved using S1 sensor with A1 matcher FAR=0.01)

18 Cross-Sensor Conclusions A1: Cross-sensor performance was between performance of individual sensors A2: In general, cross-sensor performance was between performance of individual sensors – S1 vs S2 actually performed slightly worse than either single sensor A3: Individual sensor performance is not a good predictor of cross-sensor performance – S1 vs S3 appears to perform better than S1 vs S2

19 General Conclusions Sensors and matching algorithms should be evaluated in combination, not separately In some cases, adding a new and “better” sensor for cross- sensor matching will increase performance, but in some cases it will degrade performance Single-sensor performance is not always a reliable predictor of cross-sensor performance

20 Future Work Which results are statistically significant? What factors have the largest effect on performance? – Pixels on the iris – Dilation ratio – Occlusion – Contact Lenses – Order of sensors during acquisition – Focus or other quality metrics – Illumination

21 Thanks! Questions? Acknowledgments: This work is sponsored under IARPA BAA through the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF The views and conclusions contained in this document are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of IARPA, the Army Research Laboratory, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.