1. Preliminary Assessment of Discrimination of Twins in Photographs based on Facial Blemishes Nisha Srinivas 1, Matthew Pruitt 1, Gaurav Aggarwal 1, Patrick Flynn 1, Richard Vorder Bruegge 2 1 University of Notre Dame 2 Frederal Bureau of Investigation, Digital Evidence Lab

2. Problem Statement Investigate the usefulness of facial blemishes to distinguish between identical twins – Moles, Freckles, Scars, etc Determine – Whether facial blemishes and locations can be used to distinguish between identical twins – Whether the distributions of facial blemishes are “more similar” for identical twins than unrelated persons?

3. Facial Blemishes Types of facial blemishes – Mole – Freckle – Freckle Group – Pimple – Darkened Patch – Lightened Patch – Splotchiness – Birthmark – Raised Skin – Pockmark – Scars Linear Round Mole Freckle and Freckle Group Lightened Patch Darkened Patch Raised SkinScar (Round)PockmarkPimple

4. Proposed System Overview Manual Annotation Feature Extraction Geometric Normalization Point Cloud Matching Biometric Verification Performance Evaluation

5. Manual Annotation Display Module Annotation Module Tool Module

6. Facial Blemishes Identified by Observer 1

7. Facial Blemishes Identified by Observer 2

8. Facial Blemishes Identified by Observer 3

9. Facial Blemishes Identified by Observers Total Number of Facial Blemishes Annotated by each Observer Observer 1: 3785 Observer 2: 2311 Observer 3: 5100

10. Facial Blemishes Matching N NodesM Nodes Moles

11. Matching Contd. The Edges in the bipartite graph correspond to potential matches Each potential match has a cost associated with it which is a function of the euclidean distance between the centroids of the blemishes being compared.

12. Matching Contd. Match Similarity metric=Number of matches/Max(N,M)

13. Data Twin face images were collected at the Twins Days Festival in Twinsburg, Ohio in August 2009. High Resolution Images: 4310 rows x 2868 columns Dataset Attributes – Frontal (yaw=0), Indoor, No Glasses, Neutral Expression Number of Images: 295 – Number of Subjects: 152 – Number of Twins Pairs: 76 Terminology – Target set: “gallery” of persons to be recognized – Query set: a set of images of unidentified persons to be matched against the target set

14. Experimental Setup Perform two different experiments – Individual Observer Analysis Query set and Target set are annotated by same observer – Inter-Observer Analysis Query set is annotated by one observer and the Target set is annotated by another observer – Observer 1 vs Observer 2 – Observer 2 vs Observer 3 – Observer 3 vs Observer 1

15. Subset of Facial blemishes FM={moles, freckles, freckle group, pimple, birthmark, darkened patch, lightened patch, splotchiness, raised skin, pockmark, scar round, scar linear} FM1=FM-{pimple} FM2={moles, freckles} FM3={moles, freckles, pimple}

16. Twins vs Twins Setup: Query Set Target Set Subject 1, Twin A Subject 2, Twin B Subject 3, Twin A Subject 4, Twin B Match Comparison Non-Match Comparison

17. Individual Performance Evaluation- Observer 3

18. Match Comparison Non-Match Comparison Query Set Target Set Subject 1, Twin A Subject 2, Twin B Subject 3, Twin A Subject 4, Twin B Subject 5, Twin A All vs All Setup:

19. Individual Performance Evaluation- Observer 3

20. Comparison: All vs All and Twins vs Twins

21. Inter-Observer Performance Degradation in Performance when comparing facial marks annotated by different observers

22. Conclusion There appears a correlation between the distribution of facial blemishes across twins. The number of facial blemishes across twins appears to be similar. Facial blemishes can be used as a potential biometric signature. Consistent annotation is a challenging process – It is difficult to achieve consistency

23. Thank You This research was supported by – NIJ/OJP award 2009-DN-BX-K231 – FBI through TSWG/ARMY RDECOM contract W91CRB-08-C-0093