1. 1 + 1 = You Measuring the comprehensibility of metaphors for configuring backup authentication Stuart SchechterRobert W. Reeder Symposium on Usable Privacy and Security (SOUPS), 2009 Presented by: Payas Gupta

2. Why backup authentication? No good backup authentication system till date Commonly used – Email based authentication (Gmail) – Personal authentication questions (Hotmail)

3. Email based authentication Popular but of limited use

4. Why good backup authentication? Cases like - Republican vice president candidate Sarah Palin’s Yahoo! account 20% of users forget their answers within six months 17% of answers guessed by acquaintances 13% of answers were easily cracked by 5 most popular responses

5. One-size-fits-all One-size-fits-all approach will never work Some have little to protect Some have critical information More tasks and if she fails then? Less tasks and if attackers gets access then?

6. Exam Metaphor

7. Evidence Scale Metaphor

8. Screenshots presented Windows Live ID – password reset SplendMail – Short exam P5 form – Short exam P10 form – Long exam – Evidence scale form

9. Windows Live ID password reset

10. Short exam P5 form

11. Short exam P10 form

12. Long exam form

13. Evidence scale form

14. Questions accompanying screenshots One-or-both questions for both the Live ID and P5 form. – To change her password, will [Windows Live|SplendMail] require Jane to establish her identity using both the e-mail address and the question, or is one of the two enough? Answer options – one, probably one, not sure, probably both, both

15. – Will Jane be able to change her password after performing all of the following actions (and only those actions) to prove her identity to [Windows Live|SplendMail]? Answer options - yes, probably, not sure, probably not, and no.

16. Mechanism comprehension Answer options - definitely A, probably A, not sure, probably B, and definitely B.

17. Zero-centered five point scale Integers from -2 to 2 scale For example, if the correct answer to a question was yes, a participant would receive – 2 points yes – 1 for probably yes – 0 for not sure – -1 for probably not – -2 for no.

18. Participant Demographics Total 18 – Age between 30 and 48 – 7 female 11 male – 12 have Live Id accounts

19. Hypothesis 1 When presented short exam P5, which describes how each authentication mechanism will be used, Live ID users are better able to comprehend the use of these mechanisms than when presented with Live ID's password- reset settings form.

20. Hypothesis 2 Live ID users comprehend the evidentiary requirements of authentication in the short exam form as well as they do for Live ID's current password reset settings form.

21. 2b

22. 2c

23. 2d

24. Hypothesis 3 Comprehension of the exam metaphor decreases as more authentication mechanisms are configured.

25. Hypothesis 4 The evidence scale form, which does not require mental math, is more comprehensible than the exam form, which does.

26. Hypothesis 5 Users prefer the exam form to the evidence scale form, or vice versa.

28. Result: Hypothesis 1 When presented short exam P5, which describes how each authentication mechanism will be used, Live ID users are better able to comprehend the use of these mechanisms than when presented with Live ID's password- reset settings form. Not Significant

29. Result: Hypothesis 2 Live ID users comprehend the evidentiary requirements of authentication in the short exam form as well as they do for Live ID's current password reset settings form. Strongly significant

30. Somewhat Significant

31. The difference in the mean scores for the hardest questions about the exam and the mean scores and the three simple questions about the Live ID screenshot is statistically significant -- in favour of the exam.

32. Result: Hypothesis 3 Comprehension of the exam metaphor decreases as more authentication mechanisms are configured.

33. Result: Hypothesis 3 18 participants – only 5 of 54 total responses on the longer exam were imperfect (either incorrect or not provided with full confidence). Not only was there no evidence of a decrease in comprehension, but the learning effect might well have increased comprehension. Not Significant

34. Result: Hypothesis 4 The evidence scale form, which does not require mental math, is more comprehensible than the exam form, which does. Reject This: Exam form more comprehensible

35. Result: Hypothesis 5 Users prefer the exam form to the evidence scale form, or vice versa. – Exam was better

36. Limitation/Future Work How points would be assigned to authentication tasks in the exam metaphor? The effect of user demographics?

37. Conclusion Looked into--If users cannot understand whether one or both of two tasks is required to authenticate, how could they be expected to understand which of 5 tasks would be sufficient User authentication is, after all, a complex process. 15 of our 18 participants answered all eight questions about the exam metaphor perfectly.

38. Conclusion cont… Only one of the 18 participants missed more than one question.