1. C MU U sable P rivacy and S ecurity Laboratory http://cups.cs.cmu.edu/ Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor, J. Hong, E. Nunge

2. Phishing email

3. Subject: eBay: Urgent Notification From Billing Department

4. Phishing email We regret to inform you that you eBay account could be suspended if you don’t update your account information.

5. Phishing email https://signin.ebay.com/ws/eBayISAPI.dll?SignIn&sid=veri fy&co_partnerid=2&sidteid=0

6. Phishing website

7. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 7 What is phishing? Phishing is “a broadly launched social engineering attack in which an electronic identity is misrepresented in an attempt to trick individuals into revealing personal credentials that can be used fraudulently against them.” Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service industry perspective. 2005.

8. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 8 Phishing is growing 73 million US adults received more than 50 phishing emails a year in 2005 Gartner found approx. 30% users changed online banking behavior because of attacks like phishing in 2006 Gartner predicted $2.8 billion loss in 2006

9. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 9 Why phishing is a hard problem? Semantic attacks take advantage of the way humans interact with computers Phishing is one type of semantic attack Phishers make use of the trust that users have on legitimate organizations

10. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 10 Counter measures for phishing Silently eliminating the threat Regulatory & policy solutions Email filtering (SpamAssasin) Warning users about the threat Toolbars (SpoofGuard, TrustBar) Training users not to fall for attacks

11. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 11 Why user education is hard? Security is a secondary task (Whitten et al.) Users are not motivated to read privacy policies (Anton et al.) Reading existing online training materials creates concern among users (Anandpara et al.)

12. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 12 Our hypotheses Security notices are an ineffective medium for training users Users make better decision when trained by embedded methodology compared to security notices

13. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 13 Design constraints People don’t proactively read the training materials on the web Organizations send “security notices” to train users and people don’t read security notices People can learn from web-based training materials, if only we could get people to read them! (Kumaraguru, 2006) P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. Tech. rep., Cranegie Mellon University, 2007. http://www.cylab.cmu.edu/files/cmucylab07003.pdf.

14. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 14 Embedded training We know people fall for phishing emails So make training available through the phishing emails Training materials are presented when the users actually fall for phishing emails

15. Embedded training example Subject: Revision to Your Amazon.com Information

16. Embedded training example Subject: Revision to Your Amazon.com Information Please login and enter your information http://www.amazon.com/exec/obidos/sign-in.html

17. Comic strip intervention

18. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 18 Design rationale What to show in the intervention? When to show the intervention? Analyzed instructions from most popular websites Paper and HTML prototypes, 7 users each Lessons learned Two designs Present the training materials when users click on the link

19. Comic strip intervention

20. Intervention #1 - Comic strip

23. Intervention #2 - Graphics and text

24. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 24 Study design Think aloud study Role play as Bobby Smith, 19 emails including 2 interventions, and 4 phishing emails Three conditions: security notices, text / graphics intervention, comic strip intervention 10 non-expert participants in each condition, 30 total

25. Intervention #1 - Security notices

26. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 26 Intervention #2 - Graphics and text

27. Intervention #3 - Comic strip

28. PhishTraining Legitimate Spam

29. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 29 User study - results We treated clicking on link to be falling for phishing 93% of the users who clicked went ahead and gave personal information

30. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 30 User study - results

31. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 31 User study - results Significant difference between security notices and the comic strip group (p-value < 0.05) Significant difference between the comic and the text / graphics group (p-value < 0.05)

32. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 32 Conclusion H1: Security notices are an ineffective medium for training users Supported H2: Users make better decision when trained by embedded methodology compared to security notices Supported

33. Latest comic strip design

34. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 34 Ongoing work Measuring knowledge retention and knowledge transfer Knowledge retention is the ability to apply the knowledge gained from one situation to another same or similar situation after a time period Knowledge transfer is the ability to transfer the knowledge gained from one situation to another situation after a time period Is falling for phishing necessary for training?

35. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 35 Coming up WWW 2007 CANTINA: A Content-Based Approach to Detecting Phishing Web Sites Learning to Detect Phishing Emails Our other research in anti-phishing http://cups.cs.cmu.edu/trust.php Symposium On Usable Privacy and Security (SOUPS), July 18 - 20, 2007 at Carnegie Mellon University

36. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 36 Acknowledgements Members of Supporting Trust Decision research group Members of CUPS lab

37. C MU U sable P rivacy and S ecurity Laboratory http://cups.cs.cmu.edu/