Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5.0 Presented by Yin Shi

Overview  Introduction  Understanding the Problem  Cognitive Walkthrough  User Test  Conclusion

Introduction  Effective security requires a different usability standard  Security mechanisms are effective only when used correctly –Matt Bishop claimed that configuration errors are the cause of more than 90% of all computer security failures  Making security usable will require the development of domain-specific user interface design principles and techniques  Choose PGP 5.0 for our case study –Designed by general consumer software standards –“Significantly improved graphical user interface makes complex mathematical cryptograph accessible for novice computer users.”

Understanding the Problem  Defining Usability for Security –Definition: Security software is usable if the people who are expected to use it:  Are reliably made aware of the security tasks they need to perform  Are able to figure out how to successfully perform those tasks  Don’t make dangerous errors  Are sufficiently comfortable with the interface to continue using it

Understanding the Problem  Problematic Properties of Security –Five inherent properties of security  The unmotivated user property  The abstraction property  The lack of feedback property  The barn door property  The weakest link Property  A Usability Standard for PGP –Need for privacy and authentication –What needs to be done –How to do it and avoid dangerous errors

Evaluation Methods  Two Methods –An informal cognitive walkthrough –A user test performed in a laboratory

Cognitive Walkthrough  Visual Metaphors (keys) –PGP’s user interface relies on graphical depictions of keys and locks –Improvements  An extension of the metaphor to distinguish public keys for encryption and private keys for decryption  Different icons for public and private keys

Cognitive Walkthrough  Visual Metaphors (signatures) –The icon of the blue quill pen is used to indicate signing is problematic –Quill pen icon will not help user understand they need to use their private keys to generate signatures –Improvements  Keep quill pen to represent signing, but modify it to show a private key as the nib of the pen  Use some entirely different icon for signatures

Cognitive Walkthrough  Different Key Types –Originally, PGP used the RSA algorithm for encryption and signing –PGP 5.0 uses the Diffie-Hellman/DSS algorithm –PGP 5.0 can handle RSA keys, but other version PGP can’t handle DSS keys –Lack of forward compatibility  Recipients with RSA keys can’t decrypt it  Recipients with RSA keys can’t verify signatures –PGP 5.0 alerts its users to this compatibility issues in two ways

Cognitive Walkthrough  Different Key Types –Uses different icons to depict the different key types –When user attempt to encrypt documents using mixed key types, a warning message is showed –Improvement  Double-clicking on a key pops up a Key properties window

Cognitive Walkthrough  Metaphor of choosing people  Human icons obscure the key type information  Better to display multiple keys that person owns

Cognitive Walkthrough  Key Server –Are publicly accessible databases –PGP offers three key server operations under the Keys pull-down menu

Cognitive Walkthrough  Problems with the presentation of the Key Server –Users may not realize that it exists  No representation of it in the top level of PGPkeys display –PGPkeys keeps no records of key server access –PGP’s key revocation operation does not send the resulting revocation certificate to the key server

Cognitive Walkthrough  Key Management Policy –Two ratings for each public key  Validity – how sure the user is that the key is safe to encrypt with  Trust – how much faith the user has in the key –May not realize PGP can automatically sets the validity rating of a key based on whether it has been signed by a certain number of sufficiently trusted keys.

Cognitive Walkthrough  Irreversible Actions –Accidentally deleting the private key –Accidentally publicizing a key –Accidentally revoking a key –Forgetting the passphrase –Failing to back up the key rings  Consistency –encoding  Too Much Information –PGPkeys application presents the user with too much information to make sense of  Owner’s name, validity, trust level, creation date, and size  Nothing to help the user figure out which parts of the display are the most important to pay attention to

User Test  Test Design  Initial task is to send the secret message to the team members in a signed and encrypted  Main steps –Generate a key pair, get the public keys –Make their own public key available to team members –Type the secret message into an s –Sign the using private key, encrypt the using the team member’s public keys

User Test  One of the member had an RSA key  Participant would encounter mixed key types warning message  Each of the five campaign members was represented by a dummy account and a key pair: –These were accessible to the test monitor through a network laptop  The test monitor could send to the participant from the appropriate dummy account

User Test (Results)  Avoiding dangerous errors –Three of them accidentally ed the secret without encryption –One forgot her passphrase  Figuring out how to encrypt with any key –One couldn’t figure out how to encrypt at all –A reconfiguration of PGP may required –Another one kept sending unencrypted test messages, and finally succeeded after being prompted to use the PGP plug in buttons

User Test (Results)  Figuring out the correct key to encrypt with –11 participants figured out how to encrypt, but failed to understand the public key model –Another one so completely misunderstood the model that he generated key pairs for each team member rather than for himself  Decrypting an message –Five participants received encrypted –One can’t figure how to decrypt it –Two took a very hard time to figure it out –Other two were able to decrypt without any problem

User Test (Results)  Publishing the public key –Ten could make their public key available to the team members –Two never addressed key distribution –Those ten, five sent their keys to key server –Three ed to the team members –Other two did both  Getting other people’s public keys –Eight successfully got the team members’ public keys –The others either never seemed aware they need other people’s public key, or they did know how to get it

User Test (Results)  Handing the mixed key types problem –Only four managed to send encrypted correctly –One didn’t have mixed key types problem –The other three received a reply for complaining that they couldn’t decrypt  Signing an message  Verifying a signature on an message  Creating a backup revocation certificate –Only three participants managed to successfully send encrypted and decrypt a reply –In response to direct prompting for backup  One didn’t send the key pair to the key server  One sent to the campaign manager  One simply ignored the prompt

User Test (Results)  Deciding whether to trust keys from the key server –Of the eight participants, only three expressed some concern over if they should trust the keys –None of the three made use of the validity and trust labeling provided by PGPKeys

Conclusion/Questions  PGP 5.0’s user interface does not come even reasonably close to achieving our usability standard  It does not make public key encryption of electronic mail manageable for average computer users  Public work on usability evaluation in a security context would be extremely valuable  We expect to find better design strategies