1. Page 1 Login Security Usability Test Results | August 2014 Login Security Usability Test Results Conducted by Jayne Schurick Usability Consultant jschurick@gmail.com (408)353-1293

2. Page 2 Login Security Usability Test Results | August 2014 Objectives Determine whether owners understand the 2-factor authentication process that is being released in two versions Do they understand why HomeAway is implementing this? Is the authentication process confusing? Is the phone setup process confusing?

3. Page 3 Login Security Usability Test Results | August 2014 Test Format and Participants Remote test using WebEx to allow participants to view and control the computer displaying the prototype. Conducted August 6, 2014. 6 homeowners. –2 own 2 properties, 4 own 1 property –All log into the dashboard several times per week –Dashboard usage: Check or update calendar (4 mentions) Reply to inquiries (3 mentions) Check payments (2 mentions) Request reviews (2 mentions) Get reservation list to send to housecleaner (1 mention) Get traveler phone number (1 mention) Check performance (1 mention)

4. Page 4 Login Security Usability Test Results | August 2014 2FA Challenge – V1 Consider putting periods after each sentence at the top. Consider changing “text” to “text message.” Are these cell phone numbers owners have entered and named in their profile/account? Will these numbers exist for phone verification after release of V2 or will users need to set them up? (If they need to set them up, I think they will be confused about why their numbers are no longer working.) Why does the work number start with “my”?

5. Page 5 Login Security Usability Test Results | August 2014 2FA Challenge – V1 Consider putting a period after the sentence at the top. Consider changing “text” to “text message.”

6. Page 6 Login Security Usability Test Results | August 2014 First Time Setup – V2 If users have experienced V1, they might not understand why you’re asking for a phone number. What happens if the user enters a number and then doesn’t have that phone when they log in? One participant gave an example of international travel. I can imagine this use case and people panicking, not knowing where to go to add another number (in the heat of trying to reply to an inquiry).

7. Page 7 Login Security Usability Test Results | August 2014 First Time Setup – V2 I’m wondering if you should direct them to where they should go to set up all numbers. I still think “label” is a bad label. Really, you only need to know if it’s a mobile phone. Why do you care about the label? “Send me” and “Receive” aren’t in the same language. Are all fields required? I think you should identify required fields. Is there a required format for the phone number? You should display this. One participant didn’t realize he could type in a new number; he thought he needed to select one from the list. One participant said if HA knows his numbers, they should know the country codes.

8. Page 8 Login Security Usability Test Results | August 2014 First Time Setup – V2 For consistency, should this be the same as the dialog presented after the V1 challenge?

9. Page 9 Login Security Usability Test Results | August 2014 2FA Challenge – V2 Until they set up additional numbers in Account Settings, will this only display the number they entered in the first time setup? I’m wondering if you should include an option to add another number. Add “(text)” or “(text message)”.

10. Page 10 Login Security Usability Test Results | August 2014 Account Settings We didn’t test whether users would know to go to Account Setting to edit or add phone numbers. One participant expected to add a phone number in the Account Information tab. Another expected and would have preferred a single location for managing phone numbers with a checkbox for indicating which could be used for phone verification. Maintaining phone numbers in two locations is potentially confusing. The first 2 options start with a verb and the last option doesn’t. The country code format isn’t consistent with the display of +. One participant suggesting displaying the date and time the number was verified.