1. Usability Testing and Analytics Insights
Ed Reese and Tine Reese

2. What is it?

3. Why businesses don’t test
Too much $$$
Too much time
Failure to understand the value
Source: Steve Krug

4. Fact or myth? I don’t need to test. I am the user.

6. We are not our users We don’t appreciate how diverse they are
Use is idiosyncratic:
Person
Environment
Context
We know too much!

7. Remember,… they don’t know what we know.
Actual
User Us

8. What Can I Test? Napkin Sketch Prototype Developed Interface
Source:

9. A thing is usable if...
A person of average—or even below average—ability and experience (i.e., most people)
Can figure out how to use the thing for its intended purpose
Without it being more trouble than it’s worth.
Source: Steve Krug

10. Dana Chisnell, co-author of Handbook of Usability Testing
“80% of the value of testing comes from the magic of observing and listening as people use a design. The things you see and the things you hear are often surprising, illuminating, and unpredictable. This unpredictability is tough to capture in any other way.”
Dana Chisnell, co-author of Handbook of Usability Testing

11. DIY Usability Testing Start small: focus on one important thing
3–5 Users
In person or remote
Record video and audio
Identify the biggest problems
Fix problems

12. What to test?
“Focus ruthlessly on a small number of the most important problems.”
Look at Analytics
Talk to customer service
Something your team is arguing about

13. Show users. Ask questions.
UsabilityHub.com: Free (Tests up to 2 min, self recruited)

14. (Not your friends and family)
Find Users
(Not your friends and family)
Make recruiting easier by keeping the standard very low.

15. Recruit Participants Website survey (ethn.io) Email list
Social networks
Craigslist
Go where your users are
Or pay a service to quickly access study participants.
Incentives make it easier!

16. Screeners Find representative users Meet criteria Represent diversity
Tools: Google Forms
Include a Mix
Include a Mix
Must say “no”
Must say “yes”

17. What do you want them to do?
Create the task(s) you want participants to complete.
Reserve a table at our restaurant.

18. Create Task Scenarios
Write a brief description to help users understand what you want them to do:
You are meeting friends at Luna Restaurant tonight for dinner. Use the website to make a 7:00pm reservation for four people.

19. Writing Scenarios
Make it realistic: what is the user naturally trying to do?
Use natural language
No jargon
Don’t give away clues

20. Instructions and Consent “We are testing the website, not you.”

21. Conducting the Test Goal is unbiased, honest feedback
Remind participants to “think aloud.” You can’t read their minds.
Ask questions when they’re confused or frustrated, but do not help.
Record Video

22. Remote Testing Listen carefully to users since you can’t see them
Run a pilot test
Remote Meeting Software:
Zoom
GoToMeeting
Skype

23. Look for the biggest problems, not the low- hanging fruit.
Observe Problems:
Look for the biggest problems, not the low- hanging fruit.
Sensible.com

24. What is the smallest change we can make that will solve the problem?
Tweak. Don’t redesign.
What is the smallest change we can make that will solve the problem?

25. Test Again
The value lies in doing many small tests, making improvements each time.

26. Usability Testing Resources
Sensible.com
Usability.gov

27. User Testing Examples & Analytics Insights

29. “It just feels kinda icky
“It just feels kinda icky. I can’t quite put my finger on it… It feels like a payday loan site.”
User Tester 2 (when asked about original website vibe)

31. Before
After
Website Redesign by JEB Commerce

32. Lack of trust was the main issue resolved. But how
do you measure trust?

36. We Tested… Brand & Overall Vibe
Could you see yourself wearing Cocksox?
How does the imagery & presentation resonate?
Ecommerce Functionality
Can you find the perfect Cocksox for you?
Is order placement easy?

37. We asked users about the photos

38. Cocksox has Underwear Technology

39. User Testing Video Insights
Video 1 “I think you need some average Joe’s
Video 2 “This is an ecommerce deal breaker.”
Video 3 “I’m not sure what activewear means.”

40. Cocksox Updated Homepage

41. We asked users about the photos

42. We asked users about the photos

43. What can you test or measure? Just About anything.

44. Analytics Insights Perform Baseline Reporting Before Testing
Identify pages important to measure
Establish consistent goals & KPI’s
Look for Changes before & after Implementation
Navigational Changes
Bounce Rate
Goal Conversions

45. We asked users about the photos