1. Energy Information Administration Usability Testing Colleen Blessing

2. Energy Information Administration Testing Your Web Site How do you really know if users are finding specific information they need on your site? Where are they having difficulty? Do they find the “wrong” right answer? Are they frustrated? Confused?

3. Energy Information Administration Many Ways to Get Feedback About Your Site Usability testing Don’t forget:  Telephone hotline feedback  Webmail  Feedback from internal users  Focus groups  Online web surveys

4. Energy Information Administration What is usability testing? Researchers devise a series of tasks to accomplish based on the objectives of the site Subjects are asked to perform the tasks, often in a laboratory setting Usually work with small number of subjects (6-8)

5. Energy Information Administration What is Usability Testing? Collecting Information about real users  thinking process  problem solving approaches  navigation issues - where do they go?  understanding the site  feelings about the site  measuring their success in finding information

6. Energy Information Administration Many variations of testing Question-asking protocol Performance measurement Group tests Remote testing Others

7. Energy Information Administration How Usability Tests Work Subject sits at a work station Moderator asks the questions “Scavenger hunt” test - timed exercises Note takers behind one-way glass or nearby No help- “work as if you were in your office or at home” Think aloud process/ empathetic listening One hour total with each person

8. Energy Information Administration Minimal Requirements A work station An Internet connection A moderator A note taker (can be the moderator) A test subject A recording device (audio or video)

9. Energy Information Administration How Many Tests ? Start with as few as 5 subjects Maximum 15-20 subjects Maximum 6 tests per day Make changes and test again

10. Energy Information Administration Recruiting Test Subjects Get the right mix Screening Overbooking Thank-you fee ($25-$75) Make lab arrangements

11. Energy Information Administration Developing User Tasks Representative tasks based on the objectives of the site Not too obvious but not too difficult Broad representation across the site Questions in clear language Test the questions on yourself

12. Energy Information Administration Demographic Questions Level of expertise on the Internet  Novice, intermediate, expert Familiarity with subject of site and terms Prior site use Age Other information you want to know

13. Energy Information Administration Moderator’s Guide- written protocol or script Why they are here and what they will be doing Encouragement to think aloud Testing the site, not you Confidentiality Permission to audio/videotape Test questions

14. Energy Information Administration Observe and record user actions and reactions What do they –  Do  Like  Find easy  Find difficult Were they successful? If not, where did they fail?

15. Energy Information Administration Data Collection Audio/videotaping Note taking Session summaries Debriefs with colleagues

16. Energy Information Administration Results of our tests Too many buttons/choices Search engine fails Too much jargon/technical language Too many long text files Confusing navigation Clickable items didn’t look clickable Generic categories are not selected

17. Energy Information Administration Results of our tests, cont. Users don’t see everything Design based on your organization Design based on your reports Contacts are critical/ web & phone

18. Energy Information Administration Presentation of test results Organize findings – summarize themes Support conclusions with –  Facts  Quotes  Video clips Hard part is to suggest fixes

19. Energy Information Administration Thoughts on Usability Testing Easier than you think Zero tests give zero results Get management buy-in up front Richer results than customer surveys You won’t believe it until you see it

20. Energy Information Administration Newer Thought on Usability Testing is easier than implementation Web sites run by a democracy are more difficult to change Scientific evidence is difficult to ignore, but solutions can be subjective Don’t get defensive about your results As the messenger, you may be shot (at)

21. Energy Information Administration Usability is Everywhere Life is a usability test You know how frustrating and confusing using the web can be We want customers to be successful using government sites We want finding information/forms/services to be as easy and quick as possible The key: watching actual users work