Usability Testing Instructions

Why is usability testing important? In a perfect world, we would always user test instructions before we set them loose on an unwitting world. (Here’s why.)why The best way for you to make sure your instructions really do work is to let your target users take your instructions for a “test drive” in a process known as user testing. User testing will either: 1.Shake loose all kinds of problems with instructions you throught were practically perfect (most of the time), or 2. Establish your genius by confirming that the instructions are good-to-go as-is.

Types of Usability Tests There are four main types of document tests 1.Written tests 2.Task-oriented tests 3.Attitude questionnaire 4.Informal observation and interview

Written tests Test how well users understand the document, not their ability to complete the task by following the instructions. –Use a written test when you want to test user’s understanding of the concepts after they have read the instructions. –You can use any traditional type of written examination: multiple choice, matching, true-false, completion, etc.

Task-oriented Tests Determine how well users can follow the instructions to complete specific tasks. –Use this test when you want to identify problems with the instructions that prevent the user from successfully using them to complete the specified task. –This type of test requires test subjects to use the instructions in an environment similar to the one where the instructions will actually be used.

Attitude Questionnaires Determine how the users feel about the instructions. –Use this test when you want to get user feedback about specific aspects of the instructions. –This type of test usually includes three sections that allow users to rate various aspects of the instructions: 1.A series of statements about the usability of specific parts of the document 2.A range of responses to the document 3.A comment section for each document Example: The headings in the text made it easy to find information. ____Agree _____Neutral ____Disagree _____(unable to judge) Comments:__________________________________________

Observation & Interview Help you get at issues the other tests miss. –Always use this test in conjunction with other tests. –Watch the user using the instructions and make notes as you notice places where they stall –Talk with them informally after they’ve finished the task

Planning Your User Test Identify 3-5 focal questions for your test (the main issues you want to find out about) Questions should focus on the areas where you suspect the document might have the biggest problems Examples of focal questions: Are the instructions written at the right level of detail for my users? Do the illustrations and text work together in a way that makes it easy for the user to follow the instructions?

Planning the Test You should plan to test your instructions on 3-5 users. To get the best results, you should: Conduct the test in similar/same conditions for each test subject Prepare written instructions for the test, and read the instructions out loud to each test subject before they begin the test. Use the combination of test methods that will help you get answers to your focal questions Make sure your test includes a tast-oriented test that requires your user to step through your entire set of instructions

Doing the test Read the test instructions to your user –Prepare you subject for the entire test by first reading them an overview that tells them what to expect at each stage of the test. (i.e. “Thanks for agreeing to participate in my test. First we’re going to X, then we’re going to Y….) –Read specific instructions for each stage of the test at the appropriate point, i.e. “Now that you’ve answered my survey questions, I’d like you to follow the instructions and try to build the Gobot If you get stumped at any point, you can stop what you’re doing and ask me for help, but I won’t intervene in the process unless you ask.) Pay special attention to your focal point issues, but also keep track of any other problems your subject seems to be having. It will help you if you make notes on your copy of the instructions as the test progresses. Do not intervene or offer help or suggestions to your subject. Your job is to observe and make notes. If the subject bogs down completely and can’t make additional progress, make sure you fully understand the problem they’re having before you help them move on to the next step. Make sure you thank your subject when the test is over.

What do you need to turn in for peer review? For peer review on the Instructions Project, you have to turn in your draft Instructions and a plan for conducting your usability test. The usability test plan can be a 1-page memo that outlines how you will do the test for this project. At a minimum, it should cover the following: Title of the document you’re testing Short description of your intended target audience 3-5 focal questions Description of test methods you will use, including an explanation of how the test methods you select will help you get answers to specific focal questions.

Usability Resources Resources that might give you a jump start: –Usability Testing Worksheet: A sample worksheet you would use to administer an informal testUsability Testing Worksheet –Common Problems: Common problems that might lend themselves to becoming research focal questions.Common Problems –A Formal Usability Testing Plan: This is an extensive, formal test plan (like what we’ll developed during the research design phase of the Web Usability Project).A Formal Usability Testing Plan Note: For the instructions project, we’re writing a 1-page or so informal test plan, so don’t spend time creating something this elaborate. I include it as a sample here because it might help you see how you can produce your own-scaled down informal test plan for the Instructions project. The bulleted list in the Introduction here includes all of the key components you need for you informal test plan – you can just scale back each section to a few sentences, and skip the resources and schedule altogether.