Introduction to Web Authoring Ellen Cushman Class mtg. #21

Where we are  M 3/17 assignment #4 intro, assign groups, brainstorm orgs.  W 3/19 selecting organizations, problem analysis procedure  M 3/24 proposal workshop  W 3/26 web standards (proposals due 1-2 pages)  M 3/31 designing test, planning  W 4/2 testing and compliance, executing  M 4/7  M 4/7 visual rhetoric, intro (readability (fonts & white space), rhetoric and psych of color schemes)  W 4/9  W 4/9 visual rhetoric, 2 (space: real estate, eye/hand movement, information layout)  M 4/14 Workshop usability testing (Progress report due 3-6 slides)  W 4/16 Presentation workshops.  M & W 4/21- /23 Presentations on websolutions Reports due. 4/23  M 3/17 assignment #4 intro, assign groups, brainstorm orgs.  W 3/19 selecting organizations, problem analysis procedure  M 3/24 proposal workshop  W 3/26 web standards (proposals due 1-2 pages)  M 3/31 designing test, planning  W 4/2 testing and compliance, executing  M 4/7  M 4/7 visual rhetoric, intro (readability (fonts & white space), rhetoric and psych of color schemes)  W 4/9  W 4/9 visual rhetoric, 2 (space: real estate, eye/hand movement, information layout)  M 4/14 Workshop usability testing (Progress report due 3-6 slides)  W 4/16 Presentation workshops.  M & W 4/21- /23 Presentations on websolutions Reports due. 4/23 Note the changes in content as per 3/26 discussion

Today in Class | Next Class Designing a usability test Activity: design the test materials with your group Your test materials should be done by next class so we can do a run through with them. Test the test materials! Execute the test! Analyze the results! Revise the test materials!

Steps in Planning a Usability Test  Determine User Groups We Want To Test  Determine What We Want to Test  Determine Our Test Metrics  Write or Choose our Scenario  Create data gathering tools  Create a test protocol  Create a data analysis protocol  Determine User Groups We Want To Test  Determine What We Want to Test  Determine Our Test Metrics  Write or Choose our Scenario  Create data gathering tools  Create a test protocol  Create a data analysis protocol

Planning a User Test: User Groups Your initial user needs and genre analysis work (interviews, surveys, etc.) should allow you to identify goals each user group might have when visiting the site. Eg. “prospective members” Goal: Evaluate the benefits of membership. Using what? : Info about past events and upcoming events…(what else?)

Planning a User Test: Scenarios Use the info you have gathered to write a sample scenario, including information about Who What Why Where But not How!

Try your own scenario! Write one about a typical user for your site. Remember…don’t put “how” info into the scenario. Stick to: who, what, when, why, where… For instance: Elementary school teachers what to attend the Bright Ideas Conference in the Spring. They hope to see their favorite authors and learn some new methods in the classroom. They’re around the state of MI, OH, and IN and they’re coming to MSU.

Determine Who to Test User Group: Students Target Audience: Those looking for on- campus housing for the coming year User Goal: Get first choice of places to live, roommates, etc. User Group: Students Target Audience: Those looking for on- campus housing for the coming year User Goal: Get first choice of places to live, roommates, etc. Consider your design priorities Think about a representative sample of the group you choose.

Determine What to Test Based on the user goal, list specific, observable outcomes we can solicit from users in one or more tasks. Example Outcomes:  Starting at home page, locate…  After test, recall…  Submit all information necessary to…  Answer the following question…. Based on the user goal, list specific, observable outcomes we can solicit from users in one or more tasks. Example Outcomes:  Starting at home page, locate…  After test, recall…  Submit all information necessary to…  Answer the following question….

Determine Test Metrics: Performance Criteria  User locates X piece information and writes it down on test form  User is able to find and download X to the desktop (yes/no)  User is able complete X task in less than 10 minutes, total; (anticipate sources of non-task related delays)  User locates X piece information and writes it down on test form  User is able to find and download X to the desktop (yes/no)  User is able complete X task in less than 10 minutes, total; (anticipate sources of non-task related delays) Specific Criteria for Success for each Outcome Note: Common performance metrics are based on task success, time, and # of errors

Determine Test Metrics: User Satisfaction Criteria  User finds the site helpful, well-suited to the task (4 or 5 on a 5 pt. scale)  User finds the site easy to use (4 or 5 on a 5 pt. scale)  Users are confident that they completed the task successfully (4 or 5, etc.)  User finds the site helpful, well-suited to the task (4 or 5 on a 5 pt. scale)  User finds the site easy to use (4 or 5 on a 5 pt. scale)  Users are confident that they completed the task successfully (4 or 5, etc.) Specific Criteria for Success for each Outcome Note: Common satisfaction metrics are based on confidence of task success, perceived difficulty, and frustration level

Write a Scenario + Tasks  Create a background scenario to orient the participant to their role and goals…can be drawn from your “use case”  Create individual task descriptions that match up with each observable outcome  Sequence tasks so as to avoid interference issues (e.g. learning effects. In other words, don’t ask to achieve same outcome twice because they’ll use what they learned in the first trial to accomplish the task.)  Create a background scenario to orient the participant to their role and goals…can be drawn from your “use case”  Create individual task descriptions that match up with each observable outcome  Sequence tasks so as to avoid interference issues (e.g. learning effects. In other words, don’t ask to achieve same outcome twice because they’ll use what they learned in the first trial to accomplish the task.)

Create Test Materials 1.Disclaimer; thank you note. 2.Background information sheet on user’s “role” 3.Scenarios & task sheets with blanks for observations and metrics at bottom. 4.Questionnaire/Interview Questions: post-task & post- test 5.Observation notes sheets 1.Disclaimer; thank you note. 2.Background information sheet on user’s “role” 3.Scenarios & task sheets with blanks for observations and metrics at bottom. 4.Questionnaire/Interview Questions: post-task & post- test 5.Observation notes sheets

Sample task sheet 1.You are considering attending a sailing club event. Use the site to determine the most appropriate upcoming event for learning about the club. For this, we ’ d want to identify ahead of time the “ answers ” (info located on page X) and decide on some metrics for determining success (most direct route takes 3 clicks off main page). Leave Space for your observations in the middle of the sheet. Bottom of the sheet should have your test metric: Actual number of clicks to accomplish task: Bottom of the sheet should also have a post-task question. 1.You are considering attending a sailing club event. Use the site to determine the most appropriate upcoming event for learning about the club. For this, we ’ d want to identify ahead of time the “ answers ” (info located on page X) and decide on some metrics for determining success (most direct route takes 3 clicks off main page). Leave Space for your observations in the middle of the sheet. Bottom of the sheet should have your test metric: Actual number of clicks to accomplish task: Bottom of the sheet should also have a post-task question.

Sample User Satisfaction Sheet  User finds the site helpful, well-suited to the task  5 point scale here  User finds the site navigation easy to use  5 point scale here  User’s frustration level with the site  5 point scale here  User finds the site content easy to use  5 point scale here  User is confident that s/he completed the task successfully  5 point scale here  User finds the site helpful, well-suited to the task  5 point scale here  User finds the site navigation easy to use  5 point scale here  User’s frustration level with the site  5 point scale here  User finds the site content easy to use  5 point scale here  User is confident that s/he completed the task successfully  5 point scale here

Test Day! Before the test  Remind all of your users of the agreed upon time, place, etc.  Double check the room and equipment  Be sure all test materials are present  Be sure the computer is in the correct beginning state (turn off the screen saver)  Do any pre-test data gathering that is needed  Remind all of your users of the agreed upon time, place, etc.  Double check the room and equipment  Be sure all test materials are present  Be sure the computer is in the correct beginning state (turn off the screen saver)  Do any pre-test data gathering that is needed

Homework  Finish your test materials for next class