Introduction to Evaluation without Users

Where are you at with readings? Should have read –TCUID, Chapter 4 For Next Week –Two Papers on Heuristics from Jakob Nielsen's site

Looking Ahead a little… Start working on your executable prototype –Complete all the peripheral tasks so you are in a position to work on the interface Database back-ends, network connections to DB Style sheets, banner images, backgrounds Icons (Careful!) Eventually due November 11th.

Interface Development Methodology Prototype and Iterate –keep iterating until it is good enough –evaluate along the way to assess What is Good? What is Good Enough? –set usability goals –should relate to tasks

Casual Iteration Find major usability problems –missing features –user confusion –poor interaction Try interface with specific tasks –first use designers, then move towards users –observe overall usage

Casual Iteration Remember the goal –don’t defend the interface –don’t bias the tests towards the interface If possible, allow user exploration –may even lead to capturing new tasks Consider alternative ways to fix a problem

Limits of Casual Iteration Does not indicate when to stop Financial trade-offs Harder time providing justification of delays

Evaluation – With or Without Users?? Why do without –User time is valuable – find/solve obvious problems so you don’t waste users’ time –Users aren’t perfect, so a well designed user-less evaluation can be just as good (if not more helpful). –Sometimes targeted users are not the actual users But users are necessary –Cannot be simulated perfectly –May better help you resolve tradeoffs. You need both

Evaluation Without Users Qualitative Methods –expert evaluation –cognitive walkthrough –heuristic evaluation Quantitative Methods –back-of-the-envelope action analysis – GOMS/keystroke analysis

Expert Evaluation Still considered “without users” Usability specialists are very valuable –double-specialists are even better An inexpensive way to get a lot of feedback Be sure the expert is qualified in your area Problem – specialists aren’t human

Cognitive Walkthroughs CONCEPT : –A formalized way of imagining people’s thoughts and actions when they use an interface for the first time. THEORY : –A user sets a goal to be accomplished by the system. –The user will search the system for the action that seems likely to make progress towards that goal. –By putting yourself in the user’s shoes you can figure out where the “gulfs” might occur.

Cognitive Walkthroughs – how and why Goals –imagine user’s experience –evaluate choice-points in the interface –detect confusing labels, icons, images or options –detect likely user navigation errors –improvement, not defense Start with a complete TCUID scenario –never try to “wing it” on a walkthrough

Cognitive Walkthrough How To - I Begin by collecting: –An idea of who the users will be and their characteristics –Task description –Description of the interface (a paper prototype) –Written list of the actions to complete the task given the interface (scenario)

Cognitive Walkthrough How To - II For each action in the sequence (scenario) –tell the story of why the user will do it –ask critical questions (4 plus 1) will the user be trying to produce the effect? will the user see the correct control? will the user see that the control produces the desired effect? will the user understand the feedback to proceed correctly? will the user select a different control instead? Every gap is an interface problem

Making this Approach Work Tell a Believable Story –How does the user accomplish the task, action-by-action –Based on user knowledge and system interface Work as a group –don’t partition the task Be highly skeptical –remember the goal!

Benefits of a Cognitive Walkthrough Focus most on first experiences - learnability Easy to learn Can do early in the software cycle Questions the assumptions about what a user might be thinking. Can identify controls that are obvious to the SE but not to the user It can suggest difficulties with labels and prompts It can help find inadequate feedback Can help find inadequacies in the spec

Shortcomings of Cognitive Walkthrough Is diagnostic, not prescriptive Focuses mostly on novice users Relies on the ability of engineers to put themselves in the users shoes

When to do a Cognitive Walkthrough Before you do a formal evaluation with your users Can be done on your own for small pieces of the whole Can do a walkthrough of a complete task as the interface develops

Monday’s Deliverable One copy (for me) of the tasks you submitted in week 5 (revised as appropriate). N+1 copies of at least PART of your current (revised) prototype N+1 copies of three detailed scenarios for your current prototype N+1 copies of “user profiles” for the users you reference in your scenarios

Monday’s Deliverable N+1 copies of at least PART of your revised prototype –Expect to have to add detail to your prototype (eventually, you will need enough detail to support evaluation). –Create specific, labeled (on the back) screen shots to support the three scenarios (next slide) –Make sure you have included appropriate transitional screens

Monday’s Deliverable N+1 copies of the detailed scenarios for your prototype –Write up at least three of your tasks using your current (evolving) prototype as your “how” –Should include the two tasks which you used for your previous scenarios. –Include references to labels.

Monday’s Deliverable N value Group APrestige Worldwide5 HyruleVoldemort5 SportasticsWork in Progress6 and 3

Next Friday’s deliverable As a GROUP perform the cognitive walkthrough for each of 3 scenarios (You have Monday’s class time). Walkthrough evaluation log –At each step/action in the scenario, answer the five questions. –In many cases, this should be more than simply Yes/No. Where appropriate, include WHY you think your answer is appropriate. Walkthrough evaluation report –Write up a list of interface problems discovered during the walkthrough –Add brief notes about how you discovered them Any questions?