1. Testing your design Without users: With users: Cognitive walkthrough
Action analysis
Heuristic analysis
With users:
Usability testing

2. Messages
Cognitive walkthrough enables a designer to evaluate an interface without users
Forces a designer to see the interface from the perspective of a user
Low-investment technique to identify task-related usability issues early in the design process
No implementation or users required
But can be performed on existing interfaces

3. Process Early design Analysis Late design Develop Analyze users
Define goals
Early design
Analysis
Heuristic evaluations cognitive walkthroughs
Site visits
Contextual inquiry
Analyze Tasks
Design on paper
Evaluate
Late design
User studies, function tests
Develop
user studies
Evaluate
Implement
Evaluate
Prototype
Field studies, call center, interaction logs
Ship
Evaluate

4. Cognitive Walkthrough
Question assumptions about what the users will be thinking
Identify controls that are obvious to the design engineer but may be hidden from the user's point of view
Note inadequate feedback
Uncover shortcomings in the current specification
Question assumption about users tasks
Focus most clearly on a problems that users will have when they first use an interface
A tool for developing the interface, not for validating it
Reference

5. Cognitive Theory
The user sets a goal to be accomplished with the system (for example, "check spelling of this document").
The user searches the interface for currently available actions (menu items, buttons, command-line inputs, etc.).
The user selects the action that seems likely to make progress toward the goal.
The user performs the selected action and evaluates the system's feedback for evidence that progress is being made toward the current goal.

6. Cognitive Walkthrough
Identify “big” problems before implementation
Invest a little now, save a lot later
Enables more rapid iteration earlier in design
Can do several evaluations of trouble points
Evaluations are only effective if your team
Has the right skill set
Wants to improve the design, not defend it

7. Walkthrough Basics
Imagine how well a user could perform tasks with your low-fidelity prototype
Manipulate prototype as you go
Evaluate choice-points in the interface
Evaluate labels or options
Evaluate likely user navigation errors
Revise prototype and perform again

8. When to do the Walkthrough
Have a low-fidelity prototype of the interface
Know who the users are
Have task descriptions
Have scenarios designed to complete the task
You have a “functional” paper prototype
Viable once the scenario and paper prototype are completed

9. Who should do a walkthrough, and when?
you can do your own, informal, “in your head” walkthroughs to monitor the design as you work
with a group of people, including other designers and users

10. You need 4 things. User personas
who the users will be and what kind of goals and experience they'll bring to the job
A description or a low-fidelity prototype of the user interface.
Task descriptions
A complete, written list of the actions needed to complete the task with the interface
And an evaluation team:
Design team
Design team and users together
Design team and other skilled designers

11. What to do? Tell a story
about why the user would select each action, and critique the story to make sure it's believable with these questions:
Will users be trying to produce whatever effect the action has?
Will users see the control (button, menu, switch, etc.) for the action?
Once users find the control, will they recognize that it produces the effect they want?
After the action is taken, will users understand the feedback they get, so they can go on to the next action with confidence?

12. More on Questions Some extra questions can help
What happens if the user is wrong? Is there feedback to correct the error?
How would a user of <other interface> react here?
Questions help you see problems
They are a focus, not a blindfold

13. Best Approach Work as a group Be highly skeptical of the design
don’t partition the task
Be highly skeptical of the design
remember the goal!
Each gap between what and how is a usability problem

14. Past Empirical Results
Users will try label-guided actions first (menu items, buttons, etc.) before they experiment with direct manipulations of unlabeled objects (tools, double clicking, moving of objects).
A well-labeled action will be especially salient.
Providing few actions in the search set can help to narrow the search if labeling cannot be provided, or if criteria for a "good" label are difficult to establish.
Set effects may prevent users to try untypical actions.
Users are reluctant to extend their search beyond the readily available menus and controls.
Frequently used interfaces techniques may bias users to search for them rather than for less frequent techniques.

15. Walkthrough Pros Easy to learn Can perform early in the design process
Questions assumptions about what a user may be thinking
Helps identify controls obvious to the designer but not a user
Helps identify difficulties with labels and prompts
Helps identify inadequate feedback

16. Walkthrough Cons Is diagnostic, not prescriptive
Focuses mostly on novice users
Designers must put themselves in the users mind
Focus specifically on task-related issues
The interactions are slower and not real
Does not provide quantitative results
A useful tool in conjunction with others

17. Walkthrough Example
I have a library book that needs to be returned today, but I am likely to forget. To help me remember, I want to set a reminder on my laptop. The reminder should display and beep at 5:00pm to remind me to return the book to the library.
Let’s walkthrough this task on my Outlook calendar and identify usability issues, if any

18. Walkthrough Example
Will a user try to produce the effect that the action has?
Will a user see the control for the action?
Will a user see that the control produces the desired effect?
Will a user select a different control instead?
Will a user understand the feedback to proceed correctly?

20. Your email address is invalid
Your address is invalid

21. What do you do with the results of the walkthrough?
Fix things.
make the controls more obvious
use labels that users will recognize
eliminate actions users don't think have to be performed or prompt for them.

22. Grading for Cognitive Walkthrough
User personas – 30 pts
Did they observe or analyze actual users doing related activities Grade _____ (15 pts max)   
Are the personas complete with relevant (and observed) personal details and goals Grade _____ (15 pts max)
Interface Description – 30 pts
Is there a description of a conceptual model?
Grade _____ (15 pts max)
A description or a low-fidelity prototype of the user interface. – 15 pts
Is the description detailed enough to evaluate cues, affordances, and feedback at each decision point Grade _____ (15 pts max)
Task descriptions – 40 pts
Are they concrete, detailed examples of all (or most) of the activities the system should support?
Grade _____ (20pts max)
A complete, written list of the actions needed to complete the task with the interface – 25 pts.
Do you have a step by step story of how each persona would accomplish each task with the proposed interface Grade _____ (20pts max)

23. Formal action analysis
GOMS approach (Goals Operators Methods Selection rules)
users apply specific selection rules for choosing between available methods in the pursuit of goals. Methods consist of operators.
The method breaks tasks in to smaller and smaller sub-goals until they can be accomplished with methods and operators of known durations. “keystroke-level analysis.”
Can be used to predict time it takes a skilled user to complete tasks.
Time consuming and difficult to do.
The analysis depends on the analysts.
Focuses on detail and expert performance.

24. Back-of-the-Envelope Action Analysis
List the actions and think about them.
List actions including mental actions
This will highlight long and difficult operations.

25. Heuristic Analysis Reference - Jakob Nielsen
Have several evaluators use the nine heuristics to identify problems with the interface, analyzing either a prototype or a paper description of the design.
Each evaluator should do the analysis alone.
Then combine the problems identified by the individual evaluators into a single list.
Heuristic Analysis Reference - Jakob Nielsen

26. Nielsen’s Ten Heuristics
Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Match between system and the real world
The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place.

27. Nielsen’s Ten Heuristics
Recognition rather than recall
Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

28. Evaluation Results The Need for Multiple Evaluators
Each row represents one of the 19 evaluators and each column represents one of the 16 usability problems. Each square shows whether the evaluator represented by the row found the usability problem represented by the column: The square is black if this is the case and white if the evaluator did not find the problem. The rows have been sorted in such a way that the most successful evaluators are at the bottom and the least successful are at the top. The columns have been sorted in such a way that the usability problems that are the easiest to find are to the right and the usability problems that are the most difficult to find are to the left.