Cognitive Walkthrough More evaluating with experts

Review Discount evaluation techniques – Use experts to predict usability problems – Likely to be cheaper and faster with fewer ethical issues Heuristic evaluation – Experts identify potential problems in a design or prototype based on set of guiding heuristics

Cognitive Walkthrough Assess learnability and usability through simulation of way users explore and become familiar with interactive system A usability “thought experiment” Like code walkthrough (s/w engineering) From Polson, Lewis, et al at UC Boulder

Cognitive Walkthrough Qualitative Predictive With experts to examine learnability and novice behavior

CW: Process Construct carefully designed tasks from system spec or screen mock-up Walk through (cognitive & operational) activities required to go from one screen to another Review actions needed for task, attempt to predict how users would behave and what problems they’ll encounter

CW: Assumptions User has rough plan User explores system, looking for actions to contribute to performance of action User selects action seems best for desired goal User interprets response and assesses whether progress has been made toward completing task

CW: Requirements Description of users and their backgrounds Description of task user is to perform Complete list of the actions required to complete task Prototype or description of system

CW: Methodology Step through action sequence – Action 1 – Response A, B,.. – Action 2 – Response A –... For each one, ask four questions and try to construct a believability story

CW: Questions 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? (is it visible) 3. Once found, will they know it’s the right one for desired effect? (is it correct) 4. Will users understand feedback after action?

CW: Answering the Questions 1. Will user be trying to produce effect? – Typical supporting evidence It is part of their original task They have experience using the system The system tells them to do it – No evidence? Construct a failure scenario Explain, back up opinion

CW: Next Question 2.Will user notice action is available? – Typical supporting evidence Experience Visible device, such as a button Perceivable representation of an action such as a menu item

CW: Next Question 3.Will user know it’s the right one for the effect? – Typical supporting evidence Experience Interface provides a visual item (such as prompt) to connect action to result effect All other actions look wrong

CW: Next Question 4.Will user understand the feedback? – Typical supporting evidence Experience Recognize a connection between a system response and what user was trying to do

Let’s practice: My Internet Radio

User characteristics Technology savy users Familiar with computers Understand Internet radio concept Just joined and downloaded this radio

Task: add a station to presets Click genre Scroll list and choose genre Assuming station is on first page, add station to presets -- right-click on station, choose add to presets from popup menu. Click OK on Presets

Task: Click – Pick a genre 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? 3. Once found, will they know it’s the right one for desired effect? 4. Will users understand feedback after action?

Scroll list and choose genre 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? 3. Once found, will they know it’s the right one for desired effect? 4. Will users understand feedback after action?

Action: Right click on station and choose “Add to Presets” 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? 3. Once found, will they know it’s the right one for desired effect? 4. Will users understand feedback after action?

Action: Click OK 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? 3. Once found, will they know it’s the right one for desired effect? 4. Will users understand feedback after action?

Problems Did I pick the right task? Or list out the right sequence of actions?

CW Summary Advantages Explores important characteristic of learnability Novice perspective Detailed, careful examination Working prototype not necessary Disadvantages Can be time consuming May find problems that aren’t really problems Narrow focus, may not evaluate entire interface

Your turn Library - finding book What are our tasks? What are the actions?

CW: Questions 1. Will users be trying to produce whatever effect action has? 2. Will users be able to notice that the correct action is available? (is it visible) 3. Once found, will they know it’s the right one for desired effect? (is it correct) 4. Will users understand feedback after action?

CW: responsibilities Design team creates prototype, user characteristics Design team chooses tasks, lists out every action and response Experts answer 4 questions for every action/response Design team gathers responses and feedback Design team determines how to modify the design