Extended Cognitive Walkthrough Judy Kay CHAI: Computer human adapted interaction research group School of Information Technologies

growing set of methods: HE, TA, CW

3 Overview Predictive method Cognitive Walkthrough Benefits Disadvantages 251,000 results from Google, 10,800 from Google Scholar

4 Focus on learnability Appropriate for novice or casual users cf Think-Aloud? Does not focus on speed cf Think-Aloud? Sequence is not known prior to inspection Assesses user success and recovery from errors Conducted by experts cf Think-Aloud?

5 Model of Exploratory Learning User has task –0. will user understand this sub-task is needed? Explores system for useful elements –1. will correct action be obvious? eg button visible –2. will user understand instructions? eg user understands the label on the button Selects one to try User interprets system response –3. will user know if progress has been made?

Extended Cognitive Walkthrough Takes account of mental model…..

7 User has task –0. will user understand this sub-task is needed (given their mental model)‏ Explores system for useful elements –1. will correct action be obvious? –(given their mental model)‏ –2. will user understand instructions? – (given their mental model)‏ Selects one to try User interprets system response –3. will user know if progress has been made? –(given their mental model)‏

How to add a mental model? models/

What users and user beliefs matter for this context? “A mental model is what the user believes about the system at hand. based on belief, not facts … they base their predictions [and actions] about the system on their mental models … Individual users each have their own mental model. … one of usability's big dilemmas is the common gap between designers' and users' mental models. … mental models are in flux

10 Example with basic CW Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training Abstract user goals: Buy a ticket to the required destination Determine whether I can afford to buy the ticket to a particular destination Adapted from Newman and Lamming, Interactive System Design, 1995 )‏

11 Example with basic CW Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training Designer goal breakdown to subgoals: Determine fare to pay –indicate destination –indicate one-way or return Dispense ticket –pay money –get ticket and change Adapted from Newman and Lamming, Interactive System Design, 1995 )‏

12 The interface – lo-fi prototype

Concrete user task Class activity: define 3 concrete tasks

14 The interface – lo-fi prototype

Concrete user task Buy a one-way ticket to Town Hall

16 The task: Buy a one-way ticket to Town Hall

17 How designer wants it to work – method 1 where use does not use keypad Click destination ie Town Hall Click journey type ie one way

18 0. Task? –Buy one-way ticket to Town Hall 1. Is correct action obvious? –Two possibilities: Destination One way / return –Will user know both must be set?

19 How to Help the User? Indicate extra information needed Indicate steps 1 and 2 can be done in either order Give some feedback on effect of each select action Reorganise layout so that Steps 1 and 2 are followed by the fare display

We alter our lo-fi prototype and check this

21 2. Will user understand instructions? 3. Will user interpret machine action correctly? We alter our lo-fi prototype and check this

22 2. Will user understand instructions? –Yes – due to labels and instructions 3. Will user interpret machine action correctly? –Yes (buttons light up, new state appears)‏

23

24 1. Correct action obvious? –Yes 2. Will user understand instructions? –Yes 3. Will user interpret machine action correctly? –Yes

25 Paying 1. Correct action obvious? 2. Will user understand instructions? 3. Will user interpret machine action correctly?

What are the cases to consider for payment (in cash) Consider case where user has exact change

27 Paying Designer intends user to Pay in money and click Click “4. Press for ticket” User then lifts flap to collect the ticket

28 Paying – exact change case 1. Correct action obvious? –Yes 2. Will user understand instructions? –Yes 3. Will user interpret machine action correctly? –Unclear (no feedback on money accepted so far)‏

We alter our lo-fi prototype and check this

30 Revised design gives feedback on amount paid so far

Consider other cases eg user realises they have insufficient money

32 Buying a Ticket: Insufficient Money? 1. Correct action obvious?

33 Summary of Flaws (so far)‏ Confusion about need for steps 1 and 2 No feedback on amount deposited No means to get money back So far …...

34 Goals and Tasks In this example: Goal: buy a ticket Sub-goals: (determine fare) and pay User tasks: concrete cases used in CW

35 Extended cognitive walkthrough Adding user's mental model What does user believe? How do you find this out? What did we assume about the user’s mental model? What differences are there in the MM for: A novice user An expert user

Class activity: List aspects of the user’s MM that would be relevant to the train ticket interface

What are the implications of some likely cases: user familiar with existing interface user familiar with a different bus ticket interface

Class activity on Extended CW:

39 Interface is COMP5427website Task COMP5427student has just started semester and wants to determine the deadline for the first assessed work Define the relevant mental model

40 Mental Model At start of semester….. There is a lecture and a lab each week for most classes There is assessed practical work There are fixed deadlines for such work Actually a 2-way street – think about potential mental model elements as you work through the CW

41 User has task –0. will user understand this sub-task is needed (given their mental model)‏ Explores system for useful elements –1. will correct action be obvious? –(given their mental model)‏ –2. will user understand instructions? – (given their mental model)‏ Selects one to try User interprets system response –3. will user know if progress has been made? –(given their mental model)‏

42 Potential GOTCHAs Need a prototype that is complete enough for a walkthough of an interesting concrete case But this is a technique for early in design process, where designer is ready to change it You then need to define: the user groups, and their relevant MMs a good set of tasks the correct steps (intended by designer) Evaluator must imagine people’s thoughts on first use of UI Keep referring to the mental model Carefully assess vocabulary/text in terms of mental model Repeat process over the tasks, and for each important class of mental model

How does CW actually work in industry? [In this subject, we focus on the pure form, which is purely predictive, and very useful in the hands of novice usability practitioners Week 8 reading is actual industry case]

Case study Moded interfaces: The same interface action has a different meaning depending upon the context Class activity: define some examples of moded interaction in common interfaces eg text editor, Powerpoint, Word What are the challenges for the user? In terms of their mental model as novice users? In terms of the available conceptual model of the system?

45 Summary of uses of CW Relatively inexpensive in our very, very lightweight approach Desk check –No users –Better with expert evaluators Generally applicable Novice, casual, intermittent users Focus on learnability

46 Summary of usefulness Really useful technique, even for designer Better if done by Outsider Expert But students and non-expert evaluators still can gain from using it Part of early usability evaluation because Low cost No users needed