1. Week 5 – 2015 HCI – COMP3315 Judy Kay CHAI: Computer Human Adapted Interaction research group Human Centred Technology Cluster for Teaching and Research School of Information Technologies

2. Looking back a little….

3. Some nice personas from previous students

6. Think-alouds The usability tool

7. But before we start TA….

8. Mental models Relevant things that a person believes Designer –How the system works –About the user’s mental model User(s) –Beliefs about the system What it does What the interface elements mean Relevant background knowledge

9. Concrete example: “Cannot save file…. No space”

12. http://normfujisaki.com/wp-content/uploads/2013/09/mental- model.jpg

13. Note: – Beliefs …. Not facts – Mental models drive predictions, planning actions – Individual MMs differ …. Designer MM versus “user” MM – MMs change….

14. Mental models http://www.nngroup.com/articles/mental-models/ Nielsen highlights: – Strongly impacts all aspects of interaction – Mismatched mental models are common, especially with designs that try something new What mismatches can there be?

15. Class activity Make a table of 10 core elements of the mental model for CUSP One row per concept: eg university subject, university degree, prerequisite One column each for: designers, high school student, info3315 student And in the cell for each indicate if they “know about” that concept Note: Try to think of beliefs/concepts that are understood differently by these groups

16. Rows -- concepts Columns -- 1. designers, 2. high school student, 3. info3315 student

17. Why do mental models matter for think-aloud? You will be able to answer this soon….

18. How to address mental model problems Fix the system – make it match users' mental models – eg If people look for something in the wrong place, then move it to the place where they look for it. Fix the user - improve users' mental models so that they more accurately reflect your system. – eg explaining things better – making labels clearer to make the UI more transparent (even though the underlying system remains unchanged) – Help that insists that the user pays attention

19. Class activity How do you discover a person’s mental model?

20. Some useful distinctions Slips: – correct user model, inadvertent incorrect action – eg car “malfunction”, tap caps lock accidentally Mistakes: incorrect mental model Use this tightened vocabulary for your think- aloud reports

21. The main act… Of the lecture and the semester

22. Be ready to comment on what you conclude from the next slide

23. "Thinking aloud may be the single most valuable usability engineering method." I wrote this in my 1993 book, Usability Engineering, and I stand by this assessment today. The fact that the same method has remained #1 for 19 years is a good indication of the longevity of usability methods.Usability Engineering Thinking Aloud: The #1 Usability Tool by Jakob Nielsen on January 16, 2012Jakob Nielsen

24. So how do you do a think-aloud? As you will by now be looking forward to doing in Assignment 2…

25. Demonstrating think-aloud Demonstrate Thinking Aloud by Showing Users a Video by Jakob Nielsen on September 1, 2014Jakob Nielsen Topics: User TestingUser Testing

26. Services Userzoom “can run unmoderated task-based studies with geographically dispersed participants over any web- based interface (website, prototype, mock-up). Participants take the study simultaneously, in their natural context, using their own PC or device.” Users think User testing – Let’s watch their video, noting the tasks and critiquing them (just one in class) Let’s watch their video

27. Think aloud protocols Ask user to “think aloud” as they use the interface Role of video, audio taping Importance of a team member to take notes Think aloud Helps observer interpret what is going on Gives access to user’s “mental model” Gives qualitative data mainly Gives very useful but only approximate timing data

28. Facilitating think aloud makes you – experimenter -- really valuable What are you thinking now? What do you think that message means? (only after the user has noticed the message and is clearly spending time on it)‏ don't help user except with How do you think you can do it? if user appears surprised, Is that what you expected to happen?

29. Overview This is an example of an empirical method Absolutely relies on defining good tasks for the user to do Abstract versus concrete Task design and avoiding leading the user Preliminaries for a study Conducting the study Making use of the data

30. Why do mental models matter for think-aloud? You will be able to answer this soon….

31. Design cycle While your budget can fund another iteration do: User Centred Design Define criteria for success Define abstract and matching concrete tasks users should be able to do Prototype construction (Assignment 1 is conceptual prototype….) Usability study –Decide just what data to collect –Test design of experiment for timing (trial it)‏ –Recruit users –Run study done

32. First things first Do we have the tasks right? Abstract tasks Concrete instances of them – Concrete? – Relevant? – Not lead the user? – Minimalist? – Good coverage?

33. Class activity: - define some core tasks for testing this part of CUSP

34. Recruiting users How representative are they? –similarity to intended user population –Age –Gender –experience in area –interest/motivation –computer literacy What effect does user population have for conclusions?

35. How many users? Why You Only Need to Test with 5 Users by Jakob Nielsen on March 19, 2000Jakob Nielsen http://www.nngroup.com/articles/why-you- only-need-to-test-with-5-users/

36. Insights: 0 users? 1 user? 2 users? http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

37. What does this really mean? If first test shows catastrophic problems, should you still do 5? Never bother with more than 5? 80% is good enough? NO! NO!

38. It is all about budgets, user groups…. If your interface is to be used by very different groups of people, you need to do the think- aloud with each group eg. – Children – Elderly – Different cultures, languages…… Iterate!!!!! – Use same budget of effort to test each iteraction

39. “In the early 1990s … fairly large-sample formative usability studies … determined that the … first five participants observed in a formative usability study should usually reveal about 85% of the problems available for discovery in that iteration, where the properties of the study (type of participants and tasks employed) place limits on what is discoverable. But over time, in the minds of many usability practitioners, the rule became simplified to “All you need to do is watch five people to find 85% of a product’s usability problems.”

40. Formative evaluation As in your Assignment 2 How many participants? User populations? – Within community interested in UX – Those who are not – Ages, broad range

41. Summative evaluation How many participants? – Why? User populations? – Within community interested in UX – Those who are not – Ages, broad range

42. Stages of running an evaluation 1.Preparation 2.Introduction 3.The test 4.Questionnaire/interview 5.Debriefing 6.Analysis, reflection, summarising, reporting, conclusions for action Steps 1- 5 done for each user test, as run Step 6 is applied mainly after several users

43. Preparation Materials for consent Set up machine, room, environment Check all of them Check user instructions Do a mental run-through Be sure not to waste user's time because of your lack of preparation!

44. Call for volunteer to do the think- aloud of CUSP Whole class acts as observers, making notes

45. Introduction Welcome user, explain purpose of test –make clear system tested not user –confidentiality –anonymity of reporting –opt out at any time –what is recorded Invite any other questions to here –explain procedure –if appropriate, do demo –invite questions

46. The test User works through experiment.... –recording –ensure user feels supported –show pleasure at problems identified –critical to help user if stuck Questionnaire/interview –open and closed

47. What data should you collect? Observe –direct/indirect –take notes –video/audio/software monitor –software logs for timing Questionnaire: –Open –Best things about –If you could change one thing about, what would it be? –Closed (later in the semester only)

49. Debriefing Thank user Remind them of usefulness of results Pause to make sure all data collected All notes written May ask user to confirm details collected

50. Analysis and reporting each aspect, time, success……

51. Pitfalls Defining the right concrete tasks –Test all key aspects –Multiple tasks for same aspects Instructions to the users –Do NOT lead the user –Take particular care not to use words that are identical to terms on the interface

52. Benefits of think aloud “show what users are doing and why they are doing it while they are doing it in order to avoid later rationalisations” (Nielsen, Usability Engineering, Academic press 1993, p195)‏ Cheap Slows users down –studies show users may work faster with fewer errors due to care on critical elements

53. Problems of think aloud Unnatural context and situation (do you talkto yourself?) People filter, want to please, do not want to look foolish or inept Hawthorn effect Experimenter can bias results Directly eg via task choice Inadvertently eg gasp, brief frown Not directly quantitative Add cognitive load to users User's “theories” must be interpreted with care Slows users down Users are aware they are being observed so behave accordingly

54. Naturalised think-aloud Multi-user interaction –Two (or more) users work on task –Conversation is natural –Observer collects dialogue

55. Summary Top method for formative evaluations Also good for summative but needs more users Relatively inexpensive Can identify major flaws And may indicate causes of user problems May give access to user's mental model Alters activity => meaningfulness This is a major part of your next assignment