1. SE 204, IES 506 – Human Computer Interaction Lecture 6: Evaluating Interface Designs Lecturer: Gazihan Alankuş 20.02.20121 Please look at the end of the presentation for assignments (marked with TODO)

2. © 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of Designing the User Interface: Strategies for Effective Human-Computer Interaction Fifth Edition Ben Shneiderman & Catherine Plaisant in collaboration with Maxine S. Cohen and Steven M. Jacobs CHAPTER 4: Evaluating interface Designs

3. Outline Weekly show and tell about interfaces that you use Talking about your experience with the homework Introduction to evaluating designs

4. Show and Tell What good/bad interfaces have you seen lately?

5. TCDD

6. Can you guess what kind of beds we have available? It wouldn’t be fun if we just told you now, would it? >:)

7. Others?

8. Your experience with the homework How did it go? What were the difficulties? Have you learned anything? What would you change next time?

9. Some hints about future ethnographic observations Don’t make up your mind before the user sessions! Make sure that most of the requirements come from users’ needs. Test your hypotheses with users!

10. What have we done so far? Identify a problem Identify requirements Create a prototype

11. What was left? >Evaluation IDEAS CODEDATA BUILDLEARN MEASURE

12. What to do after creating a prototype After creating a prototype, we need to learn from it – Evaluate the prototype See if it does what we thought it would If not, see how we can improve it – Use the prototype as a tool to make more effective ethnographic observation

13. A sample iteration to start collecting requirements IDEAS CODEDATA BUILDLEARN MEASURE There is a problem that needs to be solved with software These must be the users These seem to be the tasks We don’t know much, yet. Maybe we should observe them while they are in the tasks Create an environment to put users in this situation Use the situation to learn what is going on Observe Ask questions We saw that this and that happens during the task If our software did this, it would help them! Ethnographic Observation: Let’s create a paper prototype What are we going to measure? How are we going to measure it?

14. What do we need to do next? Some questions to answer through measurement – Is what we created any good? – Is it reflecting our intentions? Are we surprised? – Is it usable at all? – Can users use it the way that we assumed? – Does it handle the task that we built it for?

15. Some options You can evaluate it yourself You can get an expert to evaluate it You can evaluate it through user tests

16. Some options You can evaluate it yourself You can get an expert to evaluate it You can evaluate it through user tests

17. Evaluating the design by yourself You like what you created You really need to have a critical look You will probably fail to evaluate it adequately It’s difficult to let go – Need wisdom and humility

18. 1-18 © 2010 Pearson Addison-Wesley. All rights reserved. Expert Reviews While informal demos to colleagues or customers can provide some useful feedback, more formal expert reviews have proven to be effective Expert reviews entail one-half day to one week effort, although a lengthy training period may sometimes be required to explain the task domain or operational procedures There are a variety of expert review methods to chose from: –Heuristic evaluation –Guidelines review –Consistency inspection –Cognitive walkthrough –Metaphors of human thinking –Formal usability inspection 4-18

19. 1-19 © 2010 Pearson Addison-Wesley. All rights reserved. Expert Reviews (cont.) Expert reviews can be scheduled at several points in the development process when experts are available and when the design team is ready for feedback. Different experts tend to find different problems in an interface, so 3-5 expert reviewers can be highly productive, as can complementary usability testing. The dangers with expert reviews are that the experts may not have an adequate understanding of the task domain or user communities. Even experienced expert reviewers have great difficulty knowing how typical users, especially first-time users will really behave. 4-19

20. Some options You can evaluate it yourself You can get an expert to evaluate it You can evaluate it through user tests ^^

21. Usability Testing Observing users using the software Benefits – Confirms the progress made – Results in recommendations for changes

22. The emergence of usability testing Initially was just an interesting idea In time, companies understood how important it is – Evaluative feedback helped designers guide their work – Managers saw fewer disasters near deadlines – Sped up many projects – Reduced costs

23. 1-23 © 2010 Pearson Addison-Wesley. All rights reserved. Step-by-Step Usability Guide from http://usability.gov/ 4-23

24. Usability testing with paper prototypes You don’t have to develop the actual software and still get an idea of its usage You have great flexibility to change things at will, while doing the user tests. It’s more difficult with software prototypes. Let’s watch an example! – Usability testing with paper prototypes http://www.youtube.com/watch?v=9wQkLthhHKA – Usability testing for the Spore game http://www.youtube.com/watch?v=bz4C9XLdYzg 4-24

25. 1-25 © 2010 Pearson Addison-Wesley. All rights reserved. Usability Testing and Laboratories 4-25

26. 1-26 © 2010 Pearson Addison-Wesley. All rights reserved. Usability Testing and Laboratories (cont.) The emergence of usability testing and laboratories since the early 1980s Usability testing not only sped up many projects but that it produced dramatic cost savings. The movement towards usability testing stimulated the construction of usability laboratories. A typical modest usability lab would have two 10 by 10 foot areas, one for the participants to do their work and another, separated by a half-silvered mirror, for the testers and observers Participants should be chosen to represent the intended user communities, with attention to –background in computing, experience with the task, motivation, education, and ability with the natural language used in the interface. 4-26

27. 1-27 © 2010 Pearson Addison-Wesley. All rights reserved. Usability Testing and Laboratories (cont.) Participation should always be voluntary, and informed consent should be obtained. Professional practice is to ask all subjects to read and sign a statement like this one: –I have freely volunteered to participate in this experiment. –I have been informed in advance what my task(s) will be and what procedures will be followed. –I have been given the opportunity to ask questions, and have had my questions answered to my satisfaction. –I am aware that I have the right to withdraw consent and to discontinue participation at any time, without prejudice to my future treatment. –My signature below may be taken as affirmation of all the above statements; it was given prior to my participation in this study. Institutional Review Boards (IRB) often governs human subject test process 4-27

28. IRBs Ensures that what you are doing does not harm your users Protects your institution from lawsuits Originated from medical studies – What could go wrong in a usability study? (let’s watch an example) 4-28

29. 1-29 © 2010 Pearson Addison-Wesley. All rights reserved. Usability Testing and Laboratories (cont.) Videotaping participants performing tasks is often valuable for later review and for showing designers or managers the problems that users encounter. –Use caution in order to not interfere with participants –Invite users to think aloud (sometimes referred to as concurrent think aloud) about what they are doing as they are performing the task. Many variant forms of usability testing have been tried: –Paper mockups –Discount usability testing –Competitive usability testing –Universal usability testing –Field test and portable labs –Remote usability testing –Can-you-break-this tests 4-29

30. Steps of a usability test Preparation Introduction The test Collecting data through interviews and surveys We will dive deeper into these next week 30

31. TODO NO HOMEWORK!

32. TODO For IES 506 Keep sending the reports every week before class Give information about the undergraduate students that are helping you – If none, say so.

33. TODO Preparing for next week A future homework will be about using the paper prototypes you already created for usability testing. You may want to think about it in the meantime and get prepared mentally.