We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byMackenzie Harbach
Modified over 2 years ago
©2011 1www.id-book.com Introducing Evaluation Chapter 12
©2011 2www.id-book.com The aims Explain the key concepts used in evaluation. Introduce different evaluation methods. Show how different methods are used for different purposes at different stages of the design process and in different contexts. Show how evaluators mix and modify methods. Discuss the practical challenges Illustrate how methods discussed in Chapters 7 and 8 are used in evaluation and describe some methods that are specific to evaluation.
©2011 3www.id-book.com Why, what, where and when to evaluate Iterative design & evaluation is a continuous process that examines: Why: to check users’ requirements and that users can use the product and they like it. What: a conceptual model, early prototypes of a new system and later, more complete prototypes. Where: in natural and laboratory settings. When: throughout design; finished products can be evaluated to collect information to inform new products.
©2011 4www.id-book.com Bruce Tognazzini tells you why you need to evaluate “Iterative design, with its repeating cycle of design and testing, is the only validated methodology in existence that will consistently produce successful results. If you don’t have user-testing as an integral part of your design process you are going to throw buckets of money down the drain.” See AskTog.com for topical discussions about design and evaluation.
©2011 5www.id-book.com Types of evaluation Controlled settings involving users, eg usability testing & experiments in laboratories and living labs. Natural settings involving users, eg field studies to see how the product is used in the real world. Any settings not involving users, eg consultants critique; to predict, analyze & model aspects of the interface analytics.
©2011 6www.id-book.com Usability lab http://iat.ubalt.edu/usability_lab/
©2011 7www.id-book.com Living labs People’s use of technology in their everyday lives can be evaluated in living labs. Such evaluations are too difficult to do in a usability lab. Eg the Aware Home was embedded with a complex network of sensors and audio/video recording devices (Abowd et al., 2000).
©2011 8www.id-book.com Usability testing & field studies can compliment
©2011 9www.id-book.com Evaluation case studies Experiment to investigate a computer game In the wild field study of skiers Crowdsourcing
©2011 10www.id-book.com Challenge & engagement in a collaborative immersive game Physiological measures were used. Players were more engaged when playing against another person than when playing against a computer. What precautionary measures did the evaluators take?
©2011 11www.id-book.com What does this data tell you?
©2011 12www.id-book.com Why study skiers in the wild ? Jambon et al. (2009) User experience in the wild. In: Proceedings of CHI ’09, ACM Press, New York, p. 4070-4071.
©2011 13www.id-book.com e-skiing system components Jambon et al. (2009) User experience in the wild. In: Proceedings of CHI ’09, ACM Press, New York, p. 4072.
©2011 14www.id-book.com Crowdsourcing-when might you use it?
©2011 15www.id-book.com Evaluating an ambient system The Hello Wall is a new kind of system that is designed to explore how people react to its presence. What are the challenges of evaluating systems like this?
©2011 16www.id-book.com Evaluation methods MethodControlled settings Natural settings Without users Observing x x Asking users x x Asking experts x x Testing x Modeling x
©2011 17www.id-book.com The language of evaluation Analytics Analytical evaluation Controlled experiment Expert review or crit Field study Formative evaluation Heuristic evaluation In the wild evaluation Living laboratory Predictive evaluation Summative evaluation Usability laboratory User studies Usability testing Users or participants
©2011 18www.id-book.com Key points Evaluation & design are closely integrated in user-centered design. Some of the same techniques are used in evaluation as for establishing requirements but they are used differently (e.g. observation interviews & questionnaires). Three types of evaluation: laboratory based with users, in the field with users, studies that do not involve users The main methods are: observing, asking users, asking experts, user testing, inspection, and modeling users’ task performance, analytics. Dealing with constraints is an important skill for evaluators to develop.
©2011 19www.id-book.com A project for you … “The Butterfly Ballot: Anatomy of disaster” was written by Bruce Tognazzini, and you can find it by going to AskTog.com and looking through the 2001 column. Alternatively go directly to: http://www.asktog.com/columns/04 2ButterflyBallot.html
©2011 20www.id-book.com A project for you … continued Read Tog’s account and look at the picture of the ballot card. Make a similar ballot card for a class election and ask 10 of your friends to vote using the card. After each person has voted ask who they intended to vote for and whether the card was confusing. Note down their comments. Redesign the card and perform the same test with 10 different people. Report your findings.
Introducing Evaluation Chapter 12. What is Evaluation? Assessing and judging Reflecting on what it is to be achieved Assessing the success Identifying.
Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess CPE/CSC 484: User-Centered Design and.
WHAT IS INTERACTION DESIGN?
Chapter 12: Introducing Evaluation. The aims To illustrate how observation, interviews and questionnaires that you encountered in Chapters 7 and 8 are.
Introducing evaluation. The aims Discuss how developers cope with real-world constraints. Explain the concepts and terms used to discuss evaluation. Examine.
1 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly San Luis Obispo FJK 2005.
Introducing Evaluation: why, what, when, where Text p Text p 317 – 323;
Chapter 12/13: Evaluation/Decide Framework Question 1.
Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check.
©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.
류 현 정류 현 정 Human Computer Interaction Introducing evaluation.
Usability Evaluation 1
Level 2 Prepared by: RHR First Prepared on: Nov 23, 2006 Last Modified on: Quality checked by: MOH Copyright 2004 Asia Pacific Institute of Information.
©2011 1www.id-book.com An evaluation framework Chapter 13.
Addition 1’s to 20.
©2011 1www.id-book.com Introducing Evaluation Chapter 12 adapted by Wan C. Yoon
Chapter 14: Usability testing and field studies
1 Unit 1 Kinematics Chapter 1 Day
User Interface Evaluation Introduction Lecture #15.
0 - 0.
Chapter 15: Analytical evaluation
Chapter 11 Design, prototyping and construction 1.
CS2003 Usability Engineering Human-Centred Design Dr Steve Love.
Fact-finding Techniques Transparencies
An evaluation framework
MULT. INTEGERS 1. IF THE SIGNS ARE THE SAME THE ANSWER IS POSITIVE 2. IF THE SIGNS ARE DIFFERENT THE ANSWER IS NEGATIVE.
CS305: HCI in SW Development Evaluation (Return to…)
Human Computer Interaction
Chapter 11: An Evaluation Framework Group 4: Tony Masi, Sam Esswein, Brian Rood, & Chris Troisi.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
SUBTRACTING INTEGERS 1. CHANGE THE SUBTRACTION SIGN TO ADDITION
1 Design, Prototyping, and Evaluation in Developing Countries Jen Mankoff, Assistant Professor EECS.
From Controlled to Natural Settings
Design in the World of Business
Science as a Process Chapter 1 Section 2.
Learning Outcomes Participants will be able to analyze assessments
1 Chapter 40 - Physiology and Pathophysiology of Diuretic Action Copyright © 2013 Elsevier Inc. All rights reserved.
Evaluation of User Interface Design
25 seconds left…...
Role Play Players: Owner/mgr of Small Business Spouse who works in the business Accountant for Small Business CRA auditor Audit Interview 20 questions.
DIVIDING INTEGERS 1. IF THE SIGNS ARE THE SAME THE ANSWER IS POSITIVE 2. IF THE SIGNS ARE DIFFERENT THE ANSWER IS NEGATIVE.
Test B, 100 Subtraction Facts
User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly San Luis Obispo FJK 2005.
© 2017 SlidePlayer.com Inc. All rights reserved.