We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byMackenzie Harbach
Modified about 1 year ago
©2011 1www.id-book.com Introducing Evaluation Chapter 12
©2011 2www.id-book.com The aims Explain the key concepts used in evaluation. Introduce different evaluation methods. Show how different methods are used for different purposes at different stages of the design process and in different contexts. Show how evaluators mix and modify methods. Discuss the practical challenges Illustrate how methods discussed in Chapters 7 and 8 are used in evaluation and describe some methods that are specific to evaluation.
©2011 3www.id-book.com Why, what, where and when to evaluate Iterative design & evaluation is a continuous process that examines: Why: to check users’ requirements and that users can use the product and they like it. What: a conceptual model, early prototypes of a new system and later, more complete prototypes. Where: in natural and laboratory settings. When: throughout design; finished products can be evaluated to collect information to inform new products.
©2011 4www.id-book.com Bruce Tognazzini tells you why you need to evaluate “Iterative design, with its repeating cycle of design and testing, is the only validated methodology in existence that will consistently produce successful results. If you don’t have user-testing as an integral part of your design process you are going to throw buckets of money down the drain.” See AskTog.com for topical discussions about design and evaluation.
©2011 5www.id-book.com Types of evaluation Controlled settings involving users, eg usability testing & experiments in laboratories and living labs. Natural settings involving users, eg field studies to see how the product is used in the real world. Any settings not involving users, eg consultants critique; to predict, analyze & model aspects of the interface analytics.
©2011 6www.id-book.com Usability lab
©2011 7www.id-book.com Living labs People’s use of technology in their everyday lives can be evaluated in living labs. Such evaluations are too difficult to do in a usability lab. Eg the Aware Home was embedded with a complex network of sensors and audio/video recording devices (Abowd et al., 2000).
©2011 8www.id-book.com Usability testing & field studies can compliment
©2011 9www.id-book.com Evaluation case studies Experiment to investigate a computer game In the wild field study of skiers Crowdsourcing
© www.id-book.com Challenge & engagement in a collaborative immersive game Physiological measures were used. Players were more engaged when playing against another person than when playing against a computer. What precautionary measures did the evaluators take?
© www.id-book.com What does this data tell you?
© www.id-book.com Why study skiers in the wild ? Jambon et al. (2009) User experience in the wild. In: Proceedings of CHI ’09, ACM Press, New York, p
© www.id-book.com e-skiing system components Jambon et al. (2009) User experience in the wild. In: Proceedings of CHI ’09, ACM Press, New York, p
© www.id-book.com Crowdsourcing-when might you use it?
© www.id-book.com Evaluating an ambient system The Hello Wall is a new kind of system that is designed to explore how people react to its presence. What are the challenges of evaluating systems like this?
© www.id-book.com Evaluation methods MethodControlled settings Natural settings Without users Observing x x Asking users x x Asking experts x x Testing x Modeling x
© www.id-book.com The language of evaluation Analytics Analytical evaluation Controlled experiment Expert review or crit Field study Formative evaluation Heuristic evaluation In the wild evaluation Living laboratory Predictive evaluation Summative evaluation Usability laboratory User studies Usability testing Users or participants
© www.id-book.com Key points Evaluation & design are closely integrated in user-centered design. Some of the same techniques are used in evaluation as for establishing requirements but they are used differently (e.g. observation interviews & questionnaires). Three types of evaluation: laboratory based with users, in the field with users, studies that do not involve users The main methods are: observing, asking users, asking experts, user testing, inspection, and modeling users’ task performance, analytics. Dealing with constraints is an important skill for evaluators to develop.
© www.id-book.com A project for you … “The Butterfly Ballot: Anatomy of disaster” was written by Bruce Tognazzini, and you can find it by going to AskTog.com and looking through the 2001 column. Alternatively go directly to: 2ButterflyBallot.html
© www.id-book.com A project for you … continued Read Tog’s account and look at the picture of the ballot card. Make a similar ballot card for a class election and ask 10 of your friends to vote using the card. After each person has voted ask who they intended to vote for and whether the card was confusing. Note down their comments. Redesign the card and perform the same test with 10 different people. Report your findings.
Chapter 12: Introducing Evaluation. The aims To illustrate how observation, interviews and questionnaires that you encountered in Chapters 7 and 8 are.
Introducing evaluation. The aims Discuss how developers cope with real-world constraints. Explain the concepts and terms used to discuss evaluation. Examine.
©2011 1www.id-book.com An evaluation framework Chapter 13.
Evaluation of User Interface Design Evaluation is very important in User Interface Design and it is generally considered that there is no way round evaluation.
Chapter 14: Usability testing and field studies. The aims: Explain how to do usability testing through examples. Outline the basics of experimental design.
©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.
Chapter 13: An evaluation framework. The aims are: To discuss the conceptual, practical and ethical issues involved in evaluation. To introduce and explain.
Chapter 15: Analytical evaluation. Aims: Describe inspection methods. Show how heuristic evaluation can be adapted to evaluate different products. Explain.
Lesson 8 Data Toss. Key Vocabulary Data: Factual information derived from scientific experiments Hypothesis A possible explanation for observations, facts,
Research Methods in Crime and Justice Chapter 12 Qualitative Research Methods.
PS50118 – Interacting with Technology Laboratory vs. Field Usability Evaluation Jason Cooper.
Copyright: ©2005 by Elsevier Inc. All rights reserved. 1 Author: Graeme C. Simsion and Graham C. Witt Chapter 10 Logical Database Design.
Computing Higher - SD Process – Topic 2 St Andrew’s High School Unit 2 Software Development Process.
1 Monitoring and Evaluation What are we talking about? Module 5 Session 8.
©Ian Sommerville 2006Software Engineering, 8th edition. Chapter 22 Slide 1 Verification and Validation.
+ Privileged & Confidential Page 1 + Rich User Experience Documentation John Yesko.
Nica Valentin–Danut SEM 2012 Service system fundamentals: Work system, value chain, and life cycle.
Evaluating health informatics projects Reasons for and problems of evaluation Objective model Subjective model.
©2011 1www.id-book.com What is interaction design? Chapter 1.
Using UML, Patterns, and Java Object-Oriented Software Engineering Chapter 4, Requirements Elicitation.
1 Holistic Scoring Using ACPS Measurement Topic Rubrics Curriculum Implementation Modules ( ) Session Four: December 2011 Alexandria City Public.
DT Coursework By D. Henwood. Section √ √ Problem and analysis (with Photos/drawings)Development: Concept Modelling Confirmation of the problem: Questionnaire.
©2011 1www.id-book.com Design, prototyping and construction Chapter 11.
Final Year Projects Some tips
1 Science as a Process Chapter 1 Section 2. 2 Objectives Explain how science is different from other forms of human endeavor. Identify the steps that.
What is Interaction Design?. What is interaction design? Designing interactive products to support people in their everyday and working lives –Sharp,
©2011 1www.id-book.com Understanding and Conceptualizing interaction Chapter 2.
The Framework for Teaching Charlotte Danielson 3d: Using Assessment in Instruction 1.
2012 SQA Central Training (New Centers) Venue – BISU, Beijing 9, Sept am– 5:00 pm Mary Gao.
© 2016 SlidePlayer.com Inc. All rights reserved.