We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLandon Crabtree
Modified over 3 years ago
©2011 1www.id-book.com An evaluation framework Chapter 13
©2011 2www.id-book.com The aims are: Introduce and explain the DECIDE framework. Discuss the conceptual, practical, and ethical issues involved in evaluation.
©2011 3www.id-book.com DECIDE: a framework to guide evaluation Determine the goals. Explore the questions. Choose the evaluation methods. Identify the practical issues. Decide how to deal with the ethical issues. Evaluate, analyze, interpret and present the data.
©2011 4www.id-book.com Determine the goals What are the high-level goals of the evaluation? Who wants it and why? The goals influence the methods used for the study. Goals vary and could be to: identify the best metaphor for the design check that user requirements are met check for consistency investigate how technology affects working practices improve the usability of an existing product
©2011 5www.id-book.com Explore the questions Questions help to guide the evaluation. The goal of finding out why some customers prefer to purchase paper airline tickets rather than e-tickets can be broken down into sub- questions: –What are customers attitudes to e-tickets? –Are they concerned about security? –Is the interface for obtaining them poor? What questions might you ask about the design of a cell phone?
©2011 6www.id-book.com Choose the evaluation approach & methods The evaluation method influences how data is collected, analyzed and presented. E.g. field studies typically: –Involve observation and interviews. –Involve users in natural settings. –Do not involve controlled tests. –Produce qualitative data.
©2011 7www.id-book.com Identify practical issues For example, how to: Select users Find evaluators Select equipment Stay on budget Stay on schedule
©2011 8www.id-book.com Decide about ethical issues Develop an informed consent form Participants have a right to: - Know the goals of the study; - Know what will happen to the findings; - Privacy of personal information; - Leave when they wish; - Be treated politely.
©2011 9www.id-book.com Evaluate, interpret & present data Methods used influence how data is evaluated, interpreted and presented. The following need to be considered: - Reliability: can the study be replicated? - Validity: is it measuring what you expected? - Biases: is the process creating biases? - Scope: can the findings be generalized? - Ecological validity: is the environment influencing the findings? i.e. Hawthorn effect.
© www.id-book.com Key points Many issues to consider before conducting an evaluation study. These include: goals of the study; involvment or not of users; the methods to use; practical & ethical issues; how data will be collected, analyzed & presented. The DECIDE framework provides a useful checklist for planning an evaluation study.
© www.id-book.com A project for you … Find an evaluation study from the list of URLs on this site or one of your own choice. Use the DECIDE framework to analyze it. Describe the aspects of DECIDE that are explicitly addressed in the report and which are not. On a scale of 1-5, where 1 = poor and 5 = excellent, how would you rate this study?
Chapter 13: An evaluation framework. The aims are: To discuss the conceptual, practical and ethical issues involved in evaluation. To introduce and explain.
©2011 An Evaluation Framework Chapter 13.
Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess CPE/CSC 484: User-Centered Design and.
An evaluation framework. The aims Explain key evaluation concepts & terms. Describe the evaluation paradigms & techniques used in interaction design.
DECIDE: An evaluation framework. DECIDE: a framework to guide evaluation D D etermine the goals. E E xplore the questions. C C hoose the evaluation approach.
User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly San Luis Obispo FJK 2005.
Um ambiente para avaliação. Objetivos Explicar conceitos e termos da avaliação Descrever paradigmas de avaliação e técnicas utilizadas no design de.
CS305: HCI in SW Development Evaluation (Return to…) We’ve had an introduction to evaluation. Now for more details on…
Chapter 11: An Evaluation Framework Group 4: Tony Masi, Sam Esswein, Brian Rood, & Chris Troisi.
Chapter 12/13: Evaluation/Decide Framework Question 1.
Chapter 12/13: Evaluation/Decide Framework. Why Evaluate? Why: to check that users can use the product and that they like it. Designers need to check.
Evaluation Framework Prevention vs. Intervention CHONG POH WAN 21 JUNE 2011.
Level 2 Prepared by: RHR First Prepared on: Nov 23, 2006 Last Modified on: Quality checked by: MOH Copyright 2004 Asia Pacific Institute of Information.
Virtual University - Human Computer Interaction 1 © Imran Hussain | UMT Imran Hussain University of Management and Technology (UMT) Lecture 30 Evaluation.
Oct 211 The next two weeks Oct 21 & 23: Lectures on user interface evaluation Oct 28: Lecture by Dr. Maurice Masliah No office hours (out of town) Oct.
©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.
Chapter 14: Usability testing and field studies. The aims: Explain how to do usability testing through examples. Outline the basics of experimental design.
Chapter 7 Design in the World of Business. Chapter 7 Design in the marketplace Featuritis Timeline for moving from design to product Incremental vs. radical.
An evaluation framework. Evaluation paradigm Any kind of evaluation is guided explicitly or implicitly by a set of beliefs, which are often under-pined.
Research Methods in Psychology (Pp 45-59). Observations Can be used in both experimental and nonexperimental research; can be used quantitatively or qualitatively.
©2011 1www.id-book.com Introducing Evaluation Chapter 12.
CS3205: HCI in SW Development Evaluation (Return to…) We’ve had an introduction to evaluation. Now for more details on…
Chapter 14: Usability testing and field studies. Usability Testing Emphasizes the property of being usable Key Components –User Pre-Test –User Test –User.
Chapter 13 WHAT IS INTERACTION DESIGN?. The aims Explain the key concepts and terms used in evaluation Introduce different types of evaluation methods.
Testing Drafts for Usability and Persuasiveness Paul V. Anderson’s Technical Communication, 6 th ed.
Chapter 14: Usability testing and field studies Question 1 Right – How to observe the real effects of your fishing equipment on the fish! From
Chapter 14: Usability testing and field studies. 2 FJK User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.
Observing Users Paul Bogen, LaShon Johnson, Jehoon Park.
Qualitative research methods in psychology (Case studies and Interviews only) Javana Hardman.
1.3 – Introduction to Experimental Design Vocabulary Census Sample Simulation.
Ch 14. Testing & modeling users Steven Pautz Lauren Sullivan Jessica Herron Chris Moore.
Chapter 14 Evaluation Studies: From Controlled to Natural Settings.
Action Research March 12, 2012 Data Collection. Qualities of Data Collection Generalizability – not necessary; goal is to improve school or classroom.
Evaluation How do we test the interaction design? Several Dimensions –Qualitative vs. Quantitative assessments –Conceptual vs. Physical Design.
Chapter 1 Getting Started Understanding Basic Statistics Fifth Edition By Brase and Brase Prepared by Jon Booze.
Explain the effect of triangulation on the credibility/trustworthiness of qualitative research.
Educational Research Chapter 3 Preparing and Evaluating a Research Plan Gay and Airasian.
Part 4: Evaluation Chapter 20: Why evaluate? Chapter 21: Deciding on what to evaluate: the strategy Chapter 22: Planning who, what, where, and when Chapter.
G544 – Practical project SELF REPORT. Revision Socrative quiz In pairs – answer each question. We will then discuss each answer given.
Data gathering. Overview Four key issues of data gathering Data recording Interviews Questionnaires Observation Choosing and combining techniques.
Chapter 6: Getting the Marketing Information We Need.
Identify and List… Theory behind the study Aim of study.
1.2 Understanding the Research Process PAGES 17 – 24.
Psychology I Fri, 3/2 Please get out your textbook, Ch 2, Sec 2 Outline, & Ch 1 Summative Assessment Did you know?? Scientific research does not always.
류 현 정류 현 정 Human Computer Interaction Introducing evaluation.
Development of Questionnaire By Dr Naveed Sultana.
© 2017 SlidePlayer.com Inc. All rights reserved.