We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLandon Crabtree
Modified over 7 years ago
©2011 1www.id-book.com An evaluation framework Chapter 13
©2011 2www.id-book.com The aims are: Introduce and explain the DECIDE framework. Discuss the conceptual, practical, and ethical issues involved in evaluation.
©2011 3www.id-book.com DECIDE: a framework to guide evaluation Determine the goals. Explore the questions. Choose the evaluation methods. Identify the practical issues. Decide how to deal with the ethical issues. Evaluate, analyze, interpret and present the data.
©2011 4www.id-book.com Determine the goals What are the high-level goals of the evaluation? Who wants it and why? The goals influence the methods used for the study. Goals vary and could be to: identify the best metaphor for the design check that user requirements are met check for consistency investigate how technology affects working practices improve the usability of an existing product
©2011 5www.id-book.com Explore the questions Questions help to guide the evaluation. The goal of finding out why some customers prefer to purchase paper airline tickets rather than e-tickets can be broken down into sub- questions: –What are customers attitudes to e-tickets? –Are they concerned about security? –Is the interface for obtaining them poor? What questions might you ask about the design of a cell phone?
©2011 6www.id-book.com Choose the evaluation approach & methods The evaluation method influences how data is collected, analyzed and presented. E.g. field studies typically: –Involve observation and interviews. –Involve users in natural settings. –Do not involve controlled tests. –Produce qualitative data.
©2011 7www.id-book.com Identify practical issues For example, how to: Select users Find evaluators Select equipment Stay on budget Stay on schedule
©2011 8www.id-book.com Decide about ethical issues Develop an informed consent form Participants have a right to: - Know the goals of the study; - Know what will happen to the findings; - Privacy of personal information; - Leave when they wish; - Be treated politely.
©2011 9www.id-book.com Evaluate, interpret & present data Methods used influence how data is evaluated, interpreted and presented. The following need to be considered: - Reliability: can the study be replicated? - Validity: is it measuring what you expected? - Biases: is the process creating biases? - Scope: can the findings be generalized? - Ecological validity: is the environment influencing the findings? i.e. Hawthorn effect.
©2011 10www.id-book.com Key points Many issues to consider before conducting an evaluation study. These include: goals of the study; involvment or not of users; the methods to use; practical & ethical issues; how data will be collected, analyzed & presented. The DECIDE framework provides a useful checklist for planning an evaluation study.
©2011 11www.id-book.com A project for you … Find an evaluation study from the list of URLs on this site or one of your own choice. Use the DECIDE framework to analyze it. Describe the aspects of DECIDE that are explicitly addressed in the report and which are not. On a scale of 1-5, where 1 = poor and 5 = excellent, how would you rate this study?
©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.
Chapter 13: An evaluation framework
Chapter 14: Usability testing and field studies
Chapter 5 Development and Evolution of User Interface
Key Steps to running a survey. Aims and Objectives Have clear aims and objectives for the project. Ensure you know what you want to get out of the survey.
©2011 1www.id-book.com Introducing Evaluation Chapter 12.
Imran Hussain University of Management and Technology (UMT)
DATA TRACKING AND EVALUATION 1. Goal of the STEP program: To increase the number of STEM graduates within the five-year period of the grant. You have.
Qualitative research methods in psychology (Case studies and Interviews only) Javana Hardman.
CS305: HCI in SW Development Evaluation (Return to…)
Research Methods in Psychology (Pp 45-59). Observations Can be used in both experimental and nonexperimental research; can be used quantitatively or qualitatively.
WHAT IS INTERACTION DESIGN?
Chapter 14: Usability testing and field studies. 2 FJK User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.
Observing Users Paul Bogen, LaShon Johnson, Jehoon Park.
Chapter 14: Usability testing and field studies. Usability Testing Emphasizes the property of being usable Key Components –User Pre-Test –User Test –User.
An evaluation framework
User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly San Luis Obispo FJK 2005.
Part 4: Evaluation Chapter 20: Why evaluate? Chapter 21: Deciding on what to evaluate: the strategy Chapter 22: Planning who, what, where, and when Chapter.
© 2021 SlidePlayer.com Inc. All rights reserved.