We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLandon Crabtree
Modified over 2 years ago
©2011 1www.id-book.com An evaluation framework Chapter 13
©2011 2www.id-book.com The aims are: Introduce and explain the DECIDE framework. Discuss the conceptual, practical, and ethical issues involved in evaluation.
©2011 3www.id-book.com DECIDE: a framework to guide evaluation Determine the goals. Explore the questions. Choose the evaluation methods. Identify the practical issues. Decide how to deal with the ethical issues. Evaluate, analyze, interpret and present the data.
©2011 4www.id-book.com Determine the goals What are the high-level goals of the evaluation? Who wants it and why? The goals influence the methods used for the study. Goals vary and could be to: identify the best metaphor for the design check that user requirements are met check for consistency investigate how technology affects working practices improve the usability of an existing product
©2011 5www.id-book.com Explore the questions Questions help to guide the evaluation. The goal of finding out why some customers prefer to purchase paper airline tickets rather than e-tickets can be broken down into sub- questions: –What are customers attitudes to e-tickets? –Are they concerned about security? –Is the interface for obtaining them poor? What questions might you ask about the design of a cell phone?
©2011 6www.id-book.com Choose the evaluation approach & methods The evaluation method influences how data is collected, analyzed and presented. E.g. field studies typically: –Involve observation and interviews. –Involve users in natural settings. –Do not involve controlled tests. –Produce qualitative data.
©2011 7www.id-book.com Identify practical issues For example, how to: Select users Find evaluators Select equipment Stay on budget Stay on schedule
©2011 8www.id-book.com Decide about ethical issues Develop an informed consent form Participants have a right to: - Know the goals of the study; - Know what will happen to the findings; - Privacy of personal information; - Leave when they wish; - Be treated politely.
©2011 9www.id-book.com Evaluate, interpret & present data Methods used influence how data is evaluated, interpreted and presented. The following need to be considered: - Reliability: can the study be replicated? - Validity: is it measuring what you expected? - Biases: is the process creating biases? - Scope: can the findings be generalized? - Ecological validity: is the environment influencing the findings? i.e. Hawthorn effect.
© www.id-book.com Key points Many issues to consider before conducting an evaluation study. These include: goals of the study; involvment or not of users; the methods to use; practical & ethical issues; how data will be collected, analyzed & presented. The DECIDE framework provides a useful checklist for planning an evaluation study.
© www.id-book.com A project for you … Find an evaluation study from the list of URLs on this site or one of your own choice. Use the DECIDE framework to analyze it. Describe the aspects of DECIDE that are explicitly addressed in the report and which are not. On a scale of 1-5, where 1 = poor and 5 = excellent, how would you rate this study?
Chapter 13: An evaluation framework. The aims are: To discuss the conceptual, practical and ethical issues involved in evaluation. To introduce and explain.
Chapter 14: Usability testing and field studies. The aims: Explain how to do usability testing through examples. Outline the basics of experimental design.
©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.
LETS INVESTIGATE. Whats it all about? Making predictions Testing predictions Making decisions Collecting evidence Selecting equipment Being aware of.
Assessment Photo Album Science Fair Project 6 – 8 th grade.
Chapter 12: Introducing Evaluation. The aims To illustrate how observation, interviews and questionnaires that you encountered in Chapters 7 and 8 are.
Evaluation of User Interface Design Evaluation is very important in User Interface Design and it is generally considered that there is no way round evaluation.
Copyright All Rights Reserved 1 Chapter 5 Development and Evolution of User Interface 5.1 User-Centered Design Aim of User center design Human-Computer.
Key Steps to running a survey. Aims and Objectives Have clear aims and objectives for the project. Ensure you know what you want to get out of the survey.
Evaluation Mary Rowlatt MDR Partners. Definition of project evaluation Evaluation focuses on whether the project was effective, achieved its objectives,
Manage an IT Project. Aim This presentation is prepared to support and give a general overview of the ‘How to Manage and IT Project’ Guide and should.
To achieve a Level 7 you need to…. To achieve a Level 6 you need to…
Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
Creating an Objective-based Syllabus Danielle Mihram, Director Center for Excellence in Teaching University of Southern California.
Selection and Formulation of Research Problem One of the most difficult phases of a research project is the choice of a suitable problem (true/false) A.
1 of 20 Evaluating an Information Project From Questions to Results © FAO 2005 IMARK Investing in Information for Development Evaluating an Information.
School Based Assessment and Reporting Unit Curriculum Directorate Assessment.
THE SCIENTIFIC METHOD 8 th science Science - is a systematic enterprise that builds and organizes knowledge in the form of testable explanations.
Performance Assessment. Performance assessment is a dynamic process calling for students to be active participants, who are learning even while they are.
Research Methods in Crime and Justice Chapter 12 Qualitative Research Methods.
Educational Research Chapter 22 Evaluating a Research Report Gay, Mills, and Airasian.
Growing Every Child! The following slides are examples of questions your child will use in the classroom throughout the year. The questions progress from.
Ciaran B. Trace MAC 2010 Symposium: Archival User Studies Dayton, Ohio, October 21-23, 2010.
Chapter 2 The Process of Experimentation 2.1 Designing and Conducting Agricultural Research.
Engineering Design Process Foundations of Technology Engineering Design Process © 2013 International Technology and Engineering Educators Association STEM.
FINDING OUT WHAT PEOPLE THINK “Quizzing the community.” Data Gathering techniques including Interviews, Surveys & Questionnaires
IB Design Cycle. Essential Question: How can we tell a story using photos? How can we tell a story using photos?
Lesson 11.2 Preparing to Buy July 2011Copyright © … REMTECH, inc … All Rights Reserved1 Introduction For many, the hardest part of buying a home.
Competency Approach to Human Resource Management.
© 2016 SlidePlayer.com Inc. All rights reserved.