Session 2: Understanding Program Theory and Implementation Theory.

Slides:

Advertisements

Similar presentations

Designing and Building a Results-Based Monitoring and Evaluation System: A Tool for Public Sector Management.

Advertisements

Poverty Monitoring - Good Practice in Selecting Indicators Presentation by Dirk U. Hahn, Consultant at the Workshop on Prioritization, Operationalization.

Setting internal Quality Assurance systems

Introduction to the Results Framework. What is a Results Framework? Graphic and narrative representation of a strategy for achieving a specific objective.

Introduction to Monitoring and Evaluation

Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.

Instructional System Design.  The purpose of instructional design is to maximize the value of instruction for the learner especially the learner's time.

M&E Issues: RAFIP and REP Kaushik Barua Accra, 12 Dec

Getting on the same page… Creating a common language to use today.

Session 6: Interviewing, Document Analysis, and Observation.

Objectives for Session Eleven Turn in Data Assignment Discuss Hypothetical Formative Evaluation Design Analyzing Qualitative Data Report Writing, Organization,

Objectives of Session Seven Complete in-class survey Discuss question formats and ordering Case study: face-to-face interviews v. self-administered questionnaires.

Logic Modeling for Success Dr Kathryn Wehrmann Illinois State University.

 To assess the learners achievement at the end of a teaching-learning process, for instance, at the end of the unit.  Measures the learners attainment.

Recreational Therapy: An Introduction

Objectives of Session Three Focus Group and Survey Assignment Formative Design Proposal Construct Validity Reliability in Data In-class Data Exercise.

Types of Evaluation.

TOOLS FOR POLICY DEVELOPMENT AND ANALYSIS Goals and Objectives.

La Naturaleza.  The generic term for the five-phase instructional design model consisting of Analysis, Design, Development, Implementation, and Evaluation.

+ Program Planning Chapter 2. + Individual and/or Program Cornerstones 1. Needs assessment** Needs assessment 2. Planning Planning 3. Implementation Implementation.

Monitoring & Evaluation in World Bank Agriculture and Rural Development Operations SASAR Monitoring & Evaluation Workshop New Delhi; June 20, 2006.

Release & Deployment ITIL Version 3

Session 12 Implementing and Evaluating the Campaign Session 12 Slide Deck Slide 12-1.

Product Evaluation the outcome phase. Do the magic bullets work? How do you know when an innovative educational program has “worked”? How do you know.

Prepared by: Rasha El Hagrassy Creating Cause-and-Effect Linkages 1. Develop objectives and measures for each of the four perspectives.  The business.

Edward M. Haugh Jr. ESC Consultant. III. Recommendations for Applying Outcomes Planning to ESC  I. Introduction to Outcomes Planning II. A Sample ESC.

Preparing for the Main Event Using Logic Models as a Tool for Collaboratives Brenda M. Joly Leslie M. Beitsch August 6, 2008.

Logistics and supply chain strategy planning

Evaluation Assists with allocating resources what is working how things can work better.

1 Delivery Directorate Delivery Balanced Scorecard.

LIBERIA THE BIG PICTURE Can the Agency tell the truth about results? Can the participants tell the truth about results?

TitleHow do you know if you have got it right? Evaluation and Indicators Professor Vanessa Burholt.

Human Learning Asma Marghalani.

The Impact of Alcohol Abuse

Overview of Evaluation ED Session 1: 01/28/10.

PPA 502 – Program Evaluation Lecture 2c – Process Evaluation.

 Read through problems  Identify problems you think your team has the capacity and interest to solve  Prioritize the problems and indicate the.

DESIGNING, IMPLEMENTING, AND EVALUATING CAREER DEVELOPMENT PROGRAMS AND SERVICES Career Development Interventions in the 21 st Century 4 th Edition Spencer.

Monitoring & Evaluation (M&E) 1. What is Monitoring? A continuing function that uses systematic collection of data on specified indicators to provide.

1 Nemours Health Planning and Evaluation Collaborative Learning Session I: Designing Effective Health Programs.

Policy Development Unit 7. 2 Policy Development l Policy: Authoritative guidelines that direct human behavior toward specific goals l Politics: Use of.

THE TRAINING STRATEGY MODEL BUSINESS GOALS Establish the priorities BUSINESS PROCESSES Tasks or business model for achieving the goals JOB ROLES Identify.

From Outcome To Output. Outcome  The intended or achieved short-term and medium-term effects of an intervention’s outputs,  usually requiring the collective.

0 ©2015 U.S. Education Delivery Institute While there is no prescribed format for a good delivery plan, it should answer 10 questions What a good delivery.

Session VI Evaluation & Wrap Up. Evaluation Activities directed at collecting, analyzing, interpreting, and communicating information about the effectiveness.

Connecticut Part C State Performance Plan Indicator 11 State Systemic Improvement Plan Phase II.

Module 1 Program Evaluation and Its Practice. 2 Overview n What is evaluation? n Definitions of program evaluation n What can be evaluated? n What is.

S3.1 session day 5 2 Programme management download resources from Approved by the Advisory Group: Programme management Programme and project.

California Primary Care, Mental Health, and Substance Use Services Integration Policy Initiative Delivery System Design Workgroup: Outcomes & Measurement.

Organizations of all types and sizes face a range of risks that can affect the achievement of their objectives. Organization's activities Strategic initiatives.

Evaluation What is evaluation?

Strategic Intent: Mission-Based Work Dave Higaki Tanya A. Hills, D.H.Ed. (c) February 3, 2016.

Allison Nichols, Ed.D. Extension Specialist in Evaluation.

MGT 307 Week 4 Learning Team Evaluating Performance Through Motivation and Conflict Management Discussion To purchase this material click on below link.

MGT 307 Week 5 Learning Team Organizational Trends Discussion To purchase this material click on below link

MGT 330 Week 5 Learning Team Control Mechanisms Paper To purchase this material click on below link

MGT 350 Week 5 Individual Decisions in Paradise Part III To purchase this material click on below link 350/MGT-350-Week-5-Individual-Decisions-in-

MGT 431 Week 3 Learning Team Development and Training Paper To purchase this material click below link

CPMGT 300 Week 5 Learning Team Evaluating the Scope Check this A+ tutorial guideline at

HRM 310 Week 5 Individual Implementing Change Report To purchase this material click on below link 310-Week-5-Individual-Implementing-Change-Report.

Module 2 Basic Concepts.

Strategic Planning for Learning Organizations

MGT 572 Competitive Success/snaptutorial.com

MGT 572 Education for Service-- snaptutorial.com

MGT 572 Teaching Effectively-- snaptutorial.com

Quality Improvement Indicators and Targets

Tit Albreht | Brussels | 7 November 2017

Managing the Planning Process

Slide Deck 9: Objectives and Results

How is an M & E framework derived from the logframe?

Presentation transcript:

Session 2: Understanding Program Theory and Implementation Theory

Objectives for Today Formative v. Summative Evaluations Program Theory v. Implementation Theory Example: STOP-DWI Stages of Formative Evaluation Assign Groups for Focus Group/Survey Project

Formative Evaluation What is being implemented? –Program description –Stages? Steps? Services? Procedures? Are the necessary steps in service delivery being adequately coordinated and executed? –Consistent with expectations or guidelines? –How do clients experience the program?

Summative Evaluation How well did the program achieve its intended impact? Are there any unintended or unforeseen consequences? Identify changes in conditions, behavior, or status that are directly related to the program

Formative or Summative? Number of visits to food pantry Satisfaction with job referral system Lower rates of heart disease Response time of operators to hotline callers Cleaner drinking water Citizen involvement in decision-making Sustained employment for 6 months Lower infant mortality rates

Program v. Implementation Theory Program Theory = mechanisms of change –“specifies a chain of causal assumptions linking program resources, activities, intermediate outcomes, and ultimate goals” –“the mechanisms that mediate between service delivery (and receipt) of the program and the emergence of the outcomes of interest” Implementation Theory = program activities –Translation of program objectives into services and an operating program

STOP-DWI in NY State In 1981, NY State passed STOP-DWI law to reduce drunken-driving and alcohol-related accidents. What is the implementation theory? –What activities or services comprise the STOP-DWI program? What is the program theory? –How will the program affect behavioral change? How would we know if it was being implemented well? What might affect successful implementation of STOP-DWI programs?

Outcomes of STOP-DWI in NY Revenues from DWI fines: –1982 fines = $156,313; 2000 fines = $925, alcohol-related fatalities in 1981, 2 alcohol-related fatalities in 2000 (–94%) -67 fatalities total in 1981, 46 in 2000 (-31%) DWI arrests in Onondaga County: –1980 = 1,069; 1990 = 2,191; 2000 = 1,461 Is STOP-DWI a success?

Outcomes of STOP-DWI in NY How do we know if the program is having an impact? Program v. non-program effects What factors affect the number of accident-related fatalities? What factors other than the program might affect drunk-driving rates?

Stages in Developing an Evaluation Stage One –Defining the Evaluation –Program goals, implementation theory, type of evaluation, stakeholders Stage Two - Develop Analysis Strategy –Tracking implementation and outcomes –Proper data for the question(s) of interest Stage Three - Implementation of Data Collection Stage Four - Analysis, Report Writing, and Presentation

Readings for Next Time: Trochim, Chapters 1 – 3 Also, in-Class Data Activity next Monday