Presentation on theme: "REHABILITATION ENGINEERING RESEARCH CENTER ON TECHNOLOGY TRANSFER (T 2 RERC) TRAINING MODULES ON TECHNOLOGY TRANSFER MODULE 8: PROJECT TRACKING AND EVALUATION."— Presentation transcript:
REHABILITATION ENGINEERING RESEARCH CENTER ON TECHNOLOGY TRANSFER (T 2 RERC) TRAINING MODULES ON TECHNOLOGY TRANSFER MODULE 8: PROJECT TRACKING AND EVALUATION
Objective of the Module: Participants will: explain basic concepts and terms used in program evaluation/research; define indicators of effectiveness and efficiency for a project of their interest suggest methods to track the indicators; identify how the results would be used for project monitoring and evaluation
BASIC CONCEPTS AND TERMS Research and Evaluation Merit and Worth Management Orientation: Evaluation for Development Effectiveness and Efficiency Formative and Summative Evaluation Outcome and Impact
Merit and Worth Merit refers to intrinsic quality of person/ product/ system its potential, or promise to be “valued” in its broader context. Ex: Program inputs (resources, design, theory); processes (operations) and outcomes indicate its merit Worth refers to extrinsic value of person/ product/ system its actual “value” in the broader context Ex: program effects/ impacts on internal and external environments indicate its worth.
Arewe there yet? Recycle output or disseminate? How to do it? [Defining resources] Input Evaluation ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ →→ What is the optimal path to get to goal? [process improvement ] ↑ ↑ →→ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↓ ↓ ↑ ↑ ↑ ↑↑ ↑ → → →→→→→ Process Evaluation →→→→→ → →→
INPUT ---- PROCESS ---- OUTPUT ↑↓ ← ← ← ← FEEDBACK Effective: gets outcome at the desired quality/quantity level Efficient: Maximizes effectiveness, minimizes cost Efficiency =Output/ Input
Formative vs. Summative Evaluation FORMATIVE Assesses quality of outcome during the process, while outcome is taking shape Provides basis for improving/maintaining program actions SUMMATIVE Assesses quality of outcome at the end of process when outcome is in final stage Provides basis for establishing program effectiveness & efficiency; [accountability]
Outcomes and Impacts Outcomes direct outputs or immediate effects assess program’s performance- its effectiveness and efficiency; an indicator of merit Impacts long term effects; benefits to staff, participants, other stakeholders and community at large assess program’s validity/relevance; indicator of worth
Framing program evaluation: three basic questions Is the program effective? - has it achieved its goals? Is it efficient? - did it optimize its methods, resources to achieve the goals? Is it valid? - are its goals relevant to the context? Is it valued by its stakeholders and beneficiaries?
In Response: what to assess Effective? --- Outcome quality; outcome quantity (for duration of program) Efficient? -- Cost per outcome; Time per outcome Valid? Worth it? -- internal and external impacts; long term effects on/ benefits to stakeholders and others involved NOTE: outcome,cost/time [merit indicators] - assessed through tracking program performance. Impact [worth indicator]: - assessed through tracking effects beyond program period
Use of Information Monitoring project Performance –Improve intermediary outcomes [quality vs. timelines] –Improve process re-distribute personnel review protocols/timelines Describing performance level - [effectiveness and efficiency]
Related Issues Questions, Comments, Suggestions
Workshop Session Two For your project, identify, or discuss applicability of: Goals, Protocols, Action Plan (or similar framework) Expected Outcomes - final [and intermediary, if applicable] Outcome Tracking: (a) process and (b) use- improving outcomes (formative eval.) and finding effectiveness (summative eval.) Tracking resource utilization (time/cost …): (a) process and (b) use for process improvement and finding efficiency (summative eval.) Instrument and database alternatives