Presentation on theme: "MODULE 8: PROJECT TRACKING AND EVALUATION"— Presentation transcript:
1 MODULE 8: PROJECT TRACKING AND EVALUATION REHABILITATION ENGINEERING RESEARCH CENTER ON TECHNOLOGY TRANSFER (T2 RERC) TRAINING MODULES ON TECHNOLOGY TRANSFER
2 Objective of the Module: Participants will:explain basic concepts and terms used in program evaluation/research;define indicators of effectiveness and efficiency for a project of their interestsuggest methods to track the indicators;identify how the results would be used for project monitoring and evaluationSlides 13-15
3 BASIC CONCEPTS AND TERMS Research and EvaluationMerit and WorthManagement Orientation: Evaluation for DevelopmentEffectiveness and EfficiencyFormative and Summative EvaluationOutcome and Impact
5 Merit and Worth Merit refers to intrinsic quality of person/ product/ systemits potential, or promise to be “valued” in its broader context.Ex: Program inputs (resources, design, theory); processes (operations) and outcomes indicate its meritWorth refers toextrinsic value of person/ product/ systemits actual “value” in the broader contextEx: program effects/ impacts on internal and external environments indicate its worth.
7 Fig 2. Evaluation and the Planning Cycle ↓←ContextEvaluationCyclePlanning Cycleß↑What needsto be done?[Definingobjective]ProductÞArewe there yet?How to do it?Recycle output or[DefiningÜInputdisseminate?resources]Evaluation↓↓What is the↑↑↓↓optimal path↑↑↓→→to get to→→↑↓goal?↑↓[process↑↓improvement↑↓]↑↑↓Ý↑↓↑ ↑→→→→→→→Process→→→→→→→→Evaluation
8 System characteristics: INPUT ----àPROCESS----àOUTPUT↑↓←← ← ← ← ← ←FEEDBACKEffective: gets outcome at the desiredquality/quantity levelEfficient: Maximizes effectiveness, minimizes costSystem characteristics:.Uses the “input” resources;.“Processes” them [facilitates and manages the dynamics of their interaction];.In function of a pre-defined objective [i.e., effort directed to achieving the corresponding output].Does “self-regulation”[i.e., checks/obtains feedback information about the product taking shape, constantly adjusts its processes so the desired quality of product is obtained].Efficiency =Output/ InputSlides 13-15
9 Formative vs. Summative Evaluation Assesses quality of outcome during the process, while outcome is taking shapeProvides basis for improving/maintaining program actionsSUMMATIVEAssesses quality of outcome at the end of process when outcome is in final stageProvides basis for establishing program effectiveness & efficiency; [accountability]
10 Outcomes and Impacts Outcomes direct outputs or immediate effects assess program’s performance- its effectiveness and efficiency; an indicator of meritImpactslong term effects; benefits to staff, participants, other stakeholders and community at largeassess program’s validity/relevance; indicator of worth
11 Framing program evaluation: three basic questions Is the program effective? - has it achieved its goals?Is it efficient? - did it optimize its methods, resources to achieve the goals?Is it valid? - are its goals relevant to the context? Is it valued by its stakeholders and beneficiaries?
12 In Response: what to assess Effective? --- Outcome quality; outcome quantity (for duration of program)Efficient? -- Cost per outcome; Time per outcomeValid? Worth it? -- internal and external impacts; long term effects on/ benefits to stakeholders and others involvedNOTE: outcome,cost/time [merit indicators] - assessed through tracking program performance. Impact [worth indicator]: - assessed through tracking effects beyond program period
22 Use of Information Monitoring project Performance Improve intermediary outcomes [quality vs. timelines]Improve processre-distribute personnelreview protocols/timelinesDescribing performance level -[effectiveness and efficiency]
23 Questions, Comments, Suggestions Related IssuesQuestions, Comments, Suggestions
24 Workshop Session TwoFor your project, identify, or discuss applicability of:Goals, Protocols, Action Plan (or similar framework)Expected Outcomes - final [and intermediary, if applicable]Outcome Tracking: (a) process and (b) use- improving outcomes (formative eval.) and finding effectiveness (summative eval.)Tracking resource utilization (time/cost …): (a) process and (b) use for process improvement and finding efficiency (summative eval.)Instrument and database alternatives