WHAT is evaluation and WHY is it important? Clemens Gros Edited & Presented by Stephen Ntow, PhD June 2017

PERFORMANCE MONITORING Impact Evaluation DESIRED RESULT (Better!) LATER Formative Evaluation Policy / Programme PERFORMANCE MONITORING Monitoring Ex-ante Evaluation SITUATION SITUATION MONITORING NOW (Challenges)

Evaluation is Answering Three Questions Doing the Right Thing? Relevance Rational / justification Beneficiary satisfaction Doing the Right Thing? Answering Three Questions Effectiveness / coherence Efficiency: optimizing resources Doing it Right? Source: UNICEF Evaluation Policy Alternatives Good practices Lessons learned Better Ways of Doing it?

“An evaluation is an assessment, as systematic and impartial as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area, institutional performance, etc. It focuses on expected and achieved accomplishments examining the results chain, processes, contextual factors and causality, in order to understand achievements or the lack thereof. It aims at determining the relevance, impact, effectiveness, efficiency and sustainability of the interventions and contributions an institution or actor. An evaluation should provide evidence-based information that is credible, reliable, and useful, enabling the timely incorporation of findings, recommendations and lessons into decision-making processes.” - Source: Norms for Evaluation in the UN System and UNICEF Evaluation Policy

Standard Evaluation Criteria Relevance: The extent to which the intervention is suited to the needs of the primary target group / national priorities and policies. Standard Evaluation Criteria Effectiveness: Is the intervention achieving satisfactory results in relation to stated objectives? Relevance Effectiveness Efficiency Impact Sustainability Efficiency: measures the outputs in relation to the inputs. Does the intervention use the least costly resources possible to achieve the desired results? Impact: The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended. Sustainability: measures whether the benefits of an intervention are likely to continue after support / funding has been withdrawn.

Why evaluation? Accountability for results Capacity development Evidence-based decision making / advocacy KNOWLEDGE to change behaviour and to influence policy

The complementarity between M & E Monitoring Evaluation a. Tracking of programme /project/ activity progress Appraisal of merit, value or worth /project/activity progress b. Systematic, ongoing b. Systematic, periodic or ongoing c. During programme implementation c. Before, During and after implementation) d. Compared to criteria d. According to implementation plan (relevance, effectiveness, impact, etc.) e. For decision - making, especially e. For decision - making about future short term corrective action programming f. For accountability for implementation f. For accountability for results g. Contributes to evaluation g. For organizational learning

QUESTIONS ?

Different evaluation approaches

Different options to collect data Information from individuals Information from groups Observation Physical measurements Reviewing existing records and data

For more information A free and very informative online resource on evaluation: http://betterevaluation.org

Evaluation Types in the Implementation Process Identify a need Design an Intervention Initial Situation Reconsider, Redesign, Expand, Reduce, or End Programme / Project Start-Up Desired Result Baseline OUTPUTS OUTCOMES IMPACT RMB is a core principle of UNICEF’s work. This slide presents key RBM elements in their relation to M&E activities. We will talk about monitoring and evaluation and some other elements of the M&E function in the next slides. Ex-ante Evaluation Formative/ Process Evaluation Summative/ Outcome/ Impact Evaluation Evaluation Framework On-going Monitoring

The evaluation planning process 4. Evaluation Methodology Methodology answers questions 3. Evaluation Questions Related to each criterion/issue 2. Evaluation Objectives/Criteria Relevance, Efficiency, Effectiveness, Impact, Sustainability 1. Evaluation Purposes Why now? For what? Who? Defines the evaluation approach

Only when these 4 elements are clear, start the process Analysis & recommen-dations 5 Recruit evaluation team 3 Engage Stakeholders Prepare TOR, based on evaluation approach, incl. utilisation & dissemination strategy! 2 1 Collect evidence 4 Share & use results 6

M&E MONITORING EVALUATION Situation /events monitoring Performance Monitoring EVALUATION Doing the right thing? Doing it right? Better ways of doing it?

QUESTIONS ?