Presentation on theme: "Designing and Building a Results-Based Monitoring and Evaluation System: A Tool for Public Sector Management."— Presentation transcript:
1 Designing and Building a Results-Based Monitoring and Evaluation System: A Tool for Public Sector Management
2 The Power of Measuring Results If you do not measure results, you can not tell success from failureIf you can not see success, you can not reward itIf you can not reward success, you are probably rewarding failureIf you can not see success, you can not learn from itIf you can not recognize failure, you can not correct itIf you can demonstrate results, you can win public supportAdapted from Osborne & Gaebler, 1992
3 Ten Steps to Designing, Building and Sustaining a Results-Based Monitoring and Evaluation System Planning for Improvement — Selecting Results TargetsSelecting Key Indicators to Monitor OutcomesConducting a Readiness AssessmentThe Role of EvaluationsUsing Your Findings12345678910Agreeing on Outcomes to Monitor and EvaluateBaseline Data on Indicators—Where Are We Today?Monitoring for ResultsReporting Your FindingsSustaining the M&E System Within Your Organization
4 Introduction to Results-Based Monitoring and Evaluation What Are We Talking About?Results-based monitoring and evaluation measures how well governments are performingResults-based monitoring and evaluation is a management tool!Results-based monitoring and evaluation emphasizes assessing how outcomes are being achieved over time
5 RememberMonitoring and evaluation are two separate, but interrelated strategies to collect data and report the findings on how well (or not) the public sector is performingDuring this workshop, we will be discussing:Monitoring as a toolEvaluation as a toolHow the two interrelate to support good public managementThe ten steps to build a results-based monitoring and evaluation system to measure government performance
6 Reasons to Do Results-Based M&E Provides crucial information about public sector performanceProvides a view over time on the status of a project, program, or policyPromotes credibility and public confidence by reporting on the results of programsHelps formulate and justify budget requestsIdentifies potentially promising programs or practices
7 Reasons to Do Results-Based M&E (cont.) Focuses attention on achieving outcomes important to the organization and its stakeholdersProvides timely, frequent information to staffHelps establish key goals and objectivesPermits managers to identify and take action to correct weaknessesSupports a development agenda that is shifting towards greater accountability for aid lending
8 DefinitionResults-Based Monitoring (what we will call “monitoring”) is a continuous process of collecting and analyzing information to compare how well a project, program or policy is performing against expected results
9 Major Activities Where Results Monitoring Is Needed Setting goals and objectivesReporting to Parliament and other stakeholdersManaging projects, programs and policiesReporting to donorsAllocating resources
10 A New Emphasis on Both Implementation and Results-Based Monitoring Traditional monitoring focuses on implementation monitoringThis involves tracking inputs ($$, resources, strategies), activities (what actually took place) and outputs (the products or services produced)This approach focuses on monitoring how well a project, program or policy is being implementedOften used to assess compliance with workplans and budget
11 A New Emphasis on Both Implementation and Results-Based Monitoring Results-based monitoring involves the regular collection of information on how effectively government (or any organization) is performingResults-based monitoring demonstrates whether a project, program, or policy is achieving its stated goals
12 Results Based Monitoring Requires Attention to Causal Logic ---or The Theory of Change What is the “ logic” of the overall project, program or policy design?How do each of the components of the program help to establish an If-Then relationIs there a theory behind the change expected or seen? In other words does the change follow the logic proposed?Does this theory or logic hold during implementation?
13 Results-Based Monitoring Goal (Impacts)Long-term, widespread improvement in societyResultsOutcomesIntermediate effects of outputs on clientsOutputsProducts and services producedActivitiesTasks personnel undertake to transform inputs to outputsImplementationInputsFinancial, human, and material resources
14 Results-Based Monitoring: Adult Literacy Goal (Impacts)Higher income levels; increase access to higher skill jobsOutcomesIncreased literacy skill; more employment opportunitiesOutputsNumber of adults completing literacy coursesActivitiesLiteracy training coursesInputsFacilities, trainers, materials
15 DefinitionResults-Based Evaluation An assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact and sustainability. The intent is to incorporate lessons learned into the decision-making process.
16 Evaluation Addresses “Why” Questions – What caused the changes we are monitoring“How” QuestionsWhat was the sequence or processes that led to successful (or not) outcomes“Compliance/ Accountability Questions”Process/ Implementation QuestionsDid the promised activities actually take place and as they were planned?Was the implementation process followed as anticipated, and with what consequences
17 Designing Good Evaluations Getting the questions right is criticalAnswering the questions is criticalSupporting public sector decision-making with credible and useful information is critical
18 Designing Good Evaluations “Better to have an approximate answer to the right question, than an exact answer to the wrong question.” Paraphrased from statistician John W. Tukey
19 Designing Good Evaluations “Better to be approximately correct than precisely wrong.” Paraphrased from Bertrand Russell
20 Some Examples of Evaluation Privatizing Water SystemsResettlementPolicy EvaluationsComparing model approaches to privatizing public water suppliesComparing strategies used for resettlement of rural villages to new areasProgram EvaluationsAssessing fiscal management of government systemsAssessing the degree to which resettled village farmers maintain previous livelihoodProject EvaluationsAssessing the improvement in water fee collection rates in 2 provincesAssessing the farming practices of resettled farmers in one province
21 Some Examples of Evaluation Privatizing Water SystemsResettlementPolicy EvaluationsComparing model approaches to privatizing public water suppliesComparing strategies used for resettlement of rural villages to new areasProgram EvaluationsAssessing fiscal management of government systemsAssessing the degree to which resettled village farmers maintain previous livelihoodProject EvaluationsAssessing the improvement in water fee collection rates in 2 provincesAssessing the farming practices of resettled farmers in one province
22 Complementary Roles of Results-Based Monitoring and Evaluation Clarifies program objectivesAnalyzes why intended results were or were not achievedLinks activities and their resources to objectivesAssesses specific causal contributions of activities to resultsTranslates objectives into performance indicators and set targetsExamines implementation processRoutinely collects data on these indicators, compares actual results with targetsExplores unintended resultsReports progress to managers and alerts them to problemsProvides lessons, highlights significant accomplishment or program potential, and offers recommendations for improvement
23 Developing A Results Plan Once a set of outcomes are identified, it is time to develop a plan to assess how the organization will begin to achieve these outcomesIn the traditional approach to developing a plan, the first thing a manager usually did was to identify activities and assign responsibilitiesBut the shortcoming in this approach is that completing all the activities does not mean the same as reaching the outcome goal
24 Key Types of Monitoring ImpactResultsResults MonitoringOutcomeOutputImplementationActivityImplementation Monitoring (Means and Strategies)Input
25 Translating Outcomes to Action Note: Activities are crucial! They are the actions you take to manage and implement your programs, use your resources, and deliver the services of governmentBut the sum of these activities may or may not mean you have achieved your outcomesQuestion is: How will you know when you have been successful?
26 Implementation Monitoring Links to Results Monitoring OutcomeMeans and Strategies (Multi-Year and Annual Work Plans)Target 1Target 2Means and Strategies (Multi-Year and Annual Work Plans)Target 3Means and Strategies (Multi-Year and Annual Work Plans)
Your consent to our cookies if you continue to use this website.