Presentation is loading. Please wait.

Presentation is loading. Please wait.

Manuel Fernando Castro The Role of M&E Systems in Performance Budgeting: Good-practice and lessons learned from experience V Konferencja Ewaluacyjna Warszawa,

Similar presentations


Presentation on theme: "Manuel Fernando Castro The Role of M&E Systems in Performance Budgeting: Good-practice and lessons learned from experience V Konferencja Ewaluacyjna Warszawa,"— Presentation transcript:

1 Manuel Fernando Castro The Role of M&E Systems in Performance Budgeting: Good-practice and lessons learned from experience V Konferencja Ewaluacyjna Warszawa,

2 The Role of M&E Systems in Performance Budgeting: Good-practice and lessons learned from experience Warsaw, October 2009 V Annual Evaluation Conference

3 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Examples of good practice countries (Mexico, Chile, UK) ► Lessons from international experience

4 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Examples of good practice countries (Mexico, Chile, UK) ► Lessons from international experience

5 5 Regular tracking of indicators to establish whether planned targets have been achieved Monitoring “M” Evaluation “E” Analytical efforts to answer specific questions about performance of a program/activities Impact evaluation “IE” Analytical efforts to relate cause and effect. Establishing what would have happened in the absence of the intervention What is M&E? Each tool has strengths, costs, limitations

6 6 What is Performance Budgeting? “Form of budgeting that relates funds allocated to measurable results” (OECD, 2005a) 3 different modalities PresentationalPerformance-informed Direct/Formula funding Linkages with allocation No linksIndirect/Loose linkDirect/Tight link Main objective AccountabilityPlanning and/or accountability Resource allocation and accountability Evidence of utilization Sweden, Denmark, Colombia Most OECD (e.g., UK, Australia, Mexico) + Chile. Mostly in sectors (e.g., Denmark's school funding, Chile's per capita funding for clinics)

7 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Examples of good practice countries (Mexico, Chile, UK) ► Lessons from international experience

8 There is no simple/direct relation between spending and results Similar increases in public spending can be associated with totally different outcomes… Spending data for 1990s from World Development Indicators database. Child mortality data from Unicef Other data from World Bank. School completion from Bruns, Mingat and Rakatomalala More spending does not necessarily mean better outcomes

9 There is no simple/direct relation either from growth rates to results Tanzania and Bangladesh have similar GDP per capita, but life expectancies are 55 years and 65 years. Philippines 2006 per capita income was 30% higher than in Moldova, but Moldova’s maternal mortality ratio was 16 as opposed to 162 per 100,000 live births in the Philippines. Despite remarkable economic growth, India’s maternal mortality rate has worsened and is six times worse than China's and 14 times worse than Chile's. Higher economic growth, can mask the differences in important outcomes of development.

10 Budget officials and policy makers need to respond challenging questions What are the underlying causes of results? Are this direct, indirect, intended, non intended? What programs/policies should fbe unded, strengthened or changed, and how? STRATEGIC Are we doing the right things? Is program implementation following anticipated paths? Are promised activities taking place as planned? OPERATIONAL Are we doing things right? Are there best practices or lessons learned that can be applied? LEARNING Are there better ways of doing them?

11 M&E can be associated with better project outcomes (assist with improved management) % WB projects rated high or substantial in M&E were rated satisfactory or better in their outcomes Those with negligible ratings had satisfactory only in 32% of cases

12 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Examples of two good practice countries (UK, Mexico) ► Lessons from international experience

13 13 Key factors Evaluation measures results, Performance Budgeting tides results to funding. To be relevant and sustainable PB demands continuous performance information : But …multiple requirements to fulfill Clear program logics / results chains Timely, and quality data Good indicators and targets Robust methods and tools Quality standards and procedures Mechanisms to ensure utilization and influence

14 The right kind of information is frequently missing Conflicting demands of PI, wrong incentives, and low M&E capacities often lead to - Lack of clarity on the role of PI for budgeting - Duplication and overspending - Not standardized methods and tools (limitations and strengths) - Low quality and scarce use

15 M&E systems help tackle performance information problems Ensuring supply of performance data (records, stats, balanced M&E data, etc.) Stimulating demand, (use in policy, transparency, accountability, etc) Guaranteeing quality and credibility of data (robust methodologies, standards) But also increasing access to PI (integrated IT systems, reports, etc).

16 Country-level M&E systems are powerful budgetary tools Facilitate budget shift from traditional management (input/ activity- based) to performance management (output/outcome-based) Emphasize effective resource allocation for planning and budgeting Help countries assess if using scarce resources cost-effectively Respond to elected officials and the public’s demands for accountability in the budget Enhance the need for managers in the public sector Stress knowledge and learning for improving budget decision- making

17 NO Performance Budgeting possible without systematic and good quality M&E information. NO effective M&E without links with the budget process Development of PB and M&E systems need to go hand in hand Bottom line of the M&E and PB relationship

18 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Good practice country cases ► Lessons from international experience

19 United Kingdom: The Public Service Agreements System Main architecture features: - High level Strategy (PMSU) and Delivery Units (PMDU) - Ministerial public agreements on services to be delivered - Targets, indicators and managers for each delivery agreement - Ministries’ PSA Delivery Board (chaired by managers) - Regular performance monitoring (PMDU, Cabinet) - Dif evaluation types (expending reviews, program, impact, audits) - Public dissemination of strategy and agreements H.M. Treasury key player. Prime Minister’s (PMSU) and (PMDU) provide advise and regular updates directly to the Prime Minister.

20 United Kingdom: The Public Service Agreements System Strengths “Explains what departments plan to deliver in return for…significant extra investment” “[Sets out] demanding national targets” (Nt. Standards) Multiple agencies but H.M. Treasury guidelines allow coordination Highly credible evaluations (externally contracted) “[Reflects] the Government’s key priorities and [focus] on the outcomes that matter most to the public” Establishes responsible “managers” of each PS Leading role of Treasury and Prime Minister’s office

21 Mexico: The Performance Evaluation System – SED Main architecture features: - Independent Evaluation Unit: CONEVAL council composed of academics - Single Federal Evaluation Guidelines - “Logic Frameworks” and standards for all F. Programs Performance indicators (output and outcome based) - 6 evaluation methods: Impact, consistency, indicators, process, design, perf. - Annual Evaluation Plan (More than 100 annually). The MoF is the system’s “champion” with active support and participation of the President’s Office.

22 Mexico: The Performance Evaluation System – SED Strengths CONEVAL Model brings credibility - Contracts out evaluation (market of evaluation) - increases quality, regularity, and timeliness of evaluation work - Promotes capacity development Unique poverty measurement schema as part of the system Robust and credible evaluations, strengthening monitoring All evaluations reported to Congress (with significant budget initiative) Powerful role of MoF and Presidency increases utilization Active role of Ministries and agencies in defining evaluation agenda

23 Chile: The Management Control System Main architecture features: - Ex ante cost-benefit analysis for investment programs - 1,550 performance indicators for all government programs (2006) - “Program evaluations”: desk reviews of log frame (14 each year) - Rigorous impact evaluations (4 each year) - Comprehensive Spending Reviews: selective programs (2-5 each year) - Agency management improvement programs linked to staff payments The MoF (Budget Directorate) designed and uses the system. Congress has no spending initiative.

24 Chile: The Management Control System Strengths Performance information used to identify need for evaluations Evaluations conducted externally, in fully transparent process, and are highly credible All M&E findings reported publicly and sent to Congress M&E system closely linked to the budget information needs of MoF Performance information used to set targets for ministries – these are largely met MoF closely monitors extent of utilization of evaluation findings

25 Overview ► What do we mean by M&E and PB ► Strategic context for M&E and Performance Budgeting ► Role of country-level M&E systems in Performance Budgeting ► Good practice country cases ► Lessons from international experience

26 Main risks and difficulties 1.Deficient budget planning - No technical definition of program objectives, goals and indicators 2.Weak system design, unclear institutional arrangements - How much “M” & “E” is enough?, who plan, conduct,and use of evaluations? - Self-evaluations or independent evaluations? - Who funds the system? (a) earmarked budget funds?; (b) costs “absorbed” by ministries? How it links with the budget? 3. On supply side: caveat multiple, competing, uncoordinated systems 4. On demand side: Budget “rigidity”, incentives are key to achieve utilization 5. Low staff capacities (needs PS managers)

27 No strong preconditions but “Success” factors include 1. High Gov. commitment with quality of public spending 2. Powerful “champion”, ideally centrally driven by capable MoF Ministry 3.Careful attention to both demand and supply sides 4.Sustained, continuous effort 5. Piecemeal to ensure assimilation 6.“Quick gains” to avoid the costs of a too timid implementation 7. Avoiding complex systems (keep it simple)

28 Useful resource material Keith Mackay, How to Build M&E Systems for Better Government, World Bank, Ernesto May et al. (eds.), Towards the Institutionalization of Monitoring and Evaluation Systems in Latin America and the Caribbean, World Bank, K:146736~piPK:146830~theSitePK:258554,00.html K:146736~piPK:146830~theSitePK:258554,00.html Manuel F. Castro, Insider Insights: Building a Results-Based Management and Evaluation System in Colombia. World Bank, Gonzalo Hernandez Licona, The Role of Evaluation in Mexico: Achievements, Challenges and Opportunities. Quality of Public Expenditure. Note No. 2. World bank, QualityofPubExpenditure-No2.pdf QualityofPubExpenditure-No2.pdf

29 Useful resource material Teresa Curristine, Performance Information in the Budget Process: Results of the OECD 2005 Questionnaire, in OECD Journal on Budgeting, (2),pp OECD, Performance Budgeting in OECD Countries, OECD, Independent Evaluation Group, M&E Tools, Methods and Approaches, World Bank, Independent Evaluation Group, Influential Evaluations: Evaluations that Improved Performance and Impacts of Development Programs, World Bank, World Bank website on Building Government M&E Systems:


Download ppt "Manuel Fernando Castro The Role of M&E Systems in Performance Budgeting: Good-practice and lessons learned from experience V Konferencja Ewaluacyjna Warszawa,"

Similar presentations


Ads by Google