1. Budget support training Module 8 Implementation, monitoring and evaluation of a BS programme Version October 2013

2. Outline 1.BS monitoring: a continuous process 2.Budget support policy dialogue 3.Coordination 4.Evaluation 2

3. Monitoring the implementation of BS contracts Regular monitoring and policy dialogue are key elements of all BS operations, they should:  Be conducted according to the new governance mechanisms  Take into account pro-actively the fundamental values Focus on:  Promoting domestic ownership and accountability  Strengthening team work within EU Delegations  Strengthening coordination across all budget support providers  Paying attention to appropriate communication and visibility activities 3

4. Budget support cycle of operations 4 Programming Identi- fication Formulation Implementation Evaluation and Follow-up Evaluation: focus on joint evaluations Decision process for tranche release: Monitoring and dialogue, assessment of payment conditions oQSG2 - Action Fiche: focus on eligibility + context+ conditions. Supporting documents, financing agreement and TAPs oQSG1 - Identification Fiche: focus on eligibility criteria + next steps EC Development Policy Partner Government Policy National Indicative Programme: identify sectors of engagement BSSC: i) agreement in principle on aid modality; ii) assessment of fundamental values Budget Support Steering Committee (BSSC): may review budget support financing proposals and disbursement decisions wherever there are substantial or high political and policy implications.

5. Follow up and data collection  Notes from Central Bank, Min. Fin.,..  Economic & social indicators  IMF/WB Reports  Enacted budget & annexes  Report on budget execution  Assessment of PFM (PEFA, PER)  OBI  Monitoring & eval. of reforms  Human rights strategy  International sources Running Head 12-Point Plain, Title Case Analysis  Macroeconomic framework  Progress in implementing policies  Relevance/credibility of the policies  Progress of PFM reform  Budget execution  Availability/accessibility of budget documents  Monitoring of performance indicators, achievement of results, specific objectives.  Evolution of risks  Monitoring of risk mitigating measures Payment file/annual report  Verify continued relevance/credibility of policies  Macroeconomic assessment  Annual report on PFM  Assessment of progress in transparency  RMF  Matrix of performance indicators Monitoring a BS programme: a continuous process POLICY DIALOGUE 5

6. What to monitor? For GGDC: the continued commitment to the fundamental values (pre-condition for GGDC) For all contracts :  The continued eligibility under the 4 eligibility criteria  The progress towards the specific objectives of the contract and the results achieved  The evolution of the risks  The progress in the implementation of the mitigating measures  The needs and progress of the partner country in terms of capacity development in the areas of importance for the BS programme 6

7. Capacity development Capacity development needs must be assessed systematically:  Is the existing capacity sufficient to absorb the support programme?  Is capacity development based on adequate demand, ownership and commitment?  Is it provided through harmonized and aligned approaches? Areas of particular importance for capacity development:  Public Finance Management  Tax reforms, tax administration  Public expenditures reviews and tracking surveys  Public sector reforms and public administration  Quality and capacity of statistical systems  National control bodies (Parliament, Supreme audit institution, judiciary bodies, internal audit and control institutions, civil society organisations, local authorities and media) 7

8. Decision process (6 steps) for tranche release Partner country 1) Submits its request for tranche release, with justification (may be cover letter + annual review) Running Head 12-Point Plain, Title Case EU Delegation 2) Analyses the request and prepares “payment dossier”.  Are conditions for disbursement fulfilled?  Clear recommendation of HoD on whether and how much should be disbursed DEVCO 3) DEVCO Geographical services prepares dossier for approval of the relevant Director  in line with the procedures agreed in the BS  potentially involving BSSC 4) Geographical Director signs « payment decisions » and sends it to DEU with key messages to be relayed to partner country EU Delegation 5) pays, with formal letters with key messages 6) verifies payment (and exchange rate) 8

9. Outline 1.BS monitoring: a continuous process 2.Budget support policy dialogue 3.Coordination 4.Evaluation 9

10. BS Policy dialogue  Is a key input of BS programmes and the centre piece of mutual accountability.  It provides a framework for: taking stock of partner country’s policies and reforms as well as of the donors’ commitment assessing progress on both sides on the basis of different information, criteria and indicators and of extensive discussion with the government  It can be used as a forward looking tool for: identifying policy slippages timely reaching common understanding with the authorities on corrective measures and refining objectives 10

11. BS Policy dialogue: coverage The BS policy dialogue should cover:  the process of the programme: Formal meetings with government and stakeholders, reviews of programme, etc. Informal contacts  the content: National and sector policies Performance indicators Capacity building Should go beyond the specific elements of the BS operation and address the wider development context 11

12. BS Policy dialogue: adapted to each type of contract (1) GGDC: Dialogue on wide range of policies and reforms: Mobilisation and allocation of resources across the whole government Efficiency and effectiveness of core government systems Aimed at: Improving formulation and implementation of national policies Improving accountability Improving service delivery Promoting good governance 12

13. BS Policy dialogue: adapted to each type of contract (2) SRC Dialogue on sector specific issues Key performance indicators of the sector (input, process, outcome) Access and quality of sector delivery Aimed at Improvement of formulation and implementation of sector policies Improvement of efficiency and effectiveness of Government sectoral delivery structures and systems SBC Dialogue on Overall, coherent and coordinated strategy as regards vital state functions Provision of basic services to the population Transition toward development and democratic governance 13

14. An informed dialogue Policy dialogue should be an informed dialogue:  Drawing on the assessments of the eligibility criteria and the risks;  Requiring an understanding of the actors, rules and mechanisms governing the decision making process in the partner country ….  …… and an understanding of the country-donor coordination framework at national and subnational levels of the government. 14

15. Outline 1.BS monitoring: a continuous process 2.Budget support policy dialogue 3.Coordination 4.Evaluation 15

16. Donor coordination  Nature of donor coordination: Government-led, formal/informal, participation and roles  Existence of common framework and its functioning: harmonisation without unanimity  How does the donor-partner country coordination work?  Existence of a Budget Support Group  Leading role of Government  Predictability of donor funding  Existence of a common Performance Assessment Framework (PAF) and use of disbursement criteria  Existence of complementary initiatives (training, TA,...)  How does the donor-donor coordination work? 16

17. EU Coordination Objective of EU coordinated approach to BS:  Increase effectiveness of BS  Provide coherent and consistent response to challenges Guiding principles:  Sovereign decision making  Systematic information exchange  Supportive of wider donor coordination and country-led coordination  Involves both HQ and field levels  Builds on existing bodies, mechanisms of cooperation, and existing legal frameworks 17

18. Outline 1.BS monitoring: a continuous process 2.Budget support policy dialogue 3.Coordination 4.Evaluation 18

19. 4.1 Objectives and methodology of a BS evaluation To assess to what extent and under which circumstances the BS has successfully enhanced the policies, strategies and spending actions of the partner government so as to achieve sustainable national and/or sector level development outcomes and positive impacts on poverty reduction and economic growth.  Joint exercise of BS donors and recipient country  Improved methodological approach just been developed.  Evaluations need to be prepared right from the start of a BS contract: collection and recording of baseline information and data, and setting up a good monitoring and reporting system. 19

20. The comprehensive evaluation framework  Based on intervention logic of BS: hypothesised sequence of effects.  Central notion: BS is a contribution to the implementation of the policy and spending actions of a partner government.  The Comprehensive Evaluation Framework (CEF) consists of five levels: See page 10 of BS Guidelines.  Many external (non-BS) factors determine induced output, outcome and impact.  Broad assessment of outcomes and impact of government policies needed.  Specific CEF needed for a joint GBS programme and each joint SBS programme. 20

21. 21

22. 22 Government strategy Impacts  Sustainable and inclusive growth  Poverty reduction Budget support Outcomes  Positive responses by service users and economic actors to government policy management and service delivery Outputs  Execution of the budget  Monitoring of results  Improved macro-economic management  Improved public services  Etc. Induced outputs  Improved macro-economic management  Improved public services  Strengthened PFM  Improved policy formulation  Strengthened public sector institutions,  Etc. Inputs  National and sector policies  Budgetary resources  Human resources  Institutional structures  Strategies and operational programmes Outputs  More aid provided through the budget  More funds available for discretionary spending  Increased predictability of disbursement of external funds  Policy dialogue, conditionalities, TA and capacity building better coordinated and more conducive for government strategies  Improved harmonisation and alignment  Reduced transaction costs Inputs  Transfer of funds to the national Treasury and disbursement conditions  Policy dialogue and performance indicators  Capacity building and TA Logical frameworks of the partner government action and of the BS programme

23. Main driving forces  Flow of funds effect:  Effects of additional government spending;  Effects of pushing donor funds through the PFM system.  Policy and institutional effects, via:  Policy dialogue;  Monitoring performance indicators;  Disbursement conditions, and;  Capacity development activities. 23

24. The three step approach.  Analysis of direct causality possible up to level 3.  At level 4 and 5 too many other influencing factors.  Step 1: assessment of inputs, direct outputs and induced output, and analysis of causality.  Step 2: assessment of outcomes and impact of government policies and spending.  Step 3: Exploration of contribution of BS to the government policies and spending, which have produced the outcomes and impact. 24

25. 25 Government strategy Impacts  Sustainable and inclusive growth  Poverty reduction Budget support Outcomes  Positive responses by service users and economic actors to government policy management and service delivery Outputs  Execution of the budget  Monitoring of results  Improved macro-economic management  Improved public services  Etc. Induced outputs  Improved macro-economic management  Improved public services  Strengthened PFM  Improved policy formulation  Strengthened public sector institutions,  Etc Inputs  National and sector policies  Budgetary resources  Human resources  Institutional structures  Strategies and operational programmes Outputs  More aid provided through the budget  More funds available for discretionary spending  Increased predictability of disbursement of external funds  Policy dialogue, conditionalities, TA and capacity building better coordinated and more conducive for government strategies  Improved harmonisation and alignment  Reduced transaction costs Inputs  Transfer of funds to the national Treasury and disbursement conditions  Policy dialogue and performance indicators  Capacity building and TA Step 1: Evaluation of the chain from inputs to induced outputs of the BS Step 1

26. 26 Government strategy Impacts  Sustainable and inclusive growth  Poverty reduction Budget support Outcomes  Positive responses by service users and economic actors to government policy management and service delivery Outputs  Execution of the budget  Monitoring of results  Improved macro-economic management  Improved public services  Etc. Induced outputs  Improved macro-economic management  Improved public services  Strengthened PFM  Improved policy formulation  Strengthened public sector institutions,  Etc Inputs  National and sector policies  Budgetary resources  Human resources  Institutional structures  Strategies and operational programmes Outputs  More aid provided through the budget  More funds available for discretionary spending  Increased predictability of disbursement of external funds  Policy dialogue, conditionalities, TA and capacity building better coordinated and more conducive for government strategies  Improved harmonisation and alignment  Reduced transaction costs Inputs  Transfer of funds to the national Treasury and disbursement conditions  Policy dialogue and performance indicators  Capacity building and TA Step 2: Evaluation of the results and impacts of the Government action Step2

27. 27 Government strategy Impacts  Sustainable and inclusive growth  Poverty reduction Budget support Outcomes  Positive responses by service users and economic actors to government policy management and service delivery Outputs  Execution of the budget  Monitoring of results  Improved macro-economic management  Improved public services  Etc. Induced outputs  Improved macro-economic management  Improved public services  Strengthened PFM  Improved policy formulation  Strengthened public sector institutions,  Etc Inputs  National and sector policies  Budgetary resources  Human resources  Institutional structures  Strategies and operational programmes Outputs  More aid provided through the budget  More funds available for discretionary spending  Increased predictability of disbursement of external funds  Policy dialogue, conditionalities, TA and capacity building better coordinated and more conducive for government strategies  Improved harmonisation and alignment  Reduced transaction costs Inputs  Transfer of funds to the national Treasury and disbursement conditions  Policy dialogue and performance indicators  Capacity building and TA Step 3: Exploration of the links between the BS and the results and impacts of the Gov. action Step 3

28. 4.2 Implementing a BS evaluation (1/2): preconditions  Objectives, strategies and performance indicators of the BS need to be clearly defined at the start of the BS programme.  Implementation needs to be well monitored and documented, and reporting well organised.  Align timing of BS evaluation with other supportive studies and vice versa.  Start the preparation of the BS evaluation timely: 2 years before the report has to be finalised.  Involve and strengthen evaluation departments of MinFin and – in case of SRC – sector ministries.  Align scope and level of detail of the evaluation with availability of data and information (and available budget for the evaluation). 28

29. 4.2: Implementing a BS evaluation (2/2) : Phases and activities  Preparation: drafting ToR, establishment of management and reference groups, recruitment of consultants.  Five phases: Inception, data collection and preliminary analysis, in depth analysis and verification, synthesis and dissemination.  The phases will take 12-18 months and require 350 to 450 person days.  Alternatively: steps could be implemented at different times.  Step 1 could be carried out more often than steps 2 and 3. 29

30. 4.3Key points  Evaluation of BS should be carried out every 5-7 years  Preparation of a BS evaluation should start when the BS programme is being formulated.  Data collection and background studies and surveys should become part of the implementation process.  Partner government and donors should take joint responsibility for the BS evaluation.  Agreement to carry out a joint evaluation (and its timing) should be mentioned in the TAP/financing agreement. 30