European Commission Joint Evaluation Unit common to EuropeAid, Relex and Development Methodology for Evaluation of Budget support operations at Country level Conceptual and Methodological Note

EXISTING BUDGET SUPPORT EVALUATIONS 2004 Tanzania 2006 Joint Evaluation of 7 countries from 1994 to 2004 budget support 2007 Ghana Evaluations addressing mainly THE PROCESS And not enough the CHAIN OF RESULTS  IMPACTS AT THE GLOBAL LEVEL  IMPACTS ON SECTOR POLICIES

1.An inventory of the inputs provided through budget support during a given period. 2.An assessment of the performance of these inputs in terms of direct outputs and induced outputs. 3.An identification of the changes which have occurred during the period in the partner country and the key causal factors driving those changes related to the budget support. 4.An assessment of the extent to which budget support may have contributed to the results identified at the outcome and impact levels. 5.An assessment of the overall relevance of budget support arrangements in the light of the evolving country and sector specific context, the aid policy and the related goals.

A Three components approach methodology combining An evaluation of the budget support itself An assessment of the budget support related outcomes and impacts of the government strategy An exploration of the linkages between the budget support process and some outcomes and impacts of the government strategy Focused on the main topics of the budget support Each topic is associated with one or several Evaluation Questions, Judgement Criteria and Indicators To ensure harmonisation of the evaluations in various contexts

GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK

EVALUATION OF THE BUDGET SUPPORT Assessment of the inputs provided by the budget support Assessment of the direct outputs of the budget support Assessment of the induced outputs and the corresponding government actions Overview of the national context and its evolution Volume of funds, policy dialogue and conditionalities, technical assistance and capacity building, harmonisation and alignment Effects of the identified inputs including effects on transaction costs Entry conditions for budget support Changes that budget support has induced in comparison to what was expected

Review of the impacts of the government strategy: specifically those related to the budget support deliverables Economic growth Incidence of income and non income poverty Degree of empowerment and social inclusion Evolution of MDG indicators Achieving sector development objectives for SBS ASSESSMENT OF IMPACTS AND OUTCOMES Government e.g.

Identification of the factors external to the government strategy which could have had an impact on the mentioned changes Regional and international economic Political environment Other unforeseen event Identification of the factors that have contributed to the key changes Changes in business confidence and competitiveness Changes in the use and appreciation by the beneficiaries of goods and services provided by the public sector Changes in the degree of confidence of the general public sector Causal factors e.g.

Identification of the outputs of the government strategy which appear to have been of greatest importance in generating change at outcomes level Positive linkages between government actions and improved outcomes and Failure or inappropriate actions which can have jeopardised the quality of the outcome Relationship

Comparison of the results of component 1 and component 2 Improved understanding of the influence of budget support on the institutional and policy changes Increased consideration of the role of dominant factors in the determination of outcomes and impacts If consistencies: indicate causal links showing positive conclusion for budget support If no consistency: find a possible explanation ASSESSMENT OF THE INFLUENCE OF BUDGET SUPPORT « INTEGRATION AND CONSISTENCIES BETWEEN COMPONENTS 1 & 2 »

GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK Component 1 Component 2 Component 3

HOW TO MANAGE AND IMPLEMENT THIS EVALUATION? CHALLENGES AND PROPOSALS We must take into account : 1.THE STAKEHOLDERS 2.THE LEAD OF THE EVALUATION 3.THE CONSISTENCY, HARMONISATION AND DISSEMINATION ISSUES 4.THE IMPLEMENTATION 5.THE CAPACITY BUILDING ISSUE

STAKEHOLDERS To whom this evaluation is for? Are the donors the only stakeholders? Proposal 1 All donors, the government and other main stakeholders involved in the budget support evaluated should be part of the evaluation. The evaluation of budget support operation in a designed country should be part of a multi-donor evaluation programme.

EVALUATION LEAD Who will take the lead for managing the evaluation? From where will the evaluation be managed? Proposal 2 One of the main donors in the country should be chosen by the other donors to take the lead and consequently for contracting with the evaluators. Special arrangements should be set up to involve representatives in the country.

CONSISTENCY, HARMONISATION AND DISSEMINATION ISSUES Proposal 3 To ensure consistency, the headquarter Evaluation Service of the lead donor should be responsible for the evaluation methodology and quality assurance. Each donor can have dedicated persons in his Evaluation Service to go into the countries where the evaluation takes place, at crucial moments. Proposal 4 A network of donors should be set up to ensure harmonisation and dissemination.

IMPLEMENTATION Who will do the evaluation? Who has currently the competencies to evaluate budget support and how to manage it? Proposal 5 A team of external independent consultants familiar with the main topics of the evaluation should be contracted for each evaluation.

CAPACITY BUILDING Do we need a specific training on budget support evaluation? Proposal 6 Make sure that few consultants are immediately operational and have been already trained in existing trainings: Proposal 7 Train the identified people within the Evaluation services of the Member States, some external independent consultants and the main governmental stakeholders of the recipient countries.

1.Issue paper (May 2008) 2.Steering group at OCDE / DAC level (June 2008) 3.1 st meeting of a core group (September 2008) 4.DAC Evaluation Network (18 – 19 November 2008) 5.2 nd meeting of core group (December 2008) 6.Common methodological document and standard terms of reference (early 2009) 7.Testing methodological approach with partners’ agreement (2009)