Evaluation of the Paris Declaration Rationale and Governance of the Evaluation AEA 2011 Niels Dabelstein & Ted Kliest

Why Evaluate the Paris Declaration? Focuses on what has been achieved and what has not – and why – the key questions at HLF 4 and for the post-PD era Evaluation, with Monitoring, is built into the Declaration itself and reflects its principles. The Accra Agenda for Action called specifically for an evaluation of the implementation and effects of the PD Adds value to the Monitoring Survey and feeds into the High Level Forums in 2008 (phase 1) and 2011 (phase 2 synthesis)

The Key Evaluation Questions “What are the important factors that have affected the relevance and implementation of the Paris Declaration and its potential effects on aid effectiveness and development results?” (The Paris Declaration in context) “To what extent and how has the implementation of the Paris Declaration led to an improvement in the efficiency of aid delivery, the management and use of aid and better partnerships?” (Process and intermediate outcomes) “Has the implementation of the Paris Declaration strengthened the contribution of aid to sustainable development results? How?” (Development outcomes)

A joint evaluation Based on the principles of the Paris Declaration: partner countries and development partners develop the evaluation framework/approach and execute the evaluation jointly The evaluation itself is a tool for mutual accountability: 22 Country-level evaluations led by partner countries and managed in-country (Phase 1=7, Phase 2=21) 18 Donor/agency HQ studies (phase 1=11, Phase 2=7)

Building blocks of the Evaluation SYNTHESIS EVALUATION QUESTIONS 3. Development outcomes 2. Process and intermediate outcomes 1. Context DONOR STUDIES COUNTRY STUDIES SUPPLEMENTARY STUDIES PDE PHASE 1 RESULTS + Monitoring Information

Phase 2 Country Evaluations & Donor Studies

Supplementary Studies Support to Statistical Capacity Building The Applicability of the PD in Fragile and Conflict-affected Situations Untying of Aid: Is it Working? The PD, Aid Effectiveness and Development Effectiveness Development Resources Beyond the Current Reach of the PD

PD Evaluation Milestones 2006 March Options Paper 2007 March 1st Reference Group meeting, Paris – Eval. Framework agreed June 2nd Ref. Group meeting, Copenhagen – Launch of Phase 1 2008 Feb. 3rd. Ref. Group Meeting, South Africa – Emerging Findings March 4th Ref. Group Meeting, Paris - draft Synthesis report June Phase 1 Synthesis Report Sept. 3rd HLF in Accra, Ghana – Completion Phase 1 2009 Feb. 1st Ref. Group meeting, Auckland – Phase 2 Approach Approved Dec. 2nd Ref. Group Meeting, Paris – Launch of Phase 2 2010 Dec. 3rd Ref. Group Meeting, Bali – Emerging Findings 2011 April 4th Ref. Group Meeting, Copenhagen - Phase 2 Draft Synthesis Report June Phase 2 Synthesis Report Nov. 4th HLF in Busan, Korea – Completion Phase 2

Governance, management and implementation International Reference Group (40-plus reps. of governments, international Organizations and CSOs. Co-chaired by Malawi and Sweden) Management Group (Colombia, Malawi, Netherlands, Sweden, US, Vietnam) Evaluation Secretariat at DIIS National/Agency Reference Groups and Evaluation Coordinators National/Agency Evaluation Teams (with specified recruitment criteria, and common generic ToRs) Core Evaluation Team (7 Members, from Canada, Denmark, Nigeria, Peru, Sri Lanka, Malawi and the UK + resource persons) High Level Peer Reviewers: Dr. Mary Chinery-Hesse and Mr. Mark Malloch Brown. 9

Overseeing the evaluation International Reference Group, co-chaired by Malawi and Sweden Members of the DAC Network on Development Evaluation (who actively support) Partner countries who conduct country-level evaluations Multilaterals and CSOs (who actively support) Tasks: Endorse overall Evaluation Framework and ToRs for component & supplementary studies Review draft reports for quality, credibility & clarity 10

Members of Int. Ref. Group Australia Austria Belgium Canada Denmark Finland France Germany Ireland Japan Luxemburg The Netherlands New Zealand Norway Spain Sweden Switzerland United Kingdom USA AfDB AsDB BACG/Better Aid Concord Global Fund/GAVI OECD/DAC UNDG World Bank Afghanistan Bangladesh Benin Bolivia Cambodia Cameroun Columbia Cook Islands Indonesia Ghana Malawi Mali Mozambique Nepal Peru Philippines Samoa Senegal South Africa Sri Lanka Uganda Viet Nam Zambia

Managing the evaluation Management Group (Colombia, Malawi, Netherlands, Sweden, US, Vietnam) responsible for: Developing the overall evaluation framework and ToR for the Core Evaluation Team Coordinating and managing the evaluation process Guiding the component studies Developing and managing supplementary studies and synthesis of findings and recommendations Dissemination Evaluation Secretariat at the Danish Institute for International Studies 12

Country Reference Groups Include major stakeholders from government, donors, civil society and possibly academia. Its purpose is to ensure stakeholders’ participation and buy-in to the evaluation process and results and to assure the independence of the evaluation. It will Endorse detailed design of country study Oversee team selection Serve as resource for country evaluation teams Review draft evaluation products for quality, credibility and clarity The reference/advisory group should include major stakeholders from government, donors, civil society and possibly academia. The purpose of this group is to ensure stakeholders’ participation and buy-in to the evaluation process and results and to assure the independence of the evaluation. Ideally, the reference/advisory group should provide some standing capacity to follow up on the evaluation after completion. 13 13

Team Configurations

The Core Evaluation Team Developed detailed methodology and provides support to country teams Was responsible for the synthesis of country and donor HQ evaluation results, those from Phase 1, and other studies, and for preparing the overall Evaluation Report Reported and was responsible to the Evaluation Management Group, through the Evaluation Secretariat Was competitively recruited (by international tender) by the Evaluation Management Group Comprise six international consultants and a number of associated members for specific tasks

Peer Reviewers The task of the Review Panel was to review the final draft of the synthesis report in order advice the Management Group and Synthesis Team on quality, strategic and policy relevance and the communicative power of the Synthesis report produced by the Independent Evaluation Team

Evaluation of the Evaluation An independent review of strenghts, weaknesses and lessons for future international joint evaluations

Full reports and supporting materials All documents from the Evaluation, including the full country evaluations and donor studies, can be found in English, French and Spanish on www.busanhlf4.org and www.oecd.org/dac/evaluationnetwork/pde These sites also have links to a number of videos illustrating key aspects of the Paris Declaration and the Evaluation Thank you for your attention

Remember Saturday 5th 2:20 PM – 3:50 PM in El Capitan B Panel Session 968: The PDE: A Joint Cross National Evaluation