ADB’s Evaluation Architecture and Country Evaluations Guidelines Mala Hettige Senior Advisor Independent Evaluation, ADB Presentation at the meeting of Sub-Regional Development Institutions Johannesburg, 27 April 2014

Asian Development Bank (AsDB)  Established in 1966 – in Manila, Philippines  67 shareholding member countries  Provides $ 21b in development assistance  About 760 projects  5 regional department and PS department  Managed by MDG, 6 VPs  3000 staff in HQ and 29 RMs + 3 rep. offices  Administrative Budget about $600 million

Independent AsDB  Abides by OECD’s 4 dimensions of independence  DGE reports to the Board through the Development Effectiveness Committee (DEC)  Reports public upon DG-IE approval, discussed under Chair, DEC  DGE appointed/removed by DEC – 5 year non-renewable term.  Work Program and separate budget approved by Board  DGE has authority over staff recruitment  Staff subject to backward and forward conflict of interest policy

Major Evaluation Products Annual Reports Country Assistance Program Evaluations Corporate Evaluation Studies Impact Evaluation Study Thematic Evaluation Studies Project/Program Performance Evaluations Technical Assistance Program Evaluations Real- time Evaluation Studies Project Validation Reports Other Evaluation Reports 4

Accountability and Learning 5

Evaluation Information System (EVIS)  A database of evaluation lessons  Provides users easy access and search capability  Lessons by sector, theme, topic, country etc.  EVIS is the first system of its kind  Accessible through 6

Management Action Record System (MARS)  Tracks ADB Management actions on recommendations  IED Validates actions taken by Management  Reports this validation to the Board  Computerized and available to staff and Board  Departments/offices update progress on actions

Country Evaluations and ADB Operations Country Partnership Strategy Project Identification /Preparation Appraisal / Approval Implementation Country Evaluation 8

History of Country Evaluations in ADB CAPEs (Country Assistance Program Evaluations)  1998 first CAPE – Peoples Republic of China  2002 Mongolia 2002 questioned choices made  2005 Bhutan had recommendations + criteria ratings  2006 Indonesia added ratings by sector  2007 India had contribution to themes+ value added  2007 China had Strategic and Institutional positioning  2008 Sri Lanka considered the implications of conflict  2008 Philippines considered political economy factors  2009 Cambodia top-down + bottom up assessment  2010 Lao existing guidelines applied  2014 Tajikistan assessment of objectives- ongoing

Evolving Guidelines for CAPEs in AsDB  From 1998 to 2006 no guidelines  First Country Evaluation Guidelines 2006  Revised Guidelines issued 2010  CAPE/ validation of a country self evaluation mandatory prior to approval of a CPS  Now doing a few revisions to the guidelines  Also preparing validation guidelines

Background: GPS on Country Evaluations GPS for CSPE established in 2008 Based on OECD–DAC evaluation principles Draws on the 2007 ECG Review of CSPE A core GPS criteria allows comparability An optional GPS criteria helps accountability and learning

Application of GPS: of Country Evaluation Name of study differs and number differs Coordinated with the upcoming country strategy Substituted with validations or learning products About 9-12 months, pps, $ 250+, Lending/non lending, about 10 years (2-3 cycles) Application of principles, approach taken differs Overall Rating and Evaluation Criteria followed by about half. But try not to emphasized much

Objectives : Accountability – Board on results over a period Learning - engage with stakeholders on future CPS Timing : to feed into the next strategy Coverage : Long enough to assess results and Recent to ensure findings are relevant Key Aspects of CPSE

GPS for Country Strategy and Program Evaluation GPS for CSPE established in 2008 Based on OECD–DAC evaluation principles Draws on the 2007 ECG Review of CSPE A core GPS helps allows comparability An optional GPS helps accountability and learning

Core Criteria Relevance Effectiveness Efficiency Sustainability Impact Optional Criteria Positioning Coherence Institutional Development MDB Performance Borrower Performance Partner Coordinatio n Evaluation Criteria

Relevance GPS : Relevance refers to the degree to which MDB strategy and program were consistent to critical needs and long-term plans of the country

Relevance –> Strategic Relevance Can include optional criteria on positioning – how well did MDB respond or anticipate Evolving development challenges Priorities of the government Build on MDB’s comparative advantage Consider support from other development partners Also can include coherence – extent to which there were measures to Foster internal + external synergies within MDB Foster external partnerships that promote cost- effective division of labor through complimentarities

Efficiency GPS: Efficiency refers to the extent to which the design and delivery of assistance were most cost effective.

Effectiveness GPS: Effectiveness refers to the extent to which the assistance instruments achieved the intentions and objectives set.

Sustainability GPS: Sustainability refers to the likelihood that actual and anticipated results will be resilient to risks beyond the program period.

Impact GPS: Impact refers to an MDB’s contribution to long-term changes in development conditions.

Optional Criteria Institutional Development MDB Performance Client Performance Partnership Performance Other concerns By sector? What about cross sector impacts? By objectives? How to attribute and measure? How to account for unintended impacts?

IED Website

Thank You! Follow Us