1. ECA: New Frontiers in Achieving Results * Plenary April 1, 2008, Improving the Lives of People in ECA Vinod Thomas, Director-General Independent Evaluation Group * This presentation is based on IEGs recent evaluation findings. Ann Flanagan and Jiro Tominaga contributed to its preparation.

2. Progress and shortfalls in world development Per capita economic growth averaged 2% word-wide, 4% in developing countries and 6% in ECA (2000-06) Income levels diverged across countries, but converged across populations (due to rising income in populous China and India) Social indicators – infant mortality, school enrollment – tended to converge worldwide since 1990 The proportion of the poor declined since 1990, but the absolute number of poor is still around 1 billion Environmental degradation continued to rise everywhere -- especially pollution, land degradation, deforestation Progress in corruption has been slow in most countries, while stronger in ECA (1999-2006)

3. Growth, distribution and poverty

4. Attribution of results is not easy International Financial Institutions Local Policymakers Policies Country Outcomes Knowledge Political Economy Development Effectiveness Private Capital Flows

5. I. Connecting operations to results ECA Projects & AAA Moldova: AIDS Control Turkey: Earthquake Romania: Rural Educ Bulgaria: Rev. Admin Kyrgyz: Social Prot Ukraine: Peoples Voice Etc. Equity Efficiency Sustainability Quality of Peoples Lives Project-Program Public-Private Country-Global Knowledge- Lending 2008 2015 2030 Time

6. 1. WB projects outcomes improving Source: Staff calculations, IEG Database

7. …. but with program disconnects Summary Result Matrix Satisfactory CAE Unsatisfactory CAE Satisfactory Portfolio 58.8%29.4% Unsatisfactory Portfolio 5.9% Summary Result Matrix Satisfactory CAE Unsatisfactory CAE Satisfactory Portfolio 55.8%35.1% Unsatisfactory Portfolio 2.6%6.5% World BankECA Source: Staff calculations based on IEG project ratings (IEG Database) and IEG CAE ratings (1985-2006)

8. 2. Private sector can complement outcomes Development outcome and IFCs profitability (ECA) Additionality and project development results (ECA) Note: ECA region by number of projects based on 178 projects from 1996-2007 XPSRs.

9. Need public-private coordination for better results IEG reports document crucial gaps in WB-IFC collaboration, but there are examples of linkages: Leasing: IFC leasing projects and technical assistance contributed to the Banks rural agriculture leasing projects in Azerbaijan, Kazakhstan, Ukraine BTC pipeline: In Azerbaijan the Bank focused on transparency and fiscal management while IFC concentrated on mobilizing consortium, social and environmental sustainability and SME linkages. Sub-national finance: The Bank maintained relationships with regional governments. IFC added value with its ability to finance without a sovereign guarantee.

10. 3. Knowledge, lending and transition AAA rated as moderately effective or better in influencing government decisions by 2/3 of the MIC report client respondents: e.g., Ukraines WTO accession process Integration with country strategies: knowledge services provided entry for relationship with the government on public sector reform Knowledge-led strategies: QAG ratings of AAA programs in 8 countries with high budgets for knowledge lower than those with lower shares of AAA Difficult to use AAA on its own as a strategic instrument Some countries to agree on cost-sharing for AAA based on multi-year program – Kazakhstan JERP IEGs forthcoming AAA evaluation: 12 cases including Romania and Serbia. Role in informing Bank strategy? Link to lending? When AAA can be decoupled from lending? Adequacy of results framework and follow up in AAA? Source: IEG, Development Results in Middle-Income Countries; and Evaluation of World Bank Support for Public Sector Reform

11. Lending & policy: PSR lending, CPIA 99-06 ECA has the highest rate of CPIA governance score improvements for countries getting PSR lending The rate of improvement for non- borrowers is almost as high External factors e.g. EU accession may be in play PSR Loan YesNo AFR70%(30)47%(15) EAP70%(10)56%(9) ECA90%(20)86%(7) LCR75%(20)25%(8) MNA57%(7)0%(2) SAR50%(6)0%(1) Total73%(93)48%(42) Source: WB CPIA Scores and IEG staff calculations Note: Total number of countries in parenthesis

12. 4. Regional/global and country programs With the growing importance of trans-border issues, regional programs (such as building swifter trade routes Southeast Europe) and global programs (such as GEF) have vast potential At $3.5 billion in 2006, regional programs account for less than 1 percent of the Banks total project and partnership financing. Effectiveness hampered by lack of connection between country programs and regional/global initiatives. M&E especially weak in regional and global programs Source: IEG, Annual Report on Operations Evaluation 2004 and 2005.

13. II Role of M&E in the results agenda Source: IEG Database

14. Results chain and program effectiveness A well articulated results chain would (ARDE 2006): Results-based CASs, if done well, puts the focus on how projects add up to country impact Emphasize realism in setting objectives – a problem in 50% of Country Assistance Strategies (FY2001-FY2005) evaluated by IEG Track key synergies across sectors – for example, infrastructure for education results or environment for health results Focus on capacity building – after all, country M&E determine the application of a results chain in countries

15. ECA s experience in M&E Performance-based budgeting of various types being followed in Czech Republic, Kazakhstan, Russia, Slovenia and Turkey Especially EU accession countries are following OECD standards; some are becoming donors -- Czech Republic, Latvia, Poland and the Russian Federation. Greater accountability requires M&E capacity Projects: The South East European trade and transport facilitation project introduced electronic forms showing duties due. Turkey brought extra-budgetary funds (that had undermined fiscal discipline) into the budget and parliamentary scrutiny

16. Impact evaluation can strengthen M&E IE can promote or question programs – a crucial role of knowledge Found the T&V extension system used in Kenyan agriculture to have no impact on agricultural production It tracks linkages across sectors – including unintended and indirect Showed the linkage between a nutrition program in Bangladesh and secondary schooling for girls education It de-politicizes decision-making – contributes to sustainability Evidence-based case for the Mexican Opportunidades program made it difficult for politicians to discontinue support for after change of government Initiatives at the Bank: Development Impact Evaluation Initiative; IFC Advisory Services Impact Evaluation Initiative; Spanish World Bank Trust Fund for Impact Evaluation; African Impact Evaluation Initiative; IEG Impact Evaluation Program Challenges ahead: be demand-driven; enter programs ex ante; be theory based; focus IE on the most relevant issues

17. ECA lags in impact evaluation Source: IEG staff calculations (draft), DIME Database and Poverty Impact Evaluations Database Note: The Poverty Impact Evaluations Database contains evaluations conducted by both Bank and Non-Bank staff.

18. Four cutting edge challenges in ECA Project-program: Especially where Bank lending share is small, there is a premium on exploiting linkages across projects and sectors for the greatest country impact Public-private: It is vital to develop public-private partnerships and to make the most of public and private linkages Knowledge-lending: The knowledge-lending axis is exploited differently across countries Country-global: The country-based approach must be reformed to integrate tough issues of climate change, governance, migration…

19. Thank You