1. Evaluating social safety nets at the World Bank: Country case studies –Jamaica AEA meetings 2010 Victoria Monchuk IEG World Bank 1

2. Evaluation building blocks ►30 country case studies: looking beyond portfolio 25 randomly selected desk- and interview-based 5 purposely selected field-based (Indonesia, Ethiopia, Colombia, Jamaica, Georgia) 3 in-depth Project Performance Assessment Reviews (projects in Ethiopia, Colombia, Jamaica) Examine the relevance and nature of the Bank’s engagement; country’s application of social risk management framework (SP Strategy, 2001); and the results in countries 2

3. 30 cases: in-depth and broad dataset ►In–depth: Case studies serve as in-depth understanding of how Bank’s approaches played out in a country context, explanations for the “why”, examples to illustrate points, anecdotes, etc. ►Breadth: Data set (n=30) of qualitative data and data that can be quantified to see trends across regions, low- and middle-income countries, etc. 3

4. Jamaica context ►2 nd poorest in Caribbean after Haiti ►Decades of slow growth (ave. 1.5%) ►Sensitive to shocks ►Poverty has declined, but deep pockets remain 49% of the poor are children Unemployment high and strongly related to poverty ►Debt burden severely restricts social spending (144% of GDP, interest payments of 64% of GDP in 2001) 4

5. Safety nets in Jamaica ►Safety net consists of: Transfers: food stamps, cash and in-kind to elderly/indigent Community-based programs: Jamaica Social Investment Fund School/health-based programs: school feeding, drugs for elderly ►2000 SSN reform: consolidate several welfare programs for families and elderly into one Conditional Cash Transfer program (discontinue food stamps) ►2001 Program for the Advancement through Health and Education (PATH) launched 5

6. Bank support to Safety Nets 2000-10 ►Analytical and technical assistance ►SSN lending: Social Safety Net project (2001-09) Social Protection project (2009-13) Social Investment Fund project (1996-2001) National Community Development project (2002-08)JSIF Hurricane Dean Emergency Recovery Loan (2007-11) Development policy loan (2010) PATH CCT 6

7. Look at projects and beyond ►Ex-post project evaluations assesses relevance and effectiveness of projects against their objectives Does not look much beyond the project ►Case studies assess the effectiveness of the whole SSN system and of the entirety of Bank support Relies on project performance and impact evaluations for outcomes and impacts 7

8. Case study design ►Context (economic, political, poverty) ►Country’s objectives and SSN systems ►Bank’s objectives and SSN support ►Assessment of relevance and effectiveness of Bank-supported SSNs ►Assessment of the Bank’s engagement in developing SSNs ►Ratings ►Hypotheses 8

9. Approaches ►Literature review ►Interviews with relevant Bank and IDB staff ►Country visit: Interviews with Bank staff Interviews with government clients (Planning Institute, Social Fund, MLSS) and data collection Field visits and interviews with parish office staff, school and health center staff, beneficiaries Interviews/focus group with development partners (IDB, DfID, EU, UNDP, CIDA) ►Follow-up interviews with Bank and IDB staff 9

10. 10

11. Project findings ►Social Safety Net project achieved its objectives of strengthening the SSN system, providing more cost- effectives social assistance, and targeting the poor and vulnerable ►Small impact on human capital investment and poverty ►Although not designed as such, the PATH program has been able to provide support in response to shocks 11

12. Project findings ►National Community Development project increased the access of poor communities to good quality basic services and roads and created some temporary jobs ►Modestly increased social and community empowerment ►Did not incorporate mechanisms for ensuring unskilled employment, did not measure impact of employment on poor households 12

13. Findings on Jamaica’s SSN ►Needs stronger emphasis on creating income and job opportunities and links to labor markets ►No sustainable mechanisms for responding to systemic shocks FFF response: scale up coverage (47%) and benefit levels FFF response: universal health and sec. school fees waivers ►Because fiscal room is minimal, it is important to put in place a safety net that can be scaled up in times of crisis and down in more stable times ►Developing a social protection strategy only in 2009 13

14. Case studies link to other evaluation building blocks ►Portfolio review gave clues that it was important to distinguish between instruments and design details Cases provided explanations for choices made in the country leading to an in-depth review of results frameworks in SSN projects ►FFF survey exposed Bank readiness to help countries respond to the crisis through SSN systems Cases showed how individual countries used available resources ►Public works review provided evidence on strengths and weaknesses Cases provided context for Public Works projects and other programs 14

15. Main take-aways: ►Ex-post project evaluations give specific information about project achievements and why ►Case studies help answer questions on the country and sector level, the effects on whole SSN systems, and the Bank’s strategic relevance ►Case study data triangulate with other sources ►Project evaluations: worldbank.org/ieg 15