An impact evaluation of Ethiopias Food Security Program John Hoddinott, IFPRI (in collaboration with Dan Gilligan, Alemayehu Seyoum and Samson Dejene)

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 2 The Context of the Food Security Program Ethiopia has been one of the largest recipients of emergency food aid in Africa for the past decade emergency appeals approach has been costly: avg cost of $265 mn from , reaching > 5 mn people per year emergency appeals have had limited effectiveness at protecting productive assets and mitigating drought shocks In 2005, the Government of Ethiopia revised its strategy of distributing food aid emergency appeals replaced with a standing safety net in areas suffering from chronic food insecurity focus of new program was to provide more reliable and timely support to chronically food insecure households in more than 260 woredas (counties) across Ethiopia increased funding for complementary programs to foster graduation from the safety net

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 3 The Food Security Program Major components of the Food Security Program (FSP) 1.Productive Safety Net Program (PSNP) labor intensive public works use safety net to build productive community assets transfer payments in cash rather than food in some areas to improve market development through safety net 2.Other Food Security Programs (OFSP) Direct Support (DS): unconditional transfers to labor-scare households including elderly and disabled Other components: subsidized fertilizer, subsidized credit, other inputs or assets 3.Resettlement to other locations with more productive land Evaluation study will cover 1 & 2, not 3 resettlement

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 4 The impact evaluation: General focus Measure the impact that the FSP has on the well-being of the chronically food insecure population Investigate the complementary roles played by the PSNP and the OFSP in achieving positive outcomes for the food insecure Use the Food Security Bureaus M & E log frame to guide the choice of process and outcome indicators to be assessed

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 5 The impact evaluation: Specifics Process: Who participates? Measures of targeting effectiveness What do beneficiaries actually receive? –Timeliness –Purchasing power of cash transfers What public works are of actual use to beneficiaries? What are beneficiaries perceptions of how well the program works?

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 6 The impact evaluation: Specifics Outcomes Improvements in food security status. Examples (not exhaustive) –Percentage of households reporting no food gap –Improvements in dietary quality Changes in agricultural production –Increased use of credit, fertilizer, improved seeds –Percentage increase in crop production –Change in production of livestock products Changes in assets –Increased holdings of livestock and other productive assets –Reduction in distress sales of assets

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 7 The impact evaluation: Possible future topics 1.What are the prospects for significant graduation from the program? 2.Does the FSP lead to safety net dependency 3.What is the effect of the FSP on local labor and output markets? 4.How do these all of these impacts differ by food versus cash transfers? 5.What role does community participation play in the selection of works projects and their effectiveness?

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 8 Evaluation Design: Approach Empirically examining these issues requires new data Approach: Process Close interaction with the Food Security Bureau, CSA and the donor group on scope of evaluation, sample design and questionnaire Close collaboration with CSA on details of sampling, detailed questionnaire development and implementation Current status: Training of trainers is ongoing; survey to be implemented in July and August

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 9 Evaluation design: Sampling and survey Instruments Random sample of 68 woredas in the FSP from 4 major regions of Ethiopia; 2 enumeration areas (EAs) per woreda; 25 households per EA (N= 3500 households) Within EAs, sample stratified on participation in PSNP or Direct Support: 15 beneficiaries and 10 non-beneficiaries want comparison households to operate in the same markets and have similar distribution of unobservables Household and community surveys capture measures of program participation, wide variety of outcomes, control variables, transfers community selection criteria used to determine eligibility household knowledge of eligibility household consumption transfers from all FSP programs and labor supply in PSNP

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 10 Evaluation Design: Methods In some respects, an ideal evaluation design require before/after, with/without data based on random assignment In the case of the PSNP: Program has been purposively placed Program has been operating for at least one year Consequently, we are planning on doing the following: Interview both beneficiaries and non-beneficiaries (which gives with/without) Insert retrospective questions on past asset levels; crop choice; non- ag activity; assets (including livestock); past levels of food insecurity (which gives before/after)

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 11 Evaluation Design: Methods, contd Use a quasi-experimental approach to identification of FSP impact on household food security and welfare, Specifically, use the difference-in-differences matching estimator of Heckman, Ichimura and Todd (1997, 1998) We match using the recall data found in the baseline survey, detailed information on program eligibility and on shocks. This approach has proven to be very effective variables for the matching model in our earlier work on the impact of food aid from the ERHS. A further survey is planned for 2008 which will permit the assessment of the longer term impact of the program

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE Page 12 Summary The evaluation approach taken here is characterized by: Participation by stakeholders in design Collection of process indicators which gives a beneficiary perspective on the functioning of the program Rigorous impact evaluation