Presentation on theme: "Sudan Community Development Fund: Preliminary Slice I Impact Evaluation Results and Needs for Future Evaluations Abdulgadir Turkawi, Krishna Pidatala,"— Presentation transcript:
Sudan Community Development Fund: Preliminary Slice I Impact Evaluation Results and Needs for Future Evaluations Abdulgadir Turkawi, Krishna Pidatala, Tei Fujiwara, Ryan Sheely
Community Development Fund (CDF) National program that uses CDD approach to rapidly provide basic social infrastructure and services to war-affected and underdeveloped areas of North Sudan – i.e. In 4 states out of 15 states Two Phases Slice-1 (2006 -2008) - $25 million Slice-2 (2008 -2011) - $50 million CDF – Project Background
Project Background (2) Slice I (2006-2008) - US$ 25 million 4 states out of 15 states; 10 neediest localities within these 4 states; 20 communities per locality = 200 total communities Slice II (2008-2011) - US$ 50 million 4 states out of 15 states; 6 more communities added to the original 10 Slice-I localities; 19 new localities added in the 4 states in Slice-II to increase coverage within the states. Total Slice-I communities = 260 Total Slice-II communities = 380
Implementation Overview Baseline 1 – May 2007 (3 treatment communities per locality; 2 control communities per locality; 27 random Households in each community ) Follow-up Survey/Baseline Survey 2 – June 2008 Follow-up survey for Slice1 - panel survey for same households as in baseline 1 for Slice1 ; Baseline survey for Slice2 - 4 treatment communities per each new locality; 2 control communities per each new locality; 24 random Households in each community Going Forward – Final survey expected in 2011
Evaluation Strategy and Survey Design Targeting – based upon poverty & population assessments; 20 lowest ranked communities in each locality were selected to receive the CDF program. 21 st and 22 nd lowest communities in each locality picked as the control group Treatment & Control groups – selected communities with similar characteristics Cluster Random Sampling – Households in Treatment and Control groups were selected randomly
Selection of Treatment Communities Total of 20 Communities per Locality Needed to Ensure that there was at least one treatment Community per Administrative Unit Within each Administrative Unit, communities were ranked based on poverty, population, availability and condition of Basic Services and Population Number of Communities Chosen for Treatment Per Administrative Unit - based on above ranking (Poverty, population, availability & condition of basic infrastructure & services)
Selection of Control Group To construct a control group, all communities within each locality were ranked based on the number and condition of basic services and population The 21 st and 22 nd communities on the list were selected as a control group These were the communities that were not selected that were most similar to the selected communities
Survey Methodology Community and Household Questionnaires 50 Communities Chosen from all 10 Localities – 5 from each locality 3 Treatment Communities Randomly Chosen From Each Locality 2 Control Communities From Each Locality Selection of Households 27 Households Randomly Selected from Each of the 5 Communities Sampling Frame-Household Lists where available, “Spin the Pen” method where not available
Evaluation Questions The Slice 1 Baseline and Follow-Up Surveys were designed to assess the overall effectiveness of the project at meeting its objectives: Measurements : Access to Education, Health, and Water? Good Governance? Participation and Social Capital?
Results of Slice I Impact Evaluation Results Estimated Using Difference-in- Difference Approach Education Gains in enrollment, reduction in dropouts ▪ Decline in female dropouts Increases in number of classrooms, toilets, benches, and teachers dormitories Treatment communities 34% more satisfied with education after intervention, compared to control group
Results of Slice I Impact Evaluation Health Fewer statistically significant increases in health center functionality Increase in frequency of health center visits Satisfaction with health facilities significantly increased Water Fewer statistically significant increases in water quality Increased consumption of water Increased Number of Pump sets Increased Satisfaction with Access to Water
Results of Slice I Impact Evaluation Governance Increase in Reported Rates of Leader Compliance with Community Needs and Leader Responsiveness Decrease in Ease of Changing Leader Participation and Social Capital Increase in Community’s Ability to Solve Development Problems No significant increase in Participation in Community Activities or Meetings
Evaluation Challenges & Lessons Learned Sample attrition – possibility that some households could have moved by 2011 Data Management – difficulty in matching of some households from baseline & follow-up surveys Gender sensitivity & participation – 1 st baseline survey did not have any female respondents. Addressed this shortcoming in 2 nd baseline survey for Slice-II. Survey questionnaires too long – Need to condense follow-up questionnaires. Interview takes 1 hour 15 minutes.
Looking forward – Impact Evaluation 2011 - Final survey expected to be undertaken Impact Evaluation – Need for continued support from DIME for facilitation & technical expertise to the project Phase-II – Dependent upon the Referendum in January 2011 and the political landscape there after. We expect a more rigorous IE design for the next phase/project. Evaluation design - possible sub treatment interventions Survey design and management Sampling
Questions & Feedback Needed: Slice II Follow-Up Survey Budget constraints – For the final survey, should we reduce – (a) the number of households per community,or (b) the number of communities per locality ?
Preparing for Future Impact Evaluations Build Local capacity – involve local counterparts in IE technical design & analysis (as far as possible) Gender Sensitivity/Participation – IE expert on team to be a woman (based upon past experience) Focus on Project/Program – Develop project/program questions to be answered by IE National Statistics Bureau – Look at possibility to involve them in some way to build their capacity (most projects ignore them) DIME Support– need continued support from them. DIME to provide technical expertise, oversee IE work, ensure quality of work, etc.
Questions For Future Evaluations What is the effect of installation of solar electricity on health, education, and security outcomes? What is effect of social accountability mechanisms on Infrastructure Functionality? Due to implementation in progress, may not be able to evaluate until Phase II Will look for opportunities for evaluations in Slice II