1. The Evaluation of Programs Combating Female Genital Cutting: Data Sources and Options February 6, 2007 P. Stanley Yoder Macro International

2. Overview of presentation Assessing long-term impact DHS data Single survey data Challenges in designing evaluations

3. Current situation: Anti-FGC programs Great variety of program types - Public awareness media campaigns - Social mobilization in specific populations - Lobbying for legal action - Enlisting help from traditional and political leaders Very little empirical evidence of impact (Diop & Askew, book chapter in R.M. Abusharaf, ed. 2006)

4. Need for data on FGC program impact Statistical evidence for changes in FGC prevalence Possible use of DHS data? Surveys of sample populations - Baseline and follow-up surveys - Comparison of intervention and control regions

5. Challenges in collection of relevant data Conducting a survey in program areas only Need to have data on events 10 years ago Temptation to rely on program personnel rather than outside observers to evaluate DHS provides data on women 15-49 years old and not on young girls Need to describe situation taking account of secular trend

6. Use of DHS data for FGC evaluations Data on prevalence by age group of women Data on daughters (most recently cut) - Age at FGC - Identity of practitioner - type of cutting Does not provide data on all daughters

7. DHS data on prevalence by age group SenegalKenyaNigeriaTanzania 15-1924.8%20.3%12.9%9.1% 20-2428.0%24.8%17.0%13.7% 25-2928.4%33.0%20.8%15.2% 30-3430.1%38.1%19.4%16.0% 35-3930.5%39.7%22.2%16.0% 40-4430.3%47.5%22.2%18.8% 45-4930.6%47.7%28.4%22.9% Total28.2%32.2%19.0%14.6%

8. DHS data on distribution of FGC Prevalence by region Prevalence by ethnicity Prevalence by rural/urban residence Prevalence by religion (sometimes)

9. Example of DHS data for Senegal: Ethnicity FGC prevalencePercent of total pop. Wolof1,6%40% Serer1,8%16% Diola60%5% Fulani62%25% Mandinka74%5% Soninke78%3%

10. Limitations in use of DHS data for evaluations No complete data on FGC among young girls Sample size limits aggregation by small units such as those covered by an FGC program Provides data on FGC events 10-15 years ago and more only (women 15-24 years old)

11. Assessing long-term impact: Stand alone surveys Baseline and follow-up surveys - relatively expensive to design and implement - results after more than five years One survey with intervention & control areas - much cheaper - difficult to identify intervention & control areas, for they dont follow administrative units - very hard to match intervention & control areas

12. Example: Tostan evaluation for UNICEF New York Commissioned and funded by Child Protection Division of UNICEF New York Implemented through contract between UNICEF and Macro International Additional funding by USAID Two components: Quantitative and Qualitative - Quantitative: Centre de Recherche pour le Développement Humain (CRDH) - Qualitative: Population Council, Dakar

13. Quantitative component: intervention and control areas Designed and implemented by the CRDH Questionnaire for households and for women 15-49 years old Questionnaire similar to DHS Questions on knowing about Tostan, participation in Tostan, knowing about a Public Declaration, participation in a PD

14. Basic questions to answer How has Tostan affected: - average age of marriage? - cutting of young girls? How has participation in a Public Declaration affected: - average age of marriage? - cutting of young girls?

15. Three types of villages where FGC is practiced Basic comparison: Data on age of first marriage and rates of FGC in three types of villages: Type A: Tostan operated and population participated in a Public Declaration (PD) Type B: Population participated in PD without Tostan Type C: Control villages: No Tostan, no PD

16. Assessment of program impact What are the starting points for rates of FGC and for age of marriage in these three types of villages? What is the secular trend in these areas? How do the intervention and the control villages differ? To what extent can differences be attributed to Tostan activities?

17. Village type by prevalence of FGC for women Type A villages – FGC prevalence 64% Type B villages – FGC prevalence 81% Type C villages – FGC prevalence 87% Implication: The point of departure is different for the assessment of any change that may have occurred

18. Major challenges for village comparisons Matching control and intervention villages by: - ethnic composition - practice of FGC - literacy - development of social services Locating villages near Tostan villages that had no Tostan influence

19. Reporting on an evaluation: Being clear The questions to be answered The indicators to be followed How and why they are important The expectations of the program personnel The ways evaluators would assess the program The ways evaluators would measure success

20. Setting up a new evaluation Questionnaire that matches program activities Sample population that covers program areas plus control areas Choosing indicators that show process as well as impact Identifying a neutral agency to conduct survey Articulating how assessments will be made Plan final report that provides all this information