1. Difference-in-Difference Method in Quasi-Experimental Designs (Case: CARP) Dr. Iqbal Syed Hussain Chandra Nath Mishra Dr. Navneet Kaur 29 th April 2011 Impact Evaluation Praxis

2. What CARP tried to do Main Interventions: Improve iron intake in young children through supplements and through complementary foods; (supplements included IFA syrup containing 20 mg elemental iron in one ml syrup to be consumed twice a week for one year after consuming food) Improve young child feeding practice Improve de-worming Strengthened supply chain management for IFA 2 Child Anemia Reduction Program Funded by USAID, Managed by AED

3. Iron: The Scope of the Problem The World Health Organization (WHO) has categorized iron deficiency as one of the top ten most serious health problems in the modern world. Iron deficiency anemia (IDA): Impairs the mental development of over 40% of the developing world's infants and reduces their chances of attending or finishing primary school Decreases the health and energy of approximately 500 million women and leads to approximately 50,000 deaths in childbirth each year Is complex because it requires increased iron intake at critical stages of the life- cycle - before and during pregnancy and throughout early childhood Source: http://www.micronutrient.org/english/view.asp?x=579 3

4. Iron: The Scope of the Problem Iron deficiency has assumed even greater importance as evidence accumulates linking iron deficiency with mental impairment. Various tests of cognitive and psychomotor skills associate lack of iron during infancy and early childhood with significant levels of disadvantage, affecting IQ scores by as much as 5 to 7 points. Millions of children might be unable to complete primary school due to the impeded cognitive development they sustained as newborns and young children who couldn't get enough iron and iodine. Source: http://www.micronutrient.org/english/view.asp?x=579 4

5. Reducing anemia will decrease maternal and perinatal mortality, improve cognitive development and schooling in children, and increase physical capacity and work productivity in adults. About 42 percent of pregnant women and 47 percent of preschool children worldwide are anemic. The most common cause of anemia is iron deficiency, which is associated with 115,000 maternal deaths and 591,000 perinatal deaths each year (Stolzfus et al., 2004). 5 Iron: The Scope of the Problem

6. 6 Current Program guideline for anemia control for young children Screening of all children for anemia Those found clinically suffering from anemia appropriate treatment be given This has come with 10th five year plan objectives and goals. However actually what is followed at present is screening by ANM and those found anemic are given IFA when available Calculation of IFA small tablets (and currently provided IFA syrup bottles) also based on this criteria, they are not sufficient in number if and when all eligible children will receive them.

7. Program Period Started Training for the program Feb 2008. IFA syrup was supplied in May 2008 and repeat supply was required after one year. 5000+7000 IFA syrup bottles were supplied. A2Z Program Support continued till Dec 2009. End line study was conducted in January 2010. 7

8. Child Anemia Reduction Program: Area and Coverage StateProjectControl Uttar PradeshDistrict Sant Ravi Das Nagar District Allahabad (Block Gyanpur)(Block Handia) JharkhandDistrict KhuntiDistrict Latehar (Block Murhu)(Block Manika) 8

9. What is this programme about ? Child anemia control requires a package of services not just iron alone. Is it FEASIBLE to provide this package of services to children in rural Uttar Pradesh and Jharkhand? Is it possible to implement the program within the existing system? Do front line workers have the capacity to grasp and deliver what is necessary to control child anemia? 9

10. 10 Child Anemia Feasibility Study: Steps Formative Research Based on the findings and recommendation of TAG –Comprehensive package of services for intervention –Attempted through the system within system guidelines Training and Capacity building of AWW/ANM/ASHA and others providing services Enhanced Monitoring and evaluation –Anemia Prevalence –Worm prevalence –IFA syrup stock and distribution –Morbidity data for study and control area to check for morbidity in study area with reference to iron supplementation if any Follow up with AWW/ANM for distribution and tracking the supply of IFA syrup Post training support (mentoring) by ICDS sector meeting and joint block meeting facilitation by field staff Ensuring anemia control and worm control in mothers

11. 11 Comprehensive Package of Interventions Improving nutritional status by addressing complementary feeding practices appropriate to the age and ensuring energy rich dense food adequate in quality and quantity. Increasing iron intake of the target population through –Iron supplementation in form of syrup one ml twice a week (GOI policy) using dropper only (discontinue when sick) –Improving breast feeding practices Control of infections –Hookworm infection by de-worming and improved hygiene practice –Malaria (EDPT) Increasing iron stores at birth by improving the iron status of pregnant women with better ANC and IFA

12. IFA Syrup Supply Tracking all the way… Assumption was to give IFA in the month of May and follow up children for 9-12 months for their regular consumption Continuous monitoring only can improve the situation and detect issues 12

13. 13

14. Evaluation Design 14 Quasi Experimental Design KAP Survey using structured interview schedule Qualitative component (FGDs) Project AreaComparison Area KAP Survey using structured interview schedule Qualitative component (FGDs)

15. Why DID? Difference out unobserved heterogenity within data on project and control observation before and after the program intervention Address questions of attribution Increasing demands for rigor in evaluation Resources were available 15

16. Regression Equation yi = β0 + β1 treatmenti + β2 posti + β3 treatmenti*posti + ei Wherein post is a dummy variable, which =1 for endline, and =0 for before; Treatment is a dummy variable, which =1 if individual is in treatment block and =0 if the individual is not. DID estimator is based on strong identifying assumption that in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel paths over time 16

17. Based on the above regression equation the difference in difference can be computed as β3 as described below: 17 Treatment ControlDifference Beforeβ0 + β1β0β1 After β0 + β1 + β2 + β3β0 + β2β1 + β3 Differenceβ2 + β3 β2β3 The estimate for β3 is the DD estimator. It is the differential effect of treatment. β2 represents the time trend in the control group, β1 represents the differences between the two at the Baseline

18. time Y t1t1 t2t2 YaYa YbYb Yt1Yt1 Yt2Yt2 True effect = Yt 2 -Yt 1 Estimated effect = Y b -Y a titi 18

19. Time Y t1t1 t2t2 Y t1 Y t2 treatment control Y c1 Y c2 Treatment effect= (Y t2 -Y t1 ) – (Y c2 -Y c1 ) 19

20. Difference in Difference Result Table for change in Anemia Status (UP) 20 The result is assessed by the value of coefficient that shows the amount of change in the standardized score of dependent variable (i.e. level of anemia) for one-unit change in the standardized scores of the independent variable, while controlling the effects of other independents. The p value with a differential change of 10.2 percentage points has been established as statistically significant

21. Anemia Prevalence Difference 12.9% Reduction of Anemia 2.3% Reduction of Anemia 84% Increase in non anemic children 10.6% Reduction of Anemia due to Intervention 21