1. An OVC Survey Toolkit Jenifer Chapman, PhD February 2, 2015

2. Survey toolkit  Tools: En, Fr  Tool Manual: En, Fr  Template protocol  Data collector training manual & materials  Data analysis guide  Data management guidance  Pilot test report  Psychosocial wellbeing measurement supplement  MER Supplement

3. Why was the toolkit developed?  Standardize population-level data beyond what is available from routine surveys & outcomes monitoring  Reduce the burden

4. Is the toolkit relevant to me?  Yes!  Toolkit should be used for all PEPFAR-funded OVC program evaluation  Toolkit can support collection of essential survey indicators through evaluation  You should understand how toolkit can support your work & when to recommend it to partners

5. When is the toolkit relevant to me? Tools are useful if your question is: 1.Is my program having, or did my program have an impact on the children and households it reached? 2.What are the characteristics of children and their caregivers in my country, state/province or district/area, in terms of education, health, protection, and psychosocial support?

6. When is the toolkit relevant to me? Tools are useful if your question is: 3.Where do the children most in need of program support live? 4.Approximately how many children need services or support? 5.What are the needs of my program’s registered beneficiaries, in terms of education, health, protection, and psychosocial support?

7. These are NOT the right tools If you want to know:  Which children in selected communities to target  How a particular child/household is faring  What services to provide or refer for a particular child / household

8. These are NOT the right tools If you want to know:  The number of children/households that are receiving program support, and the types of support received  Whether staff are carrying out their responsibilities  Whether interventions are being implemented as planned

9. And, why a special OVC survey? DHS & MICS  interview all households, and not specifically program households (beneficiaries)  include some, but not all of the OVC core indicators

10. Implementation considerations 1.Data collection by trained data collectors, not service providers 2.Documented protocol is required 3.Protocol with tools needs to undergo ethical approval 4.Tools require pilot testing in new settings

11. What do the tools measure? Program outcomes amenable to change in 2 years through PEPFAR OVC programs

12. Core questionsOptional  Household schedule* (10)  Changes in household composition (4)  Demographic information* (7)  Work* (3)  Access to money (9)  Shelter (1)  Household food security (6)  General health (2)  Caregiver support (4)  Parental self-efficacy (1)  Basic HIV/AIDS knowledge* (7)  HIV testing* (3)  Attitudes to condom education (1)  Household access to services (1)  Household Economic Status (25)  Progress out of Poverty Index or similar (country specific)  Dietary Diversity (1)  Perceptions and experience of abuse, exploitation and violence (forthcoming)  Gender roles and decision making power* (10)  HIV/AIDS attitudes* (4) *DHS, bold=MER Caregiver questionnaire

13. Core questionsOptional  Confirm demographics (5)  General health & disability, too sick to participate in daily activities (4)  Birth certificate (2)  Vaccinations (<5 years) (11)  Fever (<5 years)* (1)  Diarrhea (<5 years)* (1)  Shelter (1)  Experience of neglect (<5 years) (2)  Slept under mosquito net* (1)  HIV testing experience* (2)  School attendance*, progression, drop-outs, missed school days (5+ years) (9)  Work for wages (5+ years) (2)  Early childhood stimulation (<5 years) (2)  Food consumption (2+ years) (8)  Child access to services (1)  Weight*, Height*, MUAC  Fever: extended* (4)  Diarrhea: extended* (3) Dietary diversity (1) Child questionnaire (ages 0-9)

14. Child questionnaire (ages 10-17) Core questionsOptional  Confirm demographics* (5)  Identity of caregiver (1)  Daily log (6)  School attendance*, progression, drop-outs (9)  Chores (3)  Work (7)  Food consumption (8)  Alcohol consumption (3)  Birth certificate (2)  General health & disability, too sick to participate in daily activities (3)  General support (4)  Basic HIV/AIDS knowledge* (7)  HIV testing* (3)  Child access to services (1)  Weight, Height, MUAC  Dietary diversity (1)  Perceptions/experience of abuse, exploitation, violence (forthcoming)  Child development knowledge (6)  HIV/AIDS attitudes and beliefs (4) Sexual behavior (13-17 yrs) (5)

15. Becoming familiar (20 mins) In pairs, go through questionnaires & match the questions to the MER essential survey indicators

16. OVC_HIVST% of children whose primary caregiver knows the child’s HIV status OVC_NUT% of children < 5 years of age who are undernourished OVC_SICK% of children too sick to participate in daily activities OVC_BCERT% of children who have a birth certificate OVC_SCHATT% of children regularly attending school OVC_PRGS of children who progressed in school during the last year OVC_STIM % of children < 5 years of age who recently engage in stimulating activities with any household member over 15 years of age OVC_CP % of caregivers of active beneficiaries who agree that harsh physical punishment is an appropriate means of discipline or control in the home or school OVC_MONEY% of households able to access money to pay for unexpected household expenses

17. Let’s discuss!

18. Remind me, what does the toolkit include?

19. Questionnaires 3 questionnaires

20. En Français

21. Tools Manual  Describes tools question by question  Outlines how / when the tools may be used  Includes basic implementation guidance

22. Template protocol  Resembles an actual research protocol  Includes consent/assent forms, information sheets  For each section, required information is outlined, as well as issues to consider

23. Data collector training manual  Describes the structure & content of a 6-day training  Includes PPT slides with speaker notes, and all handouts

24. Data analysis guide  Describes analyses by question  Includes refresher information on analysis and charting

25. Data management guidance  Provides standardized guidance  Outlines steps for database design  Describes best practices in data entry and cleaning

26. Pilot Test Report Describes:  methods  findings  changes made to tools after pilot tests

27. Psychosocial supplement  Resource for investigators  Questions piloted and learning in 6 areas:  social support  self-efficacy  self-esteem  hope  functional aspects of well- being  parental stress, self-efficacy

28. MER Essential Survey Indicator Supplement  Resource for Missions & research institutions  Methods for outcomes monitoring

29. Tools in a Toolbox  Remember: There is no single data collection tool that can meet all OVC program targeting, case management, monitoring and evaluation requirements.  This set of survey tools responds to distinct information needs related to program planning and evaluation, and fills a tools gap.

30. Where can I find out more? Go to our website: http://www.cpc.unc.edu/measure/ our-work/ovc Email: Jenifer Chapman: jchapman@futuresgroup.com or Lisa Parker: lparker@futuresgroup.com jchapman@futuresgroup.com lparker@futuresgroup.com

31. The research presented here has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. Views expressed are not necessarily those of PEPFAR, USAID or the United States government. MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.