MER Essential Survey Indicators Jenifer Chapman, PhD & Lisa Parker, PhD February 2, 2015

Learning outcomes  You will become familiar with the indicators and think about what they mean for your programs  You will know how to get the data  You will better understand how to choose programs for outcomes monitoring from your country HKID portfolio

MER Essential Survey Indicators  Move to outcomes, linked to program goals  Mandatory  Every two years

9 Essential Indicators  Representing holistic measures of child and family wellbeing  Linked to broader HIV response goals  Linked to broader child protection response goals  Vetted by broad stakeholder community

Indicator criteria  Amenable to change due to PEPFAR OVC programs in a 2 year period  Easy to measure by trained data collectors  Relevant over time and place  Indicators including questions that could be validated were prioritized  Many pilot tested

HIV status Percent of children whose primary caregiver knows the child’s HIV status Rationale: If HIV status is unknown, child will not access care & treatment (proxy for testing) Source: PEPFAR OVC TWG

Nutrition Percent of children <5 years of age who are undernourished Rationale: linked to infant morality and long- term child health and development Source: World Health Organization

Health Percent of children too sick to participate in daily activities Rationale: PEPFAR supports critical linkages to health services to improve functional well- being Source: MEASURE Evaluation

Legal protection Percent of children who have a birth certificate Rationale: Required to access essential services Source: DHS

Education: School attendance Percent of children regularly attending school Rationale: Important for child development; children in school are less likely to acquire HIV Source: UNESCO

Education: Progression in school Percent of children who progressed in school during the last year Rationale: More education is linked to better HIV awareness, higher contraceptive use, improved child well-being (among others) Source: MEASURE Evaluation

Early childhood development Percent of children <5 years of age who recently engaged in stimulating activities with any household member over 15 years of age Rationale: Early childhood stimulation is linked to long term child health and development Source: MICS

Perception of violence Percent of caregivers who agree that harsh physical punishment is an appropriate means of discipline or control in the home or school Rationale: Perception of violence is linked to use of violence; children experiencing violence show greater HIV risk behaviors Source: MEASURE Evaluation

Household economic resilience Percent of households able to access money to pay for unexpected household expenses Rationale: Resilience to economic shocks is linked to poverty, which impacts child and family well-being Source: MEASURE Evaluation

Disaggregation  By sex  By age group (where relevant):  0-4 years  5-9 years  years  years

Let’s discuss (30 mins) What questions do these indicators raise for you with respect to program implementation?  Hint: how does this affect targeting beneficiaries?

Feedback What did you discuss?

Getting the data  Figuring out which programs  Figuring out which approach: 1.Outcomes monitoring 2.Evaluation  Figuring out who will collect the data

Which programs?  Appropriate proportion of budget  Agency representation  Appropriate program scope and timeline  Strategic effort  Paying heed to other data collection efforts *Countries with total HKID funding of <1M USD/year are exempt from requirement

So we might have data from multiple programs in one country?  Yes. They cannot be aggregated.

What approach? Outcomes monitoring vs. Evaluation

Considerations  The information you need  Why you need it  When you need it Note that we have developed data collection tools for both approaches

EvaluationOutcomes monitoring Outcomes may be attributed to program* Attribution cannot be established Data valid at population level Data may be valid at local level* Larger number of indicatorsVery limited number of indicators 3-5 years usuallyEvery 2 years Complex samplingSimpler sampling Complex analysisSimpler analysis* Higher costLower cost

Methods for outcomes monitoring Cluster sample surveys vs. Lot quality assurance sampling (LQAS)

MethodAdvantagesDisadvantages Cluster sampling Sample large enough for sub-group analyses Simpler sampling design No weighting* Statistician needed for sample size calculation More expensive (larger) Lot quality assurance sampling Provides information valid at supervision area (SA) level May be cheaper depending on number of SAs Sampling frame needed for each SA Sample size will need to be increased for some indicators Values need to be weighted

Data collection tool

Which partner?  Surveys must be undertaken by appropriate institution, that is NOT providing services under program  Proven institutional capes:  Survey design and sampling (all methods)  Ethical and safe data collection  Data management and analysis

When? APR FY15 And again in two years.

Implications for your work (30 min)  What programs might require outcome monitoring in your country?  What are your next steps?

Feedback

Where can I find out more? Go to our website: our-work/ovc Jenifer Chapman: or Lisa Parker:

The research presented here has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L Views expressed are not necessarily those of PEPFAR, USAID or the United States government. MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.