Presentation on theme: "DC Home visiting Implementation and impact evaluation"— Presentation transcript:
1 DC Home visiting Implementation and impact evaluation Deborah F. Perry, PhDGeorgetown UniversitySarah Kaye, PhDKaye Implementation & Evaluation, LLCNovember 27, 2012
2 Importance of rigorous design Federal funding is contingent on rigorous implementation and outcomes evaluationEvidence-based home visiting (HV) modelsImplemented with fidelityDetermine impact of HV on children and families servedAll local agencies receiving funding for expanded HV must participate in the evaluation studyAlso could have to participate in the national evaluation study if DC is selected
5 Importance of STUDYING implementation Effective interventions cannot reach the majority of children and families with scaling up.Effective interventions, when implemented poorly, do not provide the same benefit for children and families (Durlak & DuPre, 2008).Implementing new programs and practices requires change (and change is hard).Practice can “drift” back to business-as usual without support and reinforcement (Breitenstein et al., 2010; McGrew et al., 1994).Children and families cannot benefit from services they don’t receive.
6 IMPLEMENTATION Questions Are Healthy Families America (HFA) services delivered with fidelity to the HFA model?Are providers operating with adherence to the 12 HFA critical elements?Are professional development activities improving staff capacity to delivery high quality HV services?How are implementation strategies related to fidelity of HFA implementation?How could implementation strategies be improved to enhance implementation outcomes?
7 Implementation Study Implementation Study Impact Study Implementation StrategiesImplementation OutcomesIntervention OutcomesImplementation StudyImpact StudyImplementation Strategies Review InstrumentReadiness: qualitative interviews and focus groupsFidelity: self report and independent raters checklistsAdherence to HFA’s critical elements: HFA Self-Assessment ChecklistCapacity: HV Capacity AssessmentMonitoring & Feedbackbased on the National Implementation Research Network’s implementation driversReadiness: agencies implementing HFAFidelity and Capacity measures to be developed
8 Impact Study Implementation Study Impact Study Monitoring & Feedback Implementation StrategiesImplementation OutcomesIntervention OutcomesImplementation StudyImpact StudyMonitoring & Feedback
9 Impact study components Random assignment of agencies awarded funding for HFA expansionCohort 1: training and implementation right awayCohort 2: baseline data collection for 6 months, then training and implementationOutcomes aligned with federal MIECHV benchmarks and DC Early Success FrameworkImprovements in early learning and development Improvements in early detection of developmental delays Improvements in family engagement and service agencies support Improvements in children‘s social, emotional, mental, and physical health
10 OutcomesWhat are the differences in outcomes when community agencies implement HFA versus a non-evidence-based home visitation model?Maternal depressive symptomsMaternal social supportMaternal efficacyBreastfeeding initiation and durationSleep routines (i.e., sleep location and positioning)These are examples…
11 Monitoring & Feedback Implementation Study Impact Study Implementation StrategiesImplementation OutcomesIntervention OutcomesImplementation StudyImpact StudyMonitoring & Feedback
12 RE-AIM Evaluation Framework ConstructData SourceRReach of evidence-based home visiting programs into the target population of high risk families with young childrenCentralized intake data systemEEffectiveness of interventions in achieving desired outcomesImpact studyAAdoption of evidence-based home visiting programs by provider agenciesIImplementation quality and consistencyImplementation studyMMaintenance of intervention effects in individuals and settings over timeDOH MISThe RE-AIM evaluation framework is designed to assess the public impact of evidence-based interventions, taking into account implementation issues as well as impact on child/family outcomes. This framework will guide the selection of indicators that will be monitored and shared with stakeholders over the course of the project.
13 Empowerment evaluation The Community Advisory Board to the evaluation will assist in:Identification of relevant indicators,Identification of stakeholders who could benefit from timely data about the status of implementation,Dissemination of data to pertinent stakeholders.Implementation and intervention indicators will be shared with DOH, HV Council, provider agency leadership, and other relevant stakeholders through defined feedback loops.