Evaluating Dissemination of AHRQ CER Products Darren Mays, PhD, MPH Department of Oncology Georgetown University Medical Center Lombardi Comprehensive Cancer Center Washington, DC

Research to Practice Gap Bernhardt, Mays, & Kreuter, 2011

How will iADAPT help? Poised to make progress – What approaches work? For whom? In what settings/conditions? Presents an evaluation challenge – Creative methods/approaches – Diverse populations – Different clinical areas Need for a flexible evaluation framework

What is RE-AIM? Evaluate public health impact Focus on dissemination Barriers include design, setting, approach Impact assessed on multiple domains RE-AIM domains: – Reach, Efficacy/Effectiveness, Adoption, Implementation, Maintenance Glasgow, Vogt, & Boles, 1999; Glasgow, Lichtenstein, & Marcus, 2003

RE-AIM Domains Reach – Did the CER products reach the intended population(s)? Participation rate(s), characteristics, baseline “risk” Efficacy/effectiveness – What is the impact on intended outcomes? Clinical outcomes, CER product utilization, occurrence of harms/unintended consequences Refer to RE-AIM domains handout; Glasgow et al., 2006

RE-AIM Domains Adoption – Did the intended units use the CER product(s)? Participation and characteristics of setting(s), delivery agents, barriers to adoption Implementation – Were the CER products implemented as intended? Adherence, fidelity, technical success Maintenance – What is the long-term impact of CER products? Long-term efficacy/effectiveness, sustained implementation, barriers to long-term use Refer to RE-AIM domains handout; Glasgow et al., 2006

Determining Impact Quantitatively determining impact Original application – Reach x Efficacy = Impact RE-AIM overall impact – Product of all 5 domains – Requires quantifiable measures Glasgow, Vogt, Boles, 1999; Glasgow et al. 2006

Application to iADAPT? REAIMAudience(s)Clinical Area(s) In Person CHW Outreach Acad. Detailing Med Low Med Patient Provider Diabetes Group CER Training School Group Therapy Med High Low Med Low Med Low Med Policy Patient Multiple Heart Disease Diabetes eHealth Clinic Kiosk Web Patient Portal Med Low Med High Med High Patient Patient & Provider Diabetes Print/Media Targeted VideoMed PatientHeart Disease Adapted from Glasgow et al., 2001

A Closer Look Clinic Kiosk R: n = 200 patients Well-controlled diabetes E: Small effect size 200 CERSGs (~1/pt.) A: 75% of clinics I: 50% completion rate Technical problems M: Few support resources Limited patient interest Web Portal R: n = 1,000 patients Poor diabetes control E: Medium effect size 2,500 CERSGs (~2.5/pt.) A: 100% of clinics I: 75% completion rate Few technical problems M: Minimal maintenance Low-cost to direct patients

Conclusions A flexible evaluation framework Multi-domain evaluation approach Identify facilitators, barriers, and future directions Creative approaches may be needed!

RE-AIM Resources NCI DCCPS web site for RE-AIM – Resources include: – Figures/graphics illustrating key concepts – Checklists and planning tools – Example measures – Publications, presentation

References Bernhardt, JM, Mays, D, & Kreuter, MW. (2011). Dissemination 2.0: Closing the gap between knowledge and practice with new media. J Health Comm, 16(S1), Glasgow, RE, Vogt, TM, & Boles, SM. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. AJPH, 89(9), Glasgow, RE, et al. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management. Patient Ed. & Counsel., 44, Glasgow, RE, Lichtenstein, E, & Marcus, AC. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. AJPH, 93(8), Glasgow, RE, et al. (2006). Using RE-AIM metrics to evaluate diabetes self-management support interventions. AJPM, 30(1), 67-73