Using RE-AIM as a tool for Program Evaluation From Research to Practice

What is RE-AIM RE-AIM is an acronym that consists of five elements, or dimensions, that relate health behavior interventions: –Reach the target population –Efficacy or effectiveness –Adoption by target settings or institutions –Implementation - consistency of delivery of intervention –Maintenance of intervention effects in individuals and populations over time Commonly used for evaluating efficacy trials (Phase III Research)

Defining Elements Reach — The absolute number, proportion, and representativeness of individuals who participate in a given program. Representativeness refers to whether participants have characteristics that reflect the target population's characteristics. For example, if your intent is to increase physical activity in sedentary people between the ages of 35 and 70, you wouldn't test your program on triathletes.

Defining Elements Efficacy/Effectiveness — The impact of an intervention on important outcomes. This includes potential negative effects, quality of life, and costs.

Defining Elements Adoption — The absolute number, proportion, and representativeness of settings and staff who are willing to offer a program.

Defining Elements Implementation — At the setting level, implementation refers to how closely staff members follow the program that the developers provide. This includes consistency of delivery as intended and the time and cost of the program.

Defining Elements Maintenance — The extent to which a program or policy becomes part of the routine organizational practices and policies. Within the RE-AIM framework, maintenance also applies at the individual level. At the individual level, maintenance refers to the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.

How do elements relate to planning? As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself. –Reach: HOW DO I REACH THE TARGETED POPULATION FOR INTERVENTION?Reach –Efficacy or effectiveness: HOW DO I KNOW THAT MY INTERVENTION IS EFFECTIVE?Efficacy –Adoption HOW DO I DEVELOP ORGANIZATIONAL SUPPORT TO DELIVERY THE INTERVENTION?Adoption

How do elements relate to planning? As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself. –Implementation: HOW DO I ENSURE THE INTERVENTION IS DELIVERED PROPRLY?Implementation –Maintenance: HOW DO I INCOPROATE THE INTERVENTION SO IT IS DELIVERED OVER THE LONG TERM?Maintenance

How is RE-AIM different from other evaluation approaches? RE-AIM draws upon previous work in several areas including: Diffusion of innovations, multi-level models, and Precede-Proceed. The primary ways that it is different is that it (a) is intended specifically to facilitate translation of research to practice, (b) it places equal emphasis on internal and external validity issues and emphasizes representativeness, and (c) it provides specific and standard ways of measuring key factors involved in evaluating potential for public health impact and widespread application.

Is RE-AIM used to design programs, or just to evaluate them? It is both. Although used more commonly at present to report results or compare interventions, it is also intended as a planning tool.

An Example Physical Activity Promotion in Primary Care: Bridging the Gap Between Research and Practice –Eakin, Brown, Marshall et al. (2004).

Translating and Disseminating Evidence- based Falls Prevention Programs into Community Li et al. (2008). American Journal of Public Health An Example of Application

Primary Aim: (a) To translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults; and (b) Using the RE-AIM framework (Glasgow et al., 1999), to pilot implement the program with a primary focus on reach, uptake (adoption), and implementation.

Secondary Aim: To evaluate the effectiveness of the program with respect to improvements in physical performance measures germane to falling. Tertiary Aim: To evaluate program maintenance with respect to the extent to which older adults would continue to practice Tai Chi beyond the period of the initial evaluation.

Translation Translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults Identify training objectives and elements Identify end users and dissemination partners Develop a dissemination package Expert evaluation Pilot testing

Program Evaluation Reach: A total of individuals/providers responded to the program promotion Effectiveness: defined as change in physical performance outcome measures taken at baseline and again at 12 weeks termination. Adoption (or uptake): defined as the proportion of local community (senior activity) providers that agreed to participate and implement the program.

Implementation: defined as the extent to which providers' implemented a Tai Chi class to participating older adults and the ability to conduct the various elements of program protocols, including the use of implementation plan, a 2-times weekly program schedule, distribution of program supplements (i.e., videotape and a guidebook), a class attendance rate of 75% or better over the 12-week class period.

Maintenance: defined at both the service provider level and participant level. At the provider’s level, it was defined as providers’ willingness to consider the program to be part of routine organizational provisions (assuming adequate financial resources). At the participant level, it was defined as the extent to which improvements in participants’ physical performance were sustained, and their continued practice of Tai Chi, 8-weeks after completion of the class.

REACH:87% (by study criteria); 45% by client attendance Dissemination Outcome ADOPTION: Six senior activity centers from five communities: 100% adoption IMPLEMENTATION: 75% completed; >85% class attendance; average 32 min. of home practice

Effectiveness: Improved physical performance and quality-of life measures (a)Functional Reach; (b) Up and Go; (c) Chair Stands, (d) 50-foot speed walk, and SF-12. Maintenances: Five centers continued; 87% participants continued Dissemination Outcome Impact Adopted by the State of Oregon: being implemented in four counties; three more starting this year

Conclusion-Tai Chi The evidence-based Tai Chi program is practical to disseminate and can be effectively implemented and maintained in community settings