1. Image courtesy of Naypong at FreeDigitalPhotos.net
Selecting an Evidence-based Intervention with the Best Fit
Talking Points:
This session will cover selecting an evidence-based strategy (EBI), you will find an strategy that fits best with your goals and objectives, organization, or community. Selecting the best match may help you be more successful in reaching your goals.
Image courtesy of Naypong at FreeDigitalPhotos.net
The Cancer Prevention and Control Research Network is supported by Cooperative Agreement Number 3 U48 DP S8 from the Centers for Disease Control and Prevention’s Prevention Research Centers Program and the National Cancer Institute. The content of this curriculum is based upon findings and experiences of workgroup members and does not necessarily represent the official position of the funders. 

2. Selecting Best Fitting EBI
Where Do We Stand?
Establishing Goals & Objectives
Assessing your Community
Finding EBI
Selecting Best Fitting EBI
Adapting EBI
Talking Points:
We are now moving to selecting Best Fitting Evidence-based Interventions (EBI) after we have found possible sources of evidence.
Implementing & Evaluating
*Adapted from Brownson et al. (2017). Evidence-Based Public Health. 3rd ed. New York, NY: Oxford University Press.

3. Session Objectives
Describe basic principles for selecting one or more EBIs that fit your goals and objectives
Assess the fit between an EBI program and your objectives, population, and setting
Combine EBI programs and strategies to target multiple levels
Talking Points:
With the different kinds of EBI in mind, after this session, you will be able to:
Describe basic principles for selecting an intervention that fits the organization and population
Assess the fit between interventions and the organization and population
Select an intervention that fits the organization and population

4. Assess EBI Fit 3. Select EBI based on fit and strength of evidence
1. Obtain information about evidence-based interventions
2. Assess fit by comparing EBIs with assessment findings according to fit criteria
3. Select EBI based on fit and strength of evidence
Talking Points:
The selection process entails three major steps, which include
1) Reviewing information about different Evidence-based interventions
2) Assessing fit
3) Selecting based on fit and strength of evidence.
We will introduce in more detail what materials and input are needed for each step.
We have a tool (HANDOUT) to help guide you through the process.

5. 1. Obtain Information about EBIs that address your goal
Find information about EBIs that address your objectives
Websites that are disseminating information about EBIs
Journal articles or other literature about relevant EBIs
Talking Points:
Begin to collect information about EBIs that address your health goal. The websites reviewed in Session 3 are one source of information. Initially you will want to pull general information and then, as you narrow down your choices, you will want to pull more detailed information to assess fit. More detailed information may be gotten by looking at the resources websites are disseminating with the EBI or by pulling journal articles about the EBI.

6. CRC morbidity & mortality
Locate EBIs that address the factors and determinants identified in your assessment
Determinants Behavioral/Environmental Problem Factors by Levels
Knowledge
Beliefs/attitude
Confidence
CRC morbidity & mortality
Individual: Not screened
Provider: Does not recommend
Awareness
Knowledge
Organization: Lacks
accessible hours; no transportation
The first step in assessing fit, involves going back to your assessment findings and the determinants and behavioral and environmental factors that are contributing to your problem and which you want to target.
Community: GI not available to do diagnostic tests

7. 2. Assess EBIs Fit to Goals and Objectives
________
EBI 2
________
Fit criteria
Health goal
Behavioral objectives
Environmental factors
Determinant objectives
EBI 1
________
Compare
Talking Points:
Comparing which EBI match assessment findings is a practical tactic because it’s easy to weed out EBI that do not fit well.
You can match an EBI to some of the areas listed here such as your health goals or behavioral or environmental factors.

8. 2. Assess EBIs Fit to Goals and Objectives
________
EBI 2
________
Fit criteria
Health goal
Increase colorectal cancer screening rates
EBI 1
________
Compare
Talking Points:
For basic fit, start with the health goal. In this example you would be looking for EBIs that successfully increased colorectal cancer screening rates.

9. 2. Assess EBIs Fit to Goals and Objectives
Fit criteria
Behavioral objectives
Patients
Decide to get screened
Complete and return test (iFOBT/FIT)
Providers
Recommend screening
EBI 3
________
EBI 2
________
EBI 1
________
Compare
Talking Points:
From within the pool of EBIs that successfully increased colorectal cancer screening rates you then would look for those that addressed the specific behavioral objectives identified during your assessment process. You might look for EBI that targeted change at the level of both patients and providers.

10. 2. Assess EBIs Fit to Goals and Objectives
Fit criteria
Environmental objectives
Setting
iFOBT/FIT kits available
Convenient hours and location
Community
GI providers available to do diagnostic tests
EBI 3
________
EBI 2
________
EBI 1
________
Compare
Based on your assessment findings, you might also be looking for EBIs that target change at the environmental level – in this case the levels of setting and community.

11. 2. Assess EBIs Fit to Goals and Objectives
Fit criteria
Personal determinants
Patient
Knowledge
Beliefs/Attitude
Provider
Aware when due
EBI 3
________
EBI 2
________
EBI 1
________
Compare
Talking Points:
As you winnow through the available EBIs you also want to look to see if they target the factors that determine the targeted health behaviors. In our example, we want patients to decide to get screened. You then want to identify EBIs that target the determinants that you have identified as central to your populations making the decision to get screened. For example, are EBIs targeting knowledge and beliefs about CRC screening?

12. 2. Assess EBIs Fit to Goals and Objectives
First step of fit assessment will result in lists of EBIs
EBI Programs (e.g., from Cancer Control Planet)
Community Cancer Screening Program (CCSP)
Family CARE (Colorectal Cancer Awareness and Risk Education) Project
Flu-FIT and Flue-FOBT Program
EBI Strategies (e.g., from the Community Guide)
One-on-one education
Client reminders
Provider reminders
This is the first step of the fit assessment and will result in one or more EBIs that fit some if not all of the criteria. These EBIs may include programs such as those listed on the slide. They may also include broader strategies. The next steps of the fit assessment are most relevant to EBI program but can also be applied to strategies.

13. 2. Assess Fit of EBI programs to context
Fit Criteria
Delivery methods
Priority population
Organization/coalition/ community (capacity and resources)
EBI 3
________
EBI 2
________
Compare
EBI 1
________
Talking Points:
In the next step of the fit assessment you will explore the EBIs fit with your specific delivery methods, priority population, and organization/coalition/community capacities. EBI strategies tend to be flexible and more easily adapted to different contexts. EBI programs often were developed for a specific population and organizational context. Assessing the fit of the EBI programs can help you determine how much adaptation might be required before you can use them in your context.
Community Assessment Findings

14. Fit of Delivery Methods
Community assessment:
Who or what delivery methods are available?
What methods will best reach your priority population?
Evidence-based intervention:
Delivery setting, format, program implementers, channels used
Talking Points:
Delivery methods:
What kind of methods were used to reach the users of the EBI? What was the delivery format (e.g., group education vs one on one education) etc.
Compare the EBI delivery methods with those your community or priority population has access to or prefers.

15. Fit to Priority Population
Community assessment:
Priority population’s characteristics in your area
Evidence-based intervention:
Statistics*
Pictures*
Language*
Talking Points: In the best of all possible worlds, the EBI will have been developed for and tested with a population similar to your own.  However, if it was tested with a different population you need to ask if is plausible that it may also work with your population. Is their evidence from a systematic review that the underlying strategy has been effective in populations like your own? Does it target the factors that are contributing to problem in your population. If a fit seems plausible, you may need to spend some time adapting the EBI materials and delivery for your population. We will talk more about adaptation in the next module.
American Cancer Society, Surveillance Research, 2013
*Kreuter M.W., Lukwago, S.N., Bucholtz, D.C., Clark, E.M., & Sanders-Thompson, V. (2003) Achieving cultural appropriateness in health promotion programs: targeted and tailored strategies. Health Educ Behav, pp

16. Fit to Capacity and Resources
Community assessment:
Existing programs/ facilities
Facilitators and barriers for implementation
Infrastructure, leadership, other roles
Capabilities, willingness, and resource
Evidence-based intervention:
What is needed for implementation?
Which contextual factors influenced effectiveness?
Talking Points
You also want to consider your organization’s capacity and resources in the community to determine fit. Consider:
Which behaviors/ determinants already addressed by existing programs/ facilities
Environmental facilitators and barriers for implementation
Infrastructure, leadership, other roles
Capabilities, willingness, and resources among your organization and partners (e.g., funders, executers, etc.)
Mission
Personnel
Time
Money
Training and technical support

17. Comparison Tool for Selecting an EBI
Talking Points:
Here’s an example of a tool for making a detailed comparison that your organization can use.
Summarizing the outcomes of the detailed comparison in the EBI selection tool might provide insight or an overview of which EBI fits best and you can use the overview to base the selection of an EBI on.
Notice that fit isn’t just yes or no.

18. Activity: Comparison Tool
Read the case study and the two EBI program descriptions individually
EBI 1:  Automated Telephone Calls Improve Completion of FOBT
EBI 2:  Flu-FIT/FOBT Program
Complete the Comparison Tool in a small group
Discuss:
Share what fit and didn’t fit for each program
Which program would you choose?
Share decisions and rationale with the large group
*You can change the example to one most relevant to the audience.
(30 minute version with small groups)
Instructions: Ask the participants to:
Take out the Comparison Tool Activity with CRC Programs handout
Read the case community information and the program handout descriptions individually (5-10 minutes)
EBI 1: Automated Telephone Calls Improve Completion of Fecal Occult Blood Testing
EBI 2 : Flu-Fit/FOBT Program
Complete the Comparison Tool in a small group and discuss what fit and didn’t fit for each program, and which program they would choose (10 minutes)
Report out and discussion: Ask participants from each small group to report what fit and didn’t fit for each program, and which program they would choose. (5-10 min)

19. 3. Select One or More EBIs
Select the EBI program that provides the best balance between fit and strength of evidence
Supplementing the EBI program with a strategy to address additional environmental factors
OR select two or more strategies to address multi-levels of factors
Now that you have assessed fit you have the opportunity to select one or more EBIs to implement. You may select an EBI program. If the program does not address all of your objectives, you may want to supplement it with a EBI strategy. On the other hand, you might decide that none of the EBI programs fit and that you will select and combine several strategies.

20. 3. Selecting Multiple EBI Strategies
Community Guide EBIs Determinants Behavior/Environmental Goal
Objectives by Level
Small media
One-on-one education
Reminders
Assessment & feedback
Alternative sites
Address transport barriers
Knowledge
Beliefs/attitude
Confidence
Individual: Gets screened
Lower CRC morbidity & mortality
Provider: Recommends screening
Awareness
Knowledge
Setting: Accessible hours; transport-ation arranged
Talking Points:
This slide shows how multiple Community Guide EBI strategies might be combined to address factors across multiple levels.
Community: GI available to do diagnostic tests

21. Selecting EBIs: Strength of Evidence
How much certainty is there about the effectiveness of the EBI?
Will that effectiveness translate to a real-world setting?
Talking Points:
Two additional considerations are shown here: relevance and strength:
The first is one that can be a barrier to adopting EBI. If an EBI is tested under ideal conditions (volunteers who don’t look like your community, who don’t face the same barriers, who don’t have the same cluster of co-morbidities) then it might not be relevant to your real-world conditions. The new REAIM score, if you look at RTIPS, is an indication of real world effectiveness.
Effectiveness is the extent to which the intended effect or benefits that were achieved under optimal conditions are also achieved in real-world settings.
Efficacy is the extent to which the intended effect or benefits were achieved under optimal conditions.
The second idea here is the strength of the evidence. In the Defining Evidence session, we discussed that systematic reviews like the Community Guide provide stronger evidence that a type of EBI works across multiple studies. The next level would be individual research or rigorous evaluation studies.
Your stakeholders and funders may help you decide what level of evidence you need. There also may not be enough evidence in your priority health area. So you may need to search for the strongest level of evidence available.

22. Research Tested Intervention Programs (RTIPS)
Evidence of Effectiveness:
Single Study vs. Review of Multiple Studies
Research Tested Intervention Programs (RTIPS) + −
Scenario 1
Scenario 3
Scenario 2
Scenario 4
Community Guide
Strategies
Talking Points:
This is one way to look at strength of evidence when considering a single study versus a review of multiple studies.
Scenario 1: Ideally, you can find one or more programs on RTIPS that have corresponding Community Guide (CG) strategies. That means the strategies utilized in the programs have been tested in other multiple studies in different populations/settings. Citing the CG strategy will help you make a strong case for the evidence base in developing grant or program proposals.
Scenario 2: However, some RTIPS programs might not have a corresponding CG strategy on its webpage (the website is being updated constantly though). This usually means that the CG has not done a full review of this specific strategy used in the program OR there is not sufficient evidence available at the time the CG was doing the review. We recommend finding an RTIPS program with corresponding CG strategy. If you can’t find one in the first search, try to find another program in RTIPS that corresponds to a CG strategy.
Scenario 3: If you can’t find any programs on RTIPS that seem to fit, you can turn to the CG website to see the “summary evidence table” and try to identify a program that might fit. You will have to find the article for more details. You might find a program described in an article used in the Community Guide review. If you still can’t find one that seems to fit, consider developing your own program based on an evidence-based strategy. However, this process is beyond the scope of this training.
Scenario 4: In the worst case scenario, you might not find either an RTIPS program or a CG strategy that seem to fit. You could look for strategies or programs that address similar behaviors (e.g., to increase CRC screening in women >50, you could also consider programs or strategies that increase breast and cervical cancer screening in older women). You could also try searching on the Cochrane databases, the ARHQ websites, PubMed or other resources.

23. Real World Effectiveness & RE-AIM
REACH your intended target population
EFFECTIVENSS or efficacy
ADOPTION by target staff, settings, or institutions
IMPLEMENTATION consistency, costs and adaptations made during delivery
MAINTENANCE of strategy effects in individuals and settings over time
Talking Points:
Beginning in 2012, new RTIPs programs are scored on RE-AIM criteria. The acronym stands for Reach, Effectiveness, Adoption, Implementation and Maintenance or RE-AIM.
For maintenance, we’re talking about maintenance on both the individual level and the strategy level.
MAINTENANCE of Health Behavior strategies is the extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies. Maintenance in the RE-AIM framework also has referents at the individual level. At the individual level, maintenance has been defined as the long-term effects of a program on outcomes after 6 or more months after the most recent strategy contact.
Note: See website for more information:

24. Talking Points:
When you are browsing a program on RTIPS, we highly recommend that you scroll down the screen and look for the Community Guide icon (see the slide) and click on the underlined strategy to see more information about the corresponding CG recommended strategy of the program.
If a program has a corresponding CG strategy, it usually means that the type of strategy utilized in the program has been tested in multiple studies in various settings or populations and proven effective.

25. Talking Points:
After you click on the underlined strategy, you will be linked to the CG website. We recommend three things to review: the “summary of task force recommendations & findings”, the “summary of findings” for the specific topic you are searching for, (e.g., CRC screening, physical activity, etc.) and related RTIPS programs (look for the RTIPS icon). This will allow you to see a bigger picture of the evidence-base and a list of other strategies that have been employed to address the same problem/issue.
If you prefer to start searching for strategies on the Community Guide . We recommend that you take a look at the “summary of findings” first to see the breadth of the strategies for the particular topic. When you find a particular strategy that seems to fit, click on the “RTIPS” examples (see the icon in the slide) to see a list of RTIPS programs using that particular strategy.
For Some CG strategies however, like the provider-oriented strategies for increasing cancer screening, there is no direct link to a list of RTIPS programs yet. We recommend that you do a separate search on RTIPS website to identify candidate programs.

26. Take-home Points
Refer to your community assessment when considering fit
Select evidence-based interventions that fit your:
Goals and objectives, including for determinant change
Delivery possibilities and preferences
Characteristics of the priority population
Organization/coalition/community context
Select EBIs with a stronger evidence base
Ask the Audience:
Facilitator may ask participants to reflect on what they have learned in this session before proceeding.
Talking Points:
Use findings from the organizational priorities assess fit.
What have previous community assessments determined to be high need areas?
Use these types of criteria to help you select programs.
Select EBI with a stronger evidence base
Consider not only demographic characteristics of your population, but also social and cultural factors that influence their behaviors or the environment.

27. Questions?
Ask the Audience:
Do you have any questions?

28. References
Bartholomew LK, Markham CM, Ruiter  R, Fernandez ME, Kok G & Parcel G. (2016) Planning Health Promotion Programs: An Intervention Mapping Approach. San Francisco: Jossey-Bass.