Abe Wandersman University of South Carolina Duane House Centers for Disease Control and Prevention Bringing Funders, Researchers/Evaluators, and Practitioners Together to Achieve Outcomes: The Interactive Systems Framework (ISF) for Dissemination and Implementation 2010 AEA-CDC Summer Evaluation Institute

OVERVIEW The need to bring researchers/evaluators, funders and practitioners together to bridge research and practice to make a difference The Interactive Systems Framework for Dissemination and Implementation The ISF in action: The PSBA-GTO project Small Group Exercise

MAKING A DIFFERENCE

THE 2015 TARGET DATE FOR ELIMINATING SUFFERING AND DEATH DUE TO CANCER:

AMBITIOUS GOALS

Dr. von Eschenbach: I believe we are at what I call a strategic inflection in biology, which means we're at a point of unprecedented growth in three key areas related to cancer research: knowledge, technology, and resources. The integration of growth in these three sectors provides an opportunity for exponential progress. To achieve this progress, we must set a clear direction and focus our efforts into a cohesive strategy.

The goal of eliminating suffering and death due to cancer provides this focus. It does not mean "curing" cancer but, rather, it means that we will eliminate many cancers and control the others, so that people can live with -- not die from -- cancer. We can do this by 2015, but we must reach for it. We owe it to cancer patients around the world -- and their families -- to meet this challenge. May 16, 2003 BenchMarks

HEALTHY PEOPLE 2010

Healthy People 2010 Objectives Target: 1.0 new case per 100,000 persons. Baseline: 19.5 cases of AIDS per 100,000 persons aged 13 years and older in Data are estimated; adjusted for delays in reporting. Target setting method: Better than the best. Data source: HIV/AIDS Surveillance System, CDC, NCHSTP.

MAKING A DIFFERENCE

 HOW DO WE GET THERE?  A NEED TO BRIDGE SCIENCE AND PRACTICE

* Why is there a gap between science and practice?

* What is the dominant scientific paradigm for developing research evidence and disseminating it?

*Why is this science model necessary but not sufficient?

* What is the responsibility of the practitioner to deliver evidence-based interventions and what is their capacity to do so?

*What is the responsibility of funders to promote the science of evidence-based interventions and to promote the practice of effective interventions in our communities?

How can evaluation help providers, local CBOS and coalitions, health districts, and state agencies reach results-based accountability ?

A CHALLENGE RESEARCH TO PRACTICE MODELS ARE NECESSARY BUT NOT SUFFICIENT

2. With an emphasis on risk and protective factors, review relevant infor- mation—both from fields outside prevention and from existing preventive intervention research programs 3. Design, conduct, and analyze pilot studies and confirmatory and replication trials of the preventive intervention program 4.Design, conduct, and analyze large- scale trails of the preventive intervention program 5.Facilitate large-scale implementation and ongoing evaluation of the preventive intervention program in the community 1. Identity problem or disorder(s) and review information to determine its extent Feedback Loop FIGURE 1.1 The preventive intervention research cycle. Preventive intervention research is represented in boxes three and four. Notre that although information from many different fields in health research, represented in the first and second boxes, is necessary to the cycle depicted here, it is the review of this information, rather than the original studies, that is considered to be part of the preventive intervention research cycle. Likewise, for the fifth box, it is the facilitation by the investigator of the shift from research project to community service program with ongoing evaluation, rather than the service program itself, that is part of the preventive intervention research cycle. Although only one feedback loop is represented here, the exchange of knowledge among researchers and between researchers and community practitioners occurs throughout the cycle.

Bringing Funders, Researchers/Evaluators, and Practitioners Together to Achieve Outcomes

ROGG Partnership Model Funder Evaluator Grantee Results

io Distilling the Information—Prevention Synthesis & Translation System Supporting the Work—Prevention Support System Putting It Into Practice—Prevention Delivery System Synthesis General Capacity Building Innovation-Specific Capacity Building General Capacity Use Innovation-Specific Capacity Use Macro Policy Climate Funding Existing Research and Theory Translation

Prevention Synthesis and Translation System

Prevention Synthesis & Translation System: Distilling the Process & the Science SynthesisTranslation

What are we trying to ‘translate’?

Prevention support system

GTO Support System Model To Achieve Desired Outcomes Training + QI/QA + Tools + TA + = Current Level of Capacity + GTO Steps: (1) Needs & Resources; (2) Goals & Desired Outcomes; (3) Science-based practices; (4) Fit; (5) Capacity ; (6) Plan; (7) Implementation & Process Evaluation; (8) Outcome evaluation; (9) Continuous Quality Improvement; and (10) Sustainability Actual Outcome s Achieved

Prevention Delivery System

Prevention Delivery System: Local Partners Implementing Prevention— General Capacity Use SBA-specific Capacity Use

Apply the ISF to your work Describe the innovation or practice change you are trying to create

Promoting Science-based Approaches - Getting To Outcomes: Applying the ISF to Teen Pregnancy Prevention

State of Prevention Field Growing number of effective prevention programs However, programs are not being implemented widely…some reasons: -do not fit community values -are not perceived as better than existing -costly to implement -funding has not been available

What is a “science-based” approach? …“science-based” program? Science-based Approach:  Is a systematic process of identifying, selecting, implementing and evaluating a teen pregnancy prevention program  Involves identifying needs/resources, using science- based prevention programs and logic models, evaluating the program (process and outcome) Science-based Programs: Science & Success Emerging Answers

Promoting Science-based Approaches (PSBA) to Teen Pregnancy Prevention: PSBA Program Partners 100+ Local Organizations 9 State Organizations 4 Regional Training Centers 3 National Organizations

PSBA Prevention Synthesis & Translation System: Distilling the Process & the Science PSBA-GTO developed to provide a systematic process for local partners to use a science-based approach in their teen pregnancy prevention work. SynthesisTranslation

What are we trying to ‘translate’?

Program Framework/Process: PSBA using Getting To Outcomes TM #1 Needs/ Resources #2 Goals #3 Best Practices #4 Fit #5 Capacities #6 Plan #7 Implement/ Process Evaluation #8 Outcome Evaluation #9 Improve / CQI #10 Sustain

Apply the ISF to your work Describe the existing or needed efforts to synthesize and translate the “science” or “evidence” related to the innovation (program, practices, or policies) you would like practitioners to adopt:

PSBA Prevention Support System: Supporting the Work of Local Partners— State, regional, & national grantees build their own capacity and provide support to local partners to use PSBA-GTO. General Capacity Building SBA-Specific Capacity Building

CDC (Support System) National Organizations (Support System) Local Organizations (Delivery System) Regional & State Organizations (Support System) PSBA-GTO Specific Capacity Building General Organizational Capacity Building PSBA-GTO Specific Capacity Building PSBA-GTO - Specific Capacity Building PSBA PREVENTION SUPPORT SYSTEM

How do we effectively and efficiently build capacity in the prevention delivery system?

PSBA-GTO Training and Technical Assistance 2007 Trained Grantees on PSBA-GTO Manual Content; developed training slides, examples, materials Hosted PSBA-GTO Roundtable Teleconferences so grantees learn from each other Developed Evaluation Capacity Building (ECB) training & TA; linked to PSBA-GTO 2009 Field Feedback on the PSBA-GTO manual and tools; for a revised resource

How Nationals build the capacity of the Prevention Support System: Training and Technical Assistance

State Grantees Knowledge and Confidence in PSBA- GTO Parts Preliminary Data

Regional Grantees Knowledge and Confidence in PSBA-GTO Parts Preliminary Data

Apply the ISF to your work Describe the support system actors and activities (existing or needed) that ensure practitioners have the necessary innovation-specific capacity to use the innovation in their delivery context.

PSBA Prevention Delivery System: Local Partners Implementing Prevention— Communities build capacity by using PSBA-GTO to plan, implement, and evaluate teen pregnancy prevention efforts. General Capacity Use SBA-specific Capacity Use

What happens when the support system interacts with the delivery system??

Exposure to PSBA-GTO through TA Providers Locals’ Decision to adopt/use PSBA-GTO Process PSBA-GTO Steps implemented and used in practice Increased Use of Science-based Prevention Programs Increased local capacity for planning, implementation & evaluation of SBA to TPP Planning, Implementation, and Evaluation of SBA to TPP PSBA PREVENTION DELIVERY SYSTEM

Total TA and Training Hours by GTO Part (N = 70) TA and Training Hours TA: Total = 1,398, Avg = 20 Training: Total = 1,794, Avg = 26

SBA Specific Capacities: Using GTO

Local Organization Confidence in PSBA-GTO Parts

Apply the ISF to your work Describe the prevention delivery system actors and how does (or will) it look when these practitioners have changed their practice by adopting and using the innovation. Be sure to describe what their innovation-specific capacity should be if they successfully adopt and use the innovation.

PSBA-GTO Evaluation Reflections Accountability Did we build the capacity of states and locals? Did we provide needed supports? Did these lead to our expected outcomes (program implementation)? Development Understanding interaction and feedback in each system Greater need for evaluation capacity in PSS and PDS (ToTAP=E) ISF Environment - Needed support for managing controversy (training developed)

Reflections cont’d Knowledge T&TA are critical to the support system but often undervalued or underfunded. However, T&TA are often not conceived in the full cycle and with clear capacity changes as the bottom line. We need to continue to plan and evaluate using ISF as a guide. ISF depends on interaction of complex systems—how can we use it to design for translation? How can we use it to support sustainability? Can technology help us?

Technology, part of the mix To increase the scale of the capacity building work, we are currently developing a web-based, interactive platform for PSBA-GTO. Prototype of the PSBA-iGTO platform (for local-level users): Real-time training and TA requests Objective builder Data entry and analysis tool Planning tool Web-based; add to in-person T&TA

Your work and the ISF

io Distilling the Information—Prevention Synthesis & Translation System Supporting the Work—Prevention Support System Putting It Into Practice—Prevention Delivery System Synthesis General Capacity Building Innovation-Specific Capacity Building General Capacity Use Innovation-Specific Capacity Use Macro Policy Climate Funding Existing Research and Theory Translation

Abe Wandersman Duane House

Discussion Thinking of an evaluation framework, such as CDC’s, how does the ISF inform evaluation? How does the ISF help us engage stakeholders? What does the ISF tell us about program activities? How can the ISF help focus your evaluation?

References * Wandersman, A. (2003) Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 31, 3/4, Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41, Lesesne et al (2008) Promoting Science Based Approaches to teen pregnancy prevention. American Journal of Community Psychology, Wandersman, A. (2009) Four keys to success (theory, implementation, evaluation, resource/system support): High hopes and challenges in participation. American Journal of Community Psychology. 43 (1/2), 3-21.