1. Dean L. Fixsen, Ph.D. Karen A. Blase, Ph.D. National Implementation Research Network Louis de la Parte Florida Mental Health Institute Implementing Innovations with Benefits to Consumers & Stakeholders SIG/SPDG Directors and Evaluators

2. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 A Little Known Fact An evidence-based program is one thing Implementation of an evidence-based program is a very different thing

3. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 – Lisbeth Schorr, 1993 Implementation Successful programs do not contain the seeds of their own replication.

4. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Effective Programs Evidence-based programs are effective when they are implemented with fidelity Fidelity = the practitioners use all the core intervention components skillfully

5. ODR = Office Discipline Referrals ; SET = Fidelity Measure (80% Score for Criterion) Data provided by Sugai, Horner, Eber, Dunlap: OSEP TA Center on SWPBS PBS: Fidelity and Outcome

6. Proportion of negative UAs PROPORTION OF NEGATIVE URINALYSES BY ADHERENT AND NON-ADHERENT DBT THERAPISTS 86.0 39.0 75.4 37.1 81.1 10.5 80.8 28.9 Linehan, Dimeff et al., 2002

7. Source: “Implementing and Evaluating Evidence-based Intervention Programs for At-Risk Youth”, Thomas L. Sexton, Indiana University, School of Education. Available at: http://www.indiana.edu/~cafs/presentations/SchoolEDRES.ppt

8. 0.28.55.831 MST Therapist Adherence Measure (TAM) Schoenwald, Sheidow, & Chapman, 2006 Multisystemic Therapy (MST) 38% Improvement LowerHigher Therapist Adherence Predicts 2-Year Post Treatment Youth Criminal Activity* (N = 1,979 Families) Information provided by Keller Strother and Sonja Schoenwald, MST Services.

9. Teaching-Family Model 0% 10% 20% 30% 40% 50% Homes 1.0 1.2 1.5 1.7 1.9 2.0 1.1 1.4 1.6 1.8 1.3 Delinquency (Outcomes) Teaching (Fidelity) r s = –.94 Percent Parental-Teaching With Youths Mean Self Reported Delinquency Ratings 123456 Bedlington, et al., 1982

10. Fidelity is Essential (Reduction in Recidivism from.50 Control Group Rate) Source: Adapted from Lipsey, 1997, 2005

11. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Summary Across many evidence-based programs, fidelity matters The best outcomes are associated with high fidelity practitioners High fidelity results from careful attention to implementation of evidence-based programs

12. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Evidence-based programs The usability of a program has little to do with the quality or weight of the evidence regarding that program Evidence about intervention effectiveness for specific populations helps us choose what to implement Evidence about the effectiveness of the intervention does not help implement the program or practice successfully

13. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Evidence-based programs Key aspects of core components Clearly described (what/how) Practical measure of fidelity Fully operationalized (do/say) Field tested (iterative revision) Contextualized (school/systems fit) Good outcomes (worth replicating)

14. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Teachers, Staff, Schools Teachers impact students It is the job of superintendents, principals, and funders to align policies and structures to facilitate effective teacher practices There is no such thing as an “administrative decision” – they are all education decisions

15. “Climate for Implementation”* & Success IMPLEMENTATION EFFECTIVENESS (e.g. fidelity) Measures Top mgmt support Goal Clarity Dedicated resources Access to training & TA Performance monitoring Reinforcement/rewards Removal of obstacles Freedom to express doubts INNOVATION EFFECTIVENESS (e.g., outcomes) +++ NS * (Practice- specific e.g., Klein & Sorra, 1996, 2001; Panzano, Roth, Crane-Ross et al, 2004, Vaidyanathan, 2004)

16. INTEGRATED & COMPENSATORY CONSULTATION & COACHING STAFF PERFORMANCE EVALUATION STAFF PERFORMANCE EVALUATION FACILITATIVE ADMINISTRATIVE SUPPORTS RECRUITMENT AND SELECTION PRESERVICE TRAINING SYSTEMS INTERVENTIONS SYSTEMS INTERVENTIONS Implementation Drivers DECISION SUPPORT DATA SYSTEMS

17. OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5%0%..+Demonstration in Training 30% 20% 0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002

18. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Stages of Implementation Exploration Installation Initial Implementation Full Implementation Innovation Sustainability Implementation occurs in stages: Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 0% 100% Implementation Outcomes Intervention Outcomes

19. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Active Purveyor Role Change the behavior of adult human service professionals “Systems don’t change, people do” (J.W.) Change organizational structures, cultures, and climates Change the thinking of system directors and policy makers Successful and sustainable implementation of evidence-based practices and programs always requires organizational change.

20. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Active Purveyor Role Organization Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Practitioner State, County and Local Context System of care Purveyor Group Simultaneous, Multi-Level Interventions

21. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 Implementation Essentials Help faculty/staff acquire the knowledge, skills, and abilities to effectively provide innovations to students Help schools/districts change to more effectively support the work of the faculty/staff Transform education systems to effectively facilitate and sustain the use of innovations statewide

22. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 For More Information Dean L. Fixsen 813-974-4446 dfixsen@fmhi.usf.edu Karen A. Blase 813-974-4463 kblase@fmhi.usf.edu National Implementation Research Network http://nirn.fmhi.usf.edu

23. Copyright © Dean L. Fixsen and Karen A. Blase, 2006 For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://nirn.fmhi.usf.edu/resources/publications/Mono graph/index.cfmhttp://nirn.fmhi.usf.edu/resources/publications/Mono graph/index.cfm