Barbara Sims Dean L. Fixsen Karen A. Blase Michelle A. Duda Implementation for a Change RtI Best Practices Institute Barbara Sims Dean L. Fixsen Karen A. Blase Michelle A. Duda Wrightsville Beach, NC October 2012 © 2012 Karen A. Blase and Dean L. Fixsen

Complex Problems Human services involve interaction-based sciences Inherently more complex than atom-based sciences E.g., atom-based ingredients don’t refuse to be ingested, talk back, or run away

Ineffective Approaches Best data show these methods, when used alone, Do Not result in uses of innovations as intended: Diffusion/ Dissemination of information Training Passing laws/ mandates/ regulations Providing funding/ incentives Organization change/ reorganization 5 to 10% return on investment NECESSARY BUT NOT SUFFICIENT Paul Nutt (2002). Why Decisions Fail

EXISTING SYSTEM EFFECTIVE INNOVATIONS ARE CHANGED TO FIT THE SYSTEM System Change EXISTING SYSTEM EFFECTIVE INNOVATIONS ARE CHANGED TO FIT THE SYSTEM EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION EFFECTIVE INNOVATION

Why Focus on Implementation? Implementation Gap Implementation is defined as a specified set of activities designed to put into practice an activity or program of known dimensions. RESEARCH IMPLEMENTATION PRACTICE GAP as a nation we have invested significantly in the research base in what works, finding interventions that might make a difference. And we have developed some very nice interventions in education that have been shown, particularly in highly-supported research settings, that they can be very effective.

But— when we check in to see to what extent those interventions are occurring in practice, it's not looking as good. "What is that that is preventing us from really benefiting from the research?" And we're going to say, "Well, there probably are many things, but implementation is significant."

A set of mechanisms, a set of strategies that really offer us a guide to bring from research into practice what it takes to do good work we think could be very helpful. Why Focus on Implementation? “Students cannot benefit from interventions they do not experience.”

Developing the Capacity to Implement Well “A serious deficiency is the lack of expertise to implement best practices and innovations effectively and efficiently to improve student outcomes.” Rhim, Kowal, Hassel, & Hassel (2007) Rhim, L. M., Kowal, J. M., Hassel, B. C., & Hassel, E. A. (2007). School turnarounds: A review of the cross-sector evidence on dramatic organizational improvement. Lincoln, IL: Public Impact, Academic Development Institute.

Implementation Science Implementation science is the scientific study of variables and conditions that impact changes at practice, organization, and systems levels; changes that are required to promote the systematic uptake, sustainability and effective use of evidence-based programs and practices in typical service and social settings. ~Blase and Fixsen, 2010 National Implementation Research Network

Implementation Science Effective NOT Effective from Mark Lipsey’s 2009 Meta-analytic overview of the primary factors that characterize effective juvenile offender interventions – “. . . in some analyses, the quality with which the intervention is implemented has been as strongly related to recidivism effects as the type of program, so much so that a well-implemented intervention of an inherently less efficacious type can outperform a more efficacious one that is poorly implemented.” Actual Benefits Inconsistent; Not Sustainable; Poor outcomes Effective INTERVENTION Unpredictable or poor outcomes; Poor outcomes; Sometimes harmful NOT Effective (Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education, 1983; Department of Health and Human Services, 1999)

Formula for Success = X Effective Intervention practices Effective Implementation practices Improved Outcomes X = 1.0 X 0.0 = 9

Plan for Change District is the point of entry for systemic support of school improvement Use short-term infusion of resources Establish long-term, district-based capacity for quality

Shifting Accountability Student Practitioner System SISEP 2012

Cascading Logic Model Improve student outcomes Improve teacher instruction Improve school supports for teachers Improve district supports for schools Improve regional supports for districts Improve State supports for outcomes Re-define relationships among system components Focus fully on student outcomes © 2012 Karen A. Blase and Dean L. Fixsen

Active Implementation Frameworks Implementation Drivers Implementation Stages Implementation Teams Improvement Cycles Briefly Summarize the Four Implementation Frameworks Set the Context for Tools and Resources State and RRC Examples Table Discussion © 2012 Karen A. Blase and Dean L. Fixsen

Active Implementation Frameworks Successful implementation on a useful scale requires. . . Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS” Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION” Organized, expert assistance – “IMPLEMENTATION TEAMS” A focus on continuous, purposeful improvement – “IMPROVEMENT CYCLES” (c) Dean Fixsen and Karen Blase, 2008 14

IMPLEMENTATION DRIVERS Common features of successful supports to help make full and effective use of a wide variety of innovations

Improved educational outcomes Consistent Use of Educational Innovations Interventions meet Implementation Performance Assessment (Fidelity) Coaching Systems Intervention Facilitative Administration Competency Drivers Training Organization Drivers Integrated & Compensatory Decision Support Data System Selection Leadership Adaptive Technical © Fixsen & Blase, 2008 16

Performance Assessment (Fidelity) Coaching Competency Drivers Training Selection Technical © Fixsen & Blase, 2008 © Fixsen & Blase, 2008 17

Build Competency and Confidence Competency Drivers Build Competency and Confidence Develop, improve, and sustain competent & confident use of innovations

Performance Assessment Measure fidelity Ensure implementation Reinforce staff and build on strengths Feedback to agency on functioning of Recruitment and Selection Practices Training Programs (pre and in-service) Supervision and Coaching Systems Interpretation of Outcome Data (c) Dean Fixsen and Karen Blase, 2004

Selection Select for the “unteachables” Screen for pre-requisites Set expectations Allow for mutual selection Improve likelihood of retention after “investment” Improve likelihood that training, coaching, and supervision will result in implementation (c) Dean Fixsen and Karen Blase, 2004

Training Develop Training Plan Define critical components Aspects requiring new knowledge Aspects requiring new skills Prioritize training topics Identify or develop fidelity measures (c) Dean Fixsen and Karen Blase, 2004

Coaching Develop Coaching Plan Ensures fidelity Ensures implementation Provides feedback to selection and training processes Grounded in “Best Practices” (c) Dean Fixsen and Karen Blase, 2004

Training and Coaching OUTCOMES 10% 5% 0% 30% 20% 60% 95% Knowledge % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% …+ Practice & Feedback in Training 60% …+ Coaching in Classroom 95% Joyce and Showers, 2002 (c) Dean Fixsen and Karen Blase, 2004

Performance Assessment (Fidelity) Systems Intervention Coaching Training Facilitative Administration Competency Drivers Organization Drivers Decision Support Data System Selection Leadership Adaptive Technical © Fixsen & Blase, 2008 24

Reflection Implementation Drivers How do we support the development of the infrastructure needed to implement well? How do we promote more hospitable organizational environments? How relevant are leadership issues? What’s our role? Supporting New Ways of Work 25

Active Implementation Frameworks Successful implementation on a useful scale requires. . . Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS” Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION” Organized, expert assistance – “IMPLEMENTATION TEAMS” A focus on continuous, purposeful improvement – “IMPROVEMENT CYCLES” (c) Dean Fixsen and Karen Blase, 2008 26

STAGES OF IMPLEMENTATION Purposeful matching of critical implementation activities to the stage of the process SISEP 2012

Stages of Implementation INITIAL IMPLEMENTATION FULL IMPLEMENTATION Integrated & Compensatory Competency Drivers Organization Drivers Leadership 2 - 4 Years INSTALLATION EXPLORATION Stages of Implementation Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

Exploration Formalize Team Structures Develop Communication Plan Determine Need and Identify Options Assess “Fit” and Feasibility Promote “Buy in” for the innovation and for implementation supports “Pay now or pay later.” What happens during Exploration? Formalize Team Structures Develop Communication Plan Determine Need and Identify Options Assess “Fit” and Feasibility Promote “Buy in” for the innovation and for implementation supports Make recommendations

Installation Structural and functional changes are made First implementers selected Define and initiate training of first implementers Develop coaching plans Evaluate readiness and sustainability of data systems Establish communication links and protocols Structural and functional changes are made Selection protocols developed First implementers selected Define and initiate training of first implementers Develop coaching plans Evaluate readiness and sustainability of data systems Analyze and problem-solve sustainability Establish communication links and protocols

Initial Implementation Initiate training plan Provide coaching Make use of improvement cycles Communication links and protocols PDSA to resolve systems issues Usability testing of selection, training and coaching Get started, then get better! Learn from mistakes Celebrate progress Continue “buy-in” efforts Make systemic changes Manage expectations All the components of the program or innovation are in place and the implementation supports begin to function.

Full Implementation “What Change?....This is our way of work!” Skillful practices by all staff Evaluation for expected outcomes Full use of Implementation Drivers Policy changes/development for sustainability “The only thing worse than failing and not knowing why you failed, is succeeding and not knowing why you succeeded.” ~ Jane Timmons-Mitchell “What Change?....This is our way of work!” Skillful Teaching and School Practices Skillful Use of the Drivers Drivers experience their own Improvement Cycles Data Systems in use, reliable, efficient, and used for Decision-Making at multiple levels to regenerate and improve Policy to Practice and Practice to Policy Feedback Cycles

ACTIVITY Stages of Implementation Analysis What are you already doing that is “stage-based”? What are the facilitators and barriers to doing stage-based work? Discuss the Exploration and Installation Stage indicators. Supporting New Ways of Work 33

Active Implementation Frameworks Successful implementation on a useful scale requires. . . Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS” Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION” Organized, expert assistance – “IMPLEMENTATION TEAMS” A focus on continuous, purposeful improvement – “IMPROVEMENT CYCLES” (c) Dean Fixsen and Karen Blase, 2008 34

IMPLEMENTATION TEAMS Organized, expert assistance to develop and sustain an accountable structure

Implementation Team 80%, 3 Yrs 14%, 17 Yrs IMPLEMENTATION Impl. Team NO Impl. Team 80%, 3 Yrs 14%, 17 Yrs Effective INTERVENTION Balas & Boren, 2000 Making it Happen Letting it Happen Helping it Happen Fixsen, D. L., Blase, K. A., Timbers, G. D., & Wolf, M. M. (2001). In search of program implementation: 792 replications of the Teaching-Family Model. In G. A. Bernfeld, D. P. Farrington & A. W. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp. 149-166). London: Wiley. It takes an estimated average of 17 years for only 14% of new scientific discoveries to enter day-to-day clinical practice (Balas & Boren, 2000) Balas EA, Boren SA. Yearbook of Medical Informatics: Managing Clinical Knowledge for Health Care Improvement. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH; 2000. Green, L. A., & Seifert, C. M. (2005). Translation of research into practice: Why we can't "Just do it". Journal of the American Board of Family Practitioners, 18(6), 541-545. With the use of competent Implementation Teams, over 80% of the implementation sites were sustained for 6 years or more (up from 30%) and the time for them to achieve Certification was reduced to 3.6 years. Fixsen, Blase, Timbers, & Wolf, 2001 Balas & Boren, 2000 Green & Seifert, 2005

Linked Team Structures School-based Implementation Team District-based Regionally-based State-based Think again about the NAEP scores. They’ve stayed flat for 40 years. How is that possible? I think this quote from Senge speaks to a piece of it. often times we focus at a particular place in the system as if that one place could then emanate its changes out kind of organically, or through some kind of mechanism that we've yet to describe.

We would suggest that there probably needs to be a real formalized structure that ensures that we have implementation capacity at every level. “We tend to focus on snapshots of isolated parts of the system and wonder why our deepest problems never seem to get solved. (Senge, 1990)

ACTIVITY Table Talk: Implementation Teams In your experience, who supports the change process? How is the transition made from external expertise to building internal capacity? Supporting New Ways of Work 38

Active Implementation Frameworks Successful implementation on a useful scale requires. . . Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS” Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION” Organized, expert assistance – “IMPLEMENTATION TEAMS” A focus on continuous, purposeful improvement – “IMPROVEMENT CYCLES” (c) Dean Fixsen and Karen Blase, 2008 39

IMPROVEMENT CYCLES Changing on purpose to support the new way of work SISEP 2012

Types of Improvement Cycles Plan-Do-Study-Act Cycles Usability testing (Neilson; Rubin) Practice-policy communication loops Deming, W. E. (1986). Out of the crisis. Cambridge, MA: MIT Press. Shewhart, W. A. (1931). Economic control of quality of manufactured product. New York: D. Van Nostrand Co. Shewhart, W. A. (1939). Statistical method from the viewpoint of quality control: Dover Publications. Nielsen, J. (2000). Why you only need to test with 5 users. Retrieved April 22, 2007, from http://www.useit.com/alertbox/20000319.html. Rubin, J. (1994). Handbook of usability testing: How to plan, design, and conduct effective tests. New York: John Wiley & Sons. Fixsen, D., Blase, K., Metz, A., & Dyke, M. V. (in press). Statewide implementation of evidence-based programs. Exceptional Children (Special Issue). 41

PDSA

Usability Testing

Policy Practice Feedback Loops Expert Implementation Support Policy (Plan) Policy Practice Policy Enabled Practices (PEP) Practice Informed Policy (PIP) Structure Feedback Study - Act Procedure So again, we've got the organized, expert assistant out on the side, trying to connect what we're learning at the practice level to the policy level, so that we really do have structures and procedures that are aligned to support new and effective practice, with the goal being, over time, an aligned system.

A system where our effective practices are really quite easily implemented by our practitioners, who are supported by agencies, who are supported by their regions, who are supported by their state department, who are supported by their national leadership, so that each level receives what it needs to effectively implement, really, the best services. The services we would want anyone to receive. Practice (Do) FORM SUPPORTS FUNCTION (c) Dean Fixsen and Karen Blase, 2008 44

ACTIVITY Table Talk: Improvement Cycles and Communication Loops How can we make use of improvement cycles in developing and implementing our improvement activities? Linking Communication Protocols Supporting New Ways of Work 45

Summary Conceptualize a change process so that effective interventions for children and families can become embedded and sustained in socially complex settings “stage-matched activities to guide the process “implementation drivers” to build the infrastructure Improvement processes are critical the work is never done because the environment is in motion Invest in the development of organized, “expert” implementation support (c) Dean Fixsen and Karen Blase, 2010

For more on Implementation Science Stay Connected! www.scalingup.org @SISEPcenter SISEP For more on Implementation Science http://nirn.fpg.unc.edu www.implementationconference.org 47