Barbara Sims Debbie Egan Dean L. Fixsen Karen A. Blase Michelle A. Duda Using Implementation Frameworks to Identify Evidence Based Practices 2011 PBIS National Forum Rosemont IL October 27, 2011

– Lisbeth Schorr, 1993 Implementation Successful programs do not contain the seeds of their own replication.

Implementation Gap RESEARCH PRACTICE GAP Implementation is defined as a specified set of activities designed to put into practice an activity or program of known dimensions. IMPLEMENTATION

Implementation Gap RESEARCH PRACTICE GAP Why Focus on Implementation? IMPLEMENTATION “Students cannot benefit from interventions they do not experience.”

Implementation Science EffectiveNOT Effective Effective NOT Effective IMPLEMENTATION INTERVENTION Actual Benefits (Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999) Inconsistent; Not Sustainable; Poor outcomes Unpredictable or poor outcomes; Poor outcomes; Sometimes harmful from Mark Lipsey’s 2009 Meta- analytic overview of the primary factors that characterize effective juvenile offender interventions – “... in some analyses, the quality with which the intervention is implemented has been as strongly related to recidivism effects as the type of program, so much so that a well-implemented intervention of an inherently less efficacious type can outperform a more efficacious one that is poorly implemented.”

Who’s Accountable for Learning? Student PractitionerSystem

Active Implementation  Letting “It” happen...  Innovation occurs without intervention  Helping “It” happen...  Interested innovators figure it out on their own  Making “It” happen...  Active use of strategies to support the adoption of the innovation  Active installation of supports for the implementation of the innovation Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004

Active Implementation Frameworks  Successful implementation on a useful scale requires...  Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION”  Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS”  Organized, expert assistance – “IMPLEMENTATION TEAMS”  A focus on continuous, purposeful improvement – “IMPROVEMENT PROCESSES”

Implementation Stages Exploration Assess needs Examine innovations Examine Implementation Assess fit Years Installation  Acquire resources  Prepare organization  Prepare implementation  Prepare staff Initial Implementation  Implementation drivers  Manage change  Data systems  Improvement cycles Full Implementation  Implementation drivers  Implementation outcomes  Innovation outcomes  Standard practice

Exploration: The Big Picture What happens during Exploration?  Formalize Team Structures  Determine Need and Identify Options  Assess “Fit” and Feasibility  Promote “Buy in” for the innovation and for implementation supports  Re-Assess

Formalize Team Structures  Who will be accountable on a day-to-day basis for ensuring this work is done?  How will State leadership be a part of this process to ensure that successes are operationalized and barriers are removed?

Determine Need and Identify Options  What do your current data suggest is the most critical or pivotal need?  What is the supporting research or evidence of the strategies you are considering?

Assess Fit and Feasibility  What are the priorities of your State?  What is your theory of change (i.e., logic model, blueprint, outcome map, etc.)?  How will you measure progress toward that goal at the SEA? At the LEA?  Who will do what differently at the SEA to impact that outcome? At the LEA?

Promote Buy-In  How will readiness be created at the SEA?  How will readiness be created at the LEA?

Re-Assess and Decide  What has emerged during Exploration that impacts your decision?

GUIDING IMPLEMENTATION QUESTIONS

EBP: 5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4. HighMediumLow Need Fit Resources Availability Evidence Readiness for Replication Capacity to Implement Total Score: Need in the Educational Setting, Socially Significant Issues, Parent & Community Perceptions of Need, Objective Data indicating Need Need Fit Fit with current - Initiatives RtI Implementation School and District Priorities Organizational structures Community Values Resource Availability Resources Curricula & Classroom Materials, IT requirements, Staffing, Training and PD, Data Systems, Coaching & Supervision, Administrative & system supports needed Evidence Outcomes – Is it worth it? Fidelity data Cost – effectiveness data Number of studies Population similarities Diverse cultural groups Efficacy or Effectiveness Evidence Assessing Evidence-Based Programs and Practices Readiness Qualified purveyor Expert or TA available Mature sites to observe # of replications How well is it operationalized? Are Imp Drivers operationalized? Intervention Readiness for Replication Capacity Staff meet minimum qualifications Able to sustain Implementation Drivers Financially Structurally Buy-in process operationalized Educators Administrators Families Capacity to Implement © National Implementation Research Network- 2009

Implementation Team Management Team Practitioners Children Policy Enabled Practice (PEP) Practice Informed Policy (PIP) System Change Organized, “Expert” Assistance Adaptive Challenges RFP methods Service Silos Salaries Funding Credentialing Licensing Time/ scheduling Union contracts Duplication Fragmentation Hiring criteria Federal/ State laws Practice-Policy Feedback Loops

Implementation Review and synthesis of the implementation research and evaluation literature (1970 – 2004)  Multi-disciplinary  Multi-sector  Multi-national