Barbara Sims Brenda Melcher Dean Fixsen Karen Blase Michelle Duda Washington, D.C. July 2013 Keep Dancing After the Music Stops OSEP Project Directors’ Conference

WHAT: Effective Interventions HOW, WHO, WHEN: Effective Implementation WHERE: Enabling Contexts WHY: Socially Significant Outcomes Formula For Success

WHO Teams WHEN Stages HOW Drivers HOW Cycles WHAT Usable Interventions Active Implementation Frameworks

USABLE INTERVENTIONS An intervention needs to be teachable, learnable, doable, and be readily assessed in practice. Usable Interventions

Operational Definitions Clear Description Performance Assessment Essential Functions

Usable Interventions Tools You Can Use Hexagon Tool Practice Profiles Usable Interventions

The Hexagon An EBP Exploration Tool NEED FIT RESOURCES EVIDENCE CAPACITY READINESS Fit with current Initiatives School, district, state priorities Organizational structures Community values Need in school, district, state Academic & socially significant Issues Parent & community perceptions of need Data indicating need Resources and supports for: Curricula & Classroom Technology supports (IT dept.) Staffing Training Data Systems Coaching & Supervision Administration & system Evidence Outcomes – Is it worth it? Fidelity data Cost – effectiveness data Number of studies Population similarities Diverse cultural groups Efficacy or Effectiveness Capacity to Implement Staff meet minimum qualifications Able to sustain Imp Drivers Financially Structurally Buy-in process operationalized Practitioners Families Readiness for Replication Qualified purveyor Expert or TA available Mature sites to observe Several replications How well is it operationalized? Are Imp Drivers operationalized? The “Hexagon” can be used as a planning tool to evaluate evidence- based programs and practices during the Exploration Stage of Implementation. Download available at: s/hexagon-tool-exploring-context EBP: 5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4. HighMedLow Need Fit Resource Availability Evidence Readiness for Replication Capacity to Implement Total Score © 2009 Karen Blase, Laurel Kiser, & Melissa Van Dyke Adapted from work by Laurel J. Kiser, Michelle Zabel, Albert A. Zachik, and Joan Smith at the University of Maryland

How: Effective Implementation Changing the behavior of educators and administrators Creating the setting conditions to facilitate these changes Creating the processes to maintain and improve these changes in both setting conditions and behavior of well-intentioned adults So that students benefit

IMPLEMENTATION DRIVERS Key components of capacity and infrastructure that influence a program’s success. Drivers

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Competency Drivers Organization Drivers Implementation Drivers © Fixsen & Blase, 2008 Leadership Drivers

CharacteristicPracticeMean Effect Size IntroductionOut-of-class learner activities/self-instruction Classroom/workshop presentations Pre-class learner exercises IllustrationTrainer role playing/simulations Learner informed input PracticingReal-life learner application Real-life learner application/role-playing EvaluationSelf assessment of strengths/weaknesses0.94 ReflectionIdentify performance-improvement goals Journaling/behavior suggestions MasteryStandards-based assessment0.86 Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), Research Synthesis of Adult Learning Studies

Training and Coaching Implementation Drivers OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10%5%0%..+Demonstration in Training 30%20%0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% —Joyce and Showers, 2002

Intervention Training Develop Intervention Training Plan  Identify initial trainers »Qualifications for trainers  If outside experts are used you need to assess »Innovation Fluency »Use of Training Best Practices »Data from people they have ‘trained’

Intervention Training Plan Aspects requiring new knowledge  Description  Purpose  Supporting data  Examples  Critical components

Intervention Training Plan Aspects requiring new skills  Key behaviors  Behavior rehearsals  Identify and prepare behavior rehearsal leaders and confederates

Intervention Training Plan Establish Coaching System  Selection Criteria  Policy/Procedures  Accountability structure  Feedback loops

Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Competency Drivers Organization Drivers Leadership Improved outcomes Interventions meet Implementation Consistent Use of Programs © Fixsen & Blase, 2008 Performance Assessment (Fidelity)

Implementation Drivers Tools You Can Use Training Plan Template Coaching System Worksheet Drivers

Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Competency Drivers Organization Drivers Implementation Drivers © Fixsen & Blase, 2008 Leadership Drivers

Practice-Policy Communication Cycle Policy Practice Policy Enables Practices Plan Do External Implementation Support Policy Practice Structure Procedure Practice Informs Policy Feedback Study - Act FORM SUPPORTS FUNCTION

Children & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

Communication Cycles Tools You Can Use Communication Protocol Worksheet Drivers

Free 24/7 Implementation Training for You and Your Team SISEP’s newly launched “Active Implementation Hub” is a free, online learning environment for use by any stakeholder — practitioners, educators, coaches, trainers, purveyors — involved in active implementation and scaling up of programs and innovations. The site goal is to increase the knowledge and improve the performance of persons engaged in actively implementing any program or practice.” The AI Hub is an initiative of the State Implementation & Scaling-up of Evidence-based Practices Center (SISEP) and The National Implementation Research Network (NIRN) located at The University of North Carolina at Chapel Hill’s FPG Child Development Institute.

Get Connected! For more on Implementation Science