Implementation Science overview Destination Transformation Nash-Rocky Mount Public Schools Heather Reynolds June 17, 2014

Context & Rationale

Activity Successful Implementation Unsuccessful Implementation ImpleMap Questions How did your LEA decide to use the intervention? Is there a process for considering needs, matching intervention to need, considering fit with current practices? How does the process work and how standardized has it become? What defines the intervention? What are the critical elements of this intervention? What do you look for when someone is using this intervention? How are practitioners supported in their work? How are staff selected? What training is provided? What coaching is provided? What data are collected and used to support implementation? How do leaders work to remove barriers and create a hospitable environment? Who provides support for practitioners? Is the intervention used by all, or just some teachers? How were they selected? By whom? What are the positions of those providing support? How are their roles and responsibilities defined?

Why Focus on Implementation? Implementation Gap Implementation is defined as a specified set of activities designed to put into practice an activity or program of known dimensions. RESEARCH IMPLEMENTATION PRACTICE GAP Why Focus on Implementation? “Students cannot benefit from interventions they do not experience.”

Business as Usual: Impact Do not Result in Implementation as Intended (used alone) Diffusion/ Dissemination of information Training Passing laws/ mandates/ regulations Providing funding/ incentives Organization change/ reorganization 5 to 10% return on investment NECESSARY BUT NOT SUFFICIENT Paul Nutt (2002). Why Decisions Fail

 OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% …+ Practice & Feedback in Training 60% …+ Coaching in Classroom 95% —Joyce and Showers, 2002

Formula for Success Implementation Math WHAT: Effective Interventions HOW: Effective Implementation WHERE: Supportive Contexts WHY: Positive Outcomes for Students

Planning for Change Point of entry is District, not school Use short-term infusion of resources Establish long-term, district-based capacity for quality Must focus on the program/practice/initiative/“IT” and the framework for installation

Implementation Frameworks

Active Implementation Frameworks Stages Drivers Usable Interventions Cycles Teams

Usable Interventions

Performance Assessment Usable Interventions Performance Assessment Operational Definitions Essential Functions Clear Description

Usable Interventions Philosophy, Values and Principles Inclusion and Exclusion Criteria Clear Description

Usable Interventions Clear description of the features that must be present to say that a program exists in a given location Core components Essential Functions

Operational Definitions Describe each core component in terms that can be taught, learned, done in practice, and assessed in practice Practice Profiles Operational Definitions

Performance Assessment Provides evidence that the program is being used as intended and is resulting in the desired outcomes Fidelity Practical enough to repeat time and time again Performance Assessment

Reflection Usable Interventions How will knowledge of Usable Interventions inform your future practice? Usable Interventions

Stages

Implementation Stages Initial Implementation Exploration Installation Initial Implementation Full Implementation Assess needs Examine intervention components Consider Implementation Drivers Assess Fit Acquire Resources Prepare Organization Prepare Implementation Drivers Prepare Staff Adjust Implementation Drivers Manage Change Deploy Data Systems Initiate Improvement Cycles Monitor & Manage Implementation Drivers Achieve Fidelity & Outcome Benchmarks Further Improve Fidelity & Outcomes 2-4 Years

Stages & Implementation Buy-in Based on assessment of needs Fit and feasibility Clear understanding of the core components of the innovation Exploration The goal of the Exploration Stage is to examine the degree to which a particular program or innovation meets the school or district’s needs and whether implementation is feasible. In this stage, schools and districts must assess the goodness of fit between potential programs and innovations, and the needs of the students they serve. For example a school or district would assess the fit between and potential evidence-based practices and the academic, behavioral, and health needs of the students they serve. Requirements for implementation must be carefully assessed and potential barriers to implementation examined. Key activities during exploration include getting key stakeholders involved, forming an implementation team, and identifying potential programs. It’s also important to ensure that core intervention components of potential programs are identified and well operationalized. Even with existing evidence-based and evidence-informed practices, more program development work might need to be done before implementation decisions to move forward can be made. © 2012 Karen A. Blase and Dean L. Fixsen

Stages & Implementation Developing communication pathways Ensuring adequate allocation of time and resources Developing team competency Develop staff competency Training, coaching, and data systems are developed and utilized Installation The Installation Stage begins as the decision is made to move ahead. The Installation Stage is often overlooked in implementation. Practical preparations needed to initiate the new program or innovation are central to the Installation Stage. Once a decision is made to adopt a program or innovation, changes often must be made in multiple settings and systems to accommodate and fully support the new practice, program or innovation. These can include practical activities such as: developing communication protocols, ensuring financial and human resources are in place, finding or reallocating physical space, and/or purchasing equipment and technology. In addition, developing the knowledge, skills and abilities of teachers and administrators is a key function of the Installation Stage. This means that training, coaching, and data systems are conceptualized, created, or purchased. And that those expected to implement the new program or practices receive the training and support needed. Well prepared teachers are more likely to feel confident and to be able to implement new programs and practices with fidelity. © 2012 Karen A. Blase and Dean L. Fixsen

Stages & Implementation Provision for coaching is critical Continuous problem-solving and modification based on data/needs Regular evaluation of implementation of core concepts Initial Implementation The Initial implementation Stage begins when the new program or practice is first being put to use. Attempts to implement a new program or innovation often falter (or end) during installation or initial implementation. This is because everyone is learning and challenges emerge as the status quo is changed. Key activities during this stage include intensive coaching to help practitioners through this awkward stage. And as problems emerge the team develops and engages in strategies to promote continuous improvement and rapid- cycle problem solving. In addition, data are used to assess the quality of implementation, identify problems and solutions, and inform decision making. It is critical to address barriers and develop systemic solutions quickly rather than allowing problems to re-emerge and reoccur. The processes for doing so are discussed later in the Improvement Cycles section. © 2012 Karen A. Blase and Dean L. Fixsen

Stages & Implementation Infrastructure to support teachers Staff skillfully implements practices Data is continuously used to guide implementation and improvement Implementation has been integrated at all levels Full Implementation Full implementation occurs as teachers skillfully provide new programs and outcomes are achieved. New learning at all levels becomes integrated into classroom, building and district settings. In full implementation, the processes and procedures to support the new way of work are in place. The system has largely been recalibrated to accommodate and support the new ways of work. The time it takes to move from initial implementation to full implementation will vary depending upon the complexity of the new program or innovation as well as the development of the infrastructure to support teachers and the availability of implementation supports and resources. © 2012 Karen A. Blase and Dean L. Fixsen

Reflection How will knowledge of Stages inform your future practice? Stages

Drivers

Implementation Drivers Performance Assessment (Fidelity) Coaching Systems Intervention Training Organization Drivers Facilitative Administration Competency Drivers Selection Decision Support Data System Leadership Adaptive Technical © Fixsen & Blase, 2008 26

Implementation Drivers Help to develop, improve, and sustain educators’ competence and confidence to implement effective educational practices and supports. Help ensure sustainability and improvement at the organization and systems level Help guide leaders to use the right leadership strategies for the situation Drivers

Positive Outcomes for Students Why: What: Effective Educational Practices Core Implementation Components Staff capacity to support children/families with the selected practices Institutional capacity to support teachers & staff in implementing practices with fidelity Competency Drivers Organization Drivers How: Leadership Capacity to provide direction and vision © Fixsen & Blase, 2008

Competency Drivers Build competency and confidence Develop, improve and sustain competent & confident use of innovations Competency Drivers Competency drivers are activities to develop, improve, and sustain teachers’ and administrators’ ability to 1) put programs and innovations into practice, to 2) benefit the students. The four competency drivers include: selection, training, coaching and performance assessment. Collectively they can effectively provide professional development that makes a difference for both teachers and students. © 2012 Karen A. Blase and Dean L. Fixsen

Implementation Drivers: Competency Performance Assessment (Fidelity) Coaching Competency Drivers Training Selection—Effective staffing requires the specification of required knowledge, skills and abilities that relate to program-specific needs. This means specifying skills and abilities that are pre-requisites for the work ahead and determining those that will be developed once the person is hired. The criteria initially are used to select candidates among those already employed in a school or district who will be among the first to implement the innovation. Subsequently, each new hiring opportunity is an opportunity to select with implementation in mind. Once requirements have been identified, schools and districts must identify methods for recruiting candidates who possess these skills and abilities, as well protocols for interviewing and criteria for selecting teachers, administrators and even program or practice leads. Training — Teachers, administrators and staff need to learn when, how, and with whom to use new skills and practices. Training should: provide information related to the theory and underlying values of the program or innovation, use training processes grounded in adult learning theory to actively engage participants, introduce the components of and rationales for key practices, and provide opportunities to practice and re-practice new skills and receive feedback in a safe and supportive training environment. Coaching — Most new skills can be introduced in training, but must be practiced and mastered on the job. Coaching is the key. Districts and schools should: actively develop and implement coaching service delivery plans that detail how often, where, when, with whom, and why coaching will occur, use multiple sources of data to provide feedback to practitioners and always include direct observation, and, use coaching data and information from coaches to inform training improvements and improve organizational supports Performance (Fidelity) Assessment — Implementing the evidence-based program or innovation as intended is the outcome of selection, training and coaching. This means that the entire organization is accountable for instructional or program quality. Teachers are not in it alone. Districts and schools should develop and implement transparent fidelity assessments, use multiple sources of data to assess fidelity, institute positive recognition so assessments are seen as an opportunity to improve fidelity and use fidelity assessment data to improve practice fidelity, organizational and system supports. Selection © Fixsen & Blase, 2008 30

Organization Drivers Change Organizations and Systems Create and sustain hospitable organizational and system environments for effective services Develop functional data systems that can be used to inform decision-making Organization Drivers Now let's turn to the Organization Drivers. Organization Drivers are used to intentionally develop the supports and infrastructures needed to create a hospitable environment for new programs and innovations. These supports may need to be developed across the building and district levels. Let’s briefly touch on each component. © 2012 Karen A. Blase and Dean L. Fixsen

Implementation Drivers: Organization Performance Assessment (Fidelity) Coaching Systems Intervention Training Organization Drivers Facilitative Administration Competency Drivers Decision-Support Data Systems —Better decisions are made when data are available to inform the decision-making process. A functional decision-support data system includes: quality assurance data, fidelity data and outcome data. Data need to be reliable, reported frequently, built into every day routines, accessible at the classroom and building levels and used to make decisions at the student, teacher, and building level. Facilitative Administration — Administrators drive decision making. They make use of a wide range of data to inform decision making to support the overall implementation processes as well as keep staff organized and focused on achieving the desired outcomes. Administrators need to be committed to facilitating the development of an organization that is committed to the new program. This means that administrators and their teams proactively look for ways to: identify and effectively address challenges, develop clear communication protocols and functional feedback loops, adjust and develop policies, procedures, and guidelines to support the new way of work, and reduce administrative barriers to using the program as intended. Systems Interventions — System Interventions are strategies to work with external systems or levels of the education system that are not under the direct control of the administrators. Systemic issues and barriers that need to be addressed at a level above the organization need to be identified, communicated to relevant system partners, and resolved. For example, there are issues that cannot be resolved at the building level and must be addressed by the district. Similarly there are issues that the district cannot address without the support and engagement of the state department of education. Resources, regulations, and systems supports at each level need to be aligned to support implementation. Selection Decision Support Data System © Fixsen & Blase, 2008 32

Leadership Drivers Purpose Identifying “wicked” problems and applying effective strategies to address those problems Leadership Drivers Leadership is foundational to the work of implementation. Volumes have been written about effective leadership and there is strong agreement about the importance of knowledgeable and engaged leadership. Within this Active Implementation Framework we will be focused on the role rather than the authority position of a leader. And on technical and adaptive leadership strategies because there is some data to indicate that the ability to engage in such leadership impacts student achievement. This does not mean that many other aspects of leadership are not important. © 2012 Karen A. Blase and Dean L. Fixsen

Implementation Drivers: Leadership Performance Assessment (Fidelity) Coaching Systems Intervention Training Organization Drivers Facilitative Administration Competency Drivers Selection Decision Support Data System Leadership Adaptive Technical © Fixsen & Blase, 2008 34

Reflection How will knowledge of Drivers inform your future practice? Drivers

Cycles

Improvement Cycles New approaches need New Ways of Work Transparent, protocol-driven feedback loops and processes Aligned policies, funding, guidance to support new ways of work There are no administrative decisions, they are all education quality decisions. Cycles

Types of Improvement Cycles Plan-Do-Study-Act Cycles Rapid cycle problem solving Usability testing Practice-policy communication cycles While there are many methods of improvement, two that can promote purposeful building and district level implementation are the Plan-Do-Study-Act Cycles (PDSA) and Policy-Practice Communication Cycles. 38

Communication & Policy Analysis Facilitative System Level State Region District School Classroom Hindering Adapted from NIRN Improvement Cycle Analysis Worksheet © 2012 Karen A. Blase and Dean L. Fixsen

Improvement Cycles: Usability Testing Plan Study Act Do 41

Reflection How will knowledge of Cycles inform your future practice? Cycles

Teams

Implementation Teams Implementation Teams ARE NOT Committees Advisory Groups Implementation Teams DO Know the innovation – the WHAT Know and use implementation science and best practices – the HOW Use Plan, Do, Study, Act Cycles to “get started” and “get better” Stay on “mission” Engage in the right work at the right level at the right time.

Linked Team Structures “We tend to focus on snapshots of isolated parts of the system and wonder why our deepest problems never seem to get solved.” —Senge, 1990 School-based Implementation Team District-based Regionally-based State-based Decision-makers: Data, Curriculum, Funding, Personnel

Implementation Teams 80%, 3 Yrs 14%, 17 Yrs IMPLEMENTATION Impl. Team NO Impl. Team 80%, 3 Yrs 14%, 17 Yrs Effective Balas & Boren, 2000 INTERVENTION Making it Happen Letting it Happen Helping it Happen Fixsen, Blase, Timbers, & Wolf, 2001 Balas & Boren, 2000 Green & Seifert, 2005

Implementation Teams Focus on: Increasing “buy-in” and readiness, Installing and sustaining the implementation infrastructure, Assessing and reporting on fidelity and outcomes, Building linkages with external systems, and Problem-solving and promoting sustainability. Too often innovations rely on just a champion or two. And those champions can move on to new challenges or burn out. So innovations come and go with individuals. An advantage of relying on implementation teams is that the team collectively has the knowledge, skills, abilities, and time to succeed and sustain the work. The team embodies the capacity needed to implement well and maintain and improve programs and practices over time and across staff. © 2012 Karen A. Blase and Dean L. Fixsen

Implementation Teams Core Competencies: Knowledge and understanding of the selected program or innovation including the linkage of components to outcomes. Knowledge of implementation science and best practices for implementation, and Applied experience in using data for program improvement. © 2012 Karen A. Blase and Dean L. Fixsen

Practice-Policy Communication Cycle External Implementation Support Practice Informs Policy Policy Enables Practices Plan Policy Practice Structure Procedure Feedback Study - Act Do Practice FORM SUPPORTS FUNCTION

Reflection How will knowledge of Cycles inform your future practice? Teams

Tools & Resources

Getting Started: Tools & Resources Hexagon Tool Initiative Inventory The Active Implementation Hub Quick Start Video

Hexagon Tool

Initiative Inventory

Activity Think about a program, practice, or initiative that you will be implementing in your classroom or your school will be working together to implement. Try out either the Hexagon Tool or the Initiative Inventory as you begin thinking about how our discussion today will shape your implementation work.

State Implementation work

Professional Development Technical Assistance Accountability K-3 Literacy Curriculum & Instruction Office of Early Learning Educator Effectiveness District & School Transformation Federal Programs Evaluation & Research Exceptional Children Communication & Visibility

Implementation Quick Start What does this work mean? https://unc-fpg-cdi.adobeconnect.com/_a992899727/ai-lesson-quickstart/

Heather Reynolds State Implementation Specialist & Data Analyst NC Department of Public Instruction heather.reynolds@dpi.nc.gov NCImplementationScience@dpi.nc.gov http://ncimplementationscience.ncdpi.wikispaces.net