1. Get Started – Get Better: Using Improvement Cycles for SSIP
OSEP PD Conference
August 2016
Caryn Ward, PhD Dean Fixsen, PhD
© 2012 Karen A. Blase and Dean L. Fixsen

2. Formula for Success Effective Innovations Effective Implementation
Enabling Contexts
Educationally Significant Outcomes
© 2012 Dean Fixsen and Karen Blase, National Implementation Research Network

3. Active Implementation Frameworks
Usable Innovations
Implementation Stages
Implementation Drivers
Improvement Cycles
Implementation Teams
Enabling Change
© 2012 Karen A. Blase and Dean L. Fixsen

4. IMPROVEMENT CYCLES
Changing on purpose to support the new way of work

5. “Key Aspects of Improvement”
“Many initiatives fail for lack of study and reflection on what is actually being done and what the results are from having done it. Observing, describing, and documenting are key aspects to a program improvement cycle, and particularly critical during the pilot phase when key functions of interventions are emerging.”
The Child Wellbeing Project, Improvement Cycle Tool

6. Improvement Cycles PDSA Cycle (Deming, 1986)
Plan Do
Study
Act
What can be changed and improved?
How can problems be fixed?
Cycle
What is it you intend to do?
Teams normalize implementation challenges through intentional use of data and Improvement Cycles
Can you do it in practice?
Did you do it? What happened?

7. Implementation of Improvement Cycles
Meta-analysis N=73 Improvement Studies
Plan: 100%
Do: ???
Study: 15%
Act: 14%
Cycle: 19%
PDSAC = 3%
= Low Fidelity use of PDSAC Process
Taylor et al. (2014)
Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bel, D., & Reed, J. E. (2014). Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. British Medical Journal of Quality and Safety, 23( ).
Fix the problem, not the blame
Plan: 30/47 = 64%
Do: No data
Study: 31/47 = 66%
Act: No data
Cycle: 14/47= 30%
Only 2 of 73 PDSA articles reviewed did all 5 (3%)
Requires discipline to have a plan and stick to it
Requires studying and acting (plan-do, plan-do, plan-do = my colleague saying …)
Requires repetition until the goal is reached or the problem is solved (my other colleague saying …. often thwarted, never stymied)
© Dean Fixsen and Karen Blase, 2015

8. Practice-Policy Communication
Improvement Cycles
Improvement Cycles
Three critical Improvement Cycles:
Rapid Cycle Problem Solving
Usability Testing
Practice-Policy Communication Cycle
Improvement
Cycles
The PDSA thinking is embedded in all improvement cycles: PDSA, Usability Testing, Practice-Policy Communication Cycles
Time frames and number of people involved increase
PDSA
Rapid cycle, short-term, few people
Usability Testing
Medium cycle, medium-term, more people
Practice-Policy Communication
Longer cycle, longer-term, layers of people
(c) Allison Metz, 2012

9. Problem Solving Protocol
Rapid-Cycle
Problem Solving
Problem Solving Protocol
Clarify and validate the problem
Decide what variable(s) to problem solve
Set improvement target
Generate possible solution to improve attribute(s)
Implement a solution
Evaluate if solution worked
Continue or expand process
Plan Do
Components:
Assign Rapid-Cycle Problem Solving Leader
Create Team (Ad Hoc)
Apply a standard problem solving protocol
Focused version of Plan-Do-Study-Act cycle applied to rapid cycle problem solving
PLAN Do Study Act
1. Clarify and validate the problem
Use fidelity and outcome data to better understand the problem
Describe problem statements
Primary problem statement
Precision problem statement
Lack of clear performance expectations and relevant feedback about the adequacy of your performance.
Lack of tools, resources, and materials to achieve performance goals
Lack of adequate incentives made contingent on performance.
Lack of training that matches the requirements of the job.
Lack of match between skills and job requirements.
Lack of motivation to perform job well.
2. Decide what variable(s) to problem solve
What change(s) do we want to test?
Timeliness
Accuracy
Quantity
Ease of use
Duration
Focus on what are the alterable variables within the setting
Since the work is already taking place, focus on a few key variables that you might what to address to make it better
3. Set improvement target
Base on identified improvement variable(s)
4. Generate possible solutions to improve identified variable(s)
Link to data
Prioritize
Plan Do STUDY Act
6. Evaluate if solution worked
Plan Do Study ACT
7. Implement a solution
Adopt translate learning to larger organization
Revise/Modify based on learning
Try something different
Implications:
Sometimes we just need to get started and then work to get better!
Beneficial strategy when the practices are already taking place and we need to work out problems
Helps us to quickly detect, define and address new problems associated with the practices.
Study
Act

10. Problem-solving during early efforts
Rapid-Cycle
Problem Solving
Examples
Problem Oriented:
Generating more timely reports for monitoring progress of students who receive new curriculum and instruction
Incremental Improvement:
Improving the integration of meaningful parent input into the school-wide anti-bullying program.
Since the work is already taking place, focus on a few key variables that you might what to address to make it better
Problem-solving during early efforts
Team Lead identified
Right people on the team
Time-limited to address the problem
Team disbands
Practice Improvement
On-going efforts to improve practices and competencies
Use data to achieve better outcomes for children and “embed” solutions

11. Improvement Cycles Medium cycle, medium-term, more people
Usability Testing
Medium cycle, medium-term, more people
It is well suite to situations where the innovation is complete and involves more people or several aspects of an orgniazation or system working tothere. There are more examples
A planned series of tests of an innovation, components of an innovation, or implementation processes. 
Makes use of a series of PDSA cycles to refine and improve:
the innovation elements
the implementation processes 
Used proactively to test the feasibility and impact of a new way of work prior to rolling out the innovation or implementation processes more broadly
NOT A PILOT
Processes are more likely to be feasible and effective
Implementation Drivers can support the “right” processes
Data collection and reports are feasible, reliable, valid, useful
Much more is learned from 4 cycles with 5 participants in each cycle than from one pilot test with 20 participants (Nielsen, 2000) 11

12. Usability Testing (cont.)

13. Practice Policy Communication Cycle
External Implementation Support
Practice Informs Policy
Policy Enables Practices
Plan
Policy
Practice
Structure
Procedure
Feedback
Study - Act
Longer cycle, longer-term, layers of people
The process occurs less frequently and at a slower pace than rapid cycle problem solving and usability testing
Design to provide to provide organization al leaders and policy makers with information about implementation barriers and successes so that a more aligned system can be developed.
Feedback from the practice level engages and informs organizational leaders so that they can ensure policy, procedures, resources et enable innovative an d effective practices as intended
End goal is to have communication process between and among levels routine, transparent, reinforced, and functional
Practice-Policy Communication
Never ends; a new way of work for the SMT and the education system
Avenues of direct communication focused on supporting teachers and producing greatly improved student outcomes Do
Practice
FORM SUPPORTS FUNCTION

14. Improvement Cycles: Comparison
Focus
Primary Point of Application
Cycle Duration
Purposeful Use
Rapid-Cycle Problem Solving
Individual Practices
Initially identifying problems and solutions
Daily or weekly cycles
Development team
and/or ad-hoc team
Usability Testing
Groups of practices; operating methods
Testing the feasibility of solutions and developing administrative supports
Weekly or monthly cycles
Practice-Policy Communication Cycles: System Change
Modification of units and relationship between units
Executive leadership and others with authority to change system units
Monthly or bimonthly cycles
On-going improvement team

15. District Scenarios On your own, read the four district scenarios.
With a partner, answer the following questions:
What are the similarities amongst the four approaches?
What are the differences?
Which one is making intention use of improvement cycles? 15

16. Putting it all together….
Rapid Cycle Problem Solving
Usability Testing
Practice Policy Communication Cycle

17. Transformation Zone “Vertical slice” of the system Representative
Small enough to be manageable
Large enough to “disturb” the system and impact key aspects of the system, yet not impact the entire system

18. Transformation Zone Diagram
SISEP Support
State Management Team
State Implementation Team
Repurpose Roles, Functions, & Structures to Maximize Outcomes
Regional Implementation Team
Zone
Transformation
The model for implementing this work focuses on one initiative and starts with two transformation zones (TZ). Each year the TZs build capacity for scaling-up activities so that in three to five years our state grows to full implementation (80% of districts within our states are implementing the innovation with high fidelity).
The Association of Educational Service Districts participated in the Mutual Exploration Process and through this process our state has identified two Transformation Zones. NEWESD 101 as one Transformation Zone and PSESD 121 has partnered with Capitol ESD 113 to create the second Transformation Zones in our state.
Now that Washington is partnering with regions and beginning active implementation we are receiving “Intensive Technical Support” from SISEP. This “intensive” SISEP support includes training and coaching from SISEP’s Transformation Specialist to support our state in using free and nationally vetted assessment tools to drive action planning,
These national-level Transformation Specialists provide active modeling, on-going coaching and training to State Transformation Specialists and all Implementation teams in the first transformation zones to build capacity so that our state can scale-up the system.
State Transformation Specialist will work with the state management team, state design team members and Regional Implementation Teams to build capacity within each TZ. This includes working with Implementation teams from the Regional level, to the District- and School Building-level to ensure the infrastructure supports implementation in each classroom.
District Implementation Team
District Implementation Team
District Implementation Team
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
Building Implementation Teams
(c) Dean Fixsen and Karen Blase, 2004
(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 18 18

19. Factors to Consider in Size and Location of the TZ
How deep and significant does this change the status quo?
Do we have the necessary expertise and knowledge related to the innovation itself?
How experienced and accessible are our implementation teams?
How well resourced is the implementation effort?
How deep and significantly does this change the status quo?
How different is the new way of work?
How much change will be required at each level of the system?
How much competence and confidence building will be needed at every level?
Do we have the necessary knowledge and expertise related to the innovation itself?
Are there many people available who are “experts” in the new way of work?  Or are we relying on a few?
Is the new way of work well operationalized or is that part of the work in the first Transformation Zone experience?
Is formal technical assistance and support available to us or are we ‘building and flying the plane’?
How experienced and accessible are our Implementation Teams?
Is this their first experience or are we able to “seed” new teams with experienced people who can guide and support the new teams?
Can we manage the geographic spread of the sites we have are considering?  Can we get to them frequently enough to do the work required?
How well resourced is the implementation effort?
Are we layering this onto the work of already too-busy people?
Are other initiatives actively competing for attention?
Do we have the ability to for the Implementation Team or Teams to meet frequently, communicate regularly and use PDSA processes?
© Dean Fixsen and Karen Blase, 2015

20. Transformation Zone Components Clearly defined program Core Components
Indicators of Fidelity
Selection of initial implementers
Selection Criteria
Requisite variety
System components
Want to test worthy elements to test – worthy elements are the instructional, intervention, setting or implementation processes that increase liklihood of getting good outcomes.

21. Transformation Zone Training Just in time Just enough
Components (cont.)
Training
Just in time
Just enough
New knowledge vs. new skills
Data Collection
What information?
Reported to whom?
On what schedule?

22. Transformation Zone Components (cont.) Criteria for Success
Indicators for next iteration
Indicators for re-design
Program vs. system information
Next Iteration

23. Transformation Zone Criteria for Success
State Capacity Assessment at 80% or better on each subscale
Regional Capacity Assessment at 80% or better on each subscale
District Capacity Assessment at 80% or better on each subscale
Drivers Best Practices Assessment at 80% or better
Fidelity Assessment criteria met for 50% or more of the teachers

24. Within a Transformation Zone:
Cohort 1:
District A
District B
District C
Cohort 2:
District D
District E
District F
Cohort 3:
District G
District H
District I
Cohort 4:
District J
District K
District L
State & Regional Implementation Teams

25. State Management & Implementation Teams
Scaling within a SEA
Cohort 1:
Transformation Zone A
Cohort 2:
Transformation Zone B
Cohort 3:
Transformation Zone C
Cohort 4:
Transformation Zone D
State Management & Implementation Teams

26. Case Study
In pairs:
Review the state case study and underline the critical characteristics of a transformation zone
How is it similar to your state’s SSIP respective project work?
How is it different to your state’s SSIP or respective project? 26

27. Reinvention for Impact
Existing System
Effective Innovations
Are Changed to
Fit The System
Existing System Is
Changed To Support
The Effectiveness Of
The Innovation
Effective Innovation
Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children (Special Issue), 79(2),
Fixsen, D., Blase, K., & Van Dyke, M. (2012). From ghost systems to host systems via transformation zones. Washington, DC: U.S. Department of Education Office of Vocational and Adult Education.

28. Get Started Get Better Get Started, Get Better Purposeful = Improvable
Nielsen, J. (2000). Why you only need to test with 5 users. Retrieved April 22, 2007, from 28

29. For more information Allison Metz Caryn Ward Allison.metz@unc.edu
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
(c) Allison Metz, 2014

30. GIC 2017

31. For more on Implementation Science
Get Connected!
Workgroups BIC CoP
@SISEPcenter
SISEP
For more on Implementation Science 31

32. Citation and Copyright Information
This document is based on the work of the National Implementation Research Network (NIRN).
© Allison Metz, Leah Bartley, Jonathan Green, Laura Louison, Sandy Naoom, Barbara Sims, and Caryn Ward
This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs . You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter, transform, or build upon this work. Any of the above conditions can be waived if you get permission from the copyright holder.
web:
The mission of the National Implementation Research Network (NIRN) is to contribute to the best practices and science of implementation, organization change, and system reinvention to improve outcomes across the spectrum of human services.