1. Implementation and Scaling Literacy Programs
Literacy Funders Network 2011
Dean L. Fixsen, Karen A. Blase,
Leah Bartley, Michelle Duda,
Sandra Naoom, Allison Metz,
Barbara Sims, Melissa Van Dyke
National Implementation Research Network
Frank Porter Graham Child Development Institute
University of North Carolina at Chapel Hill

2. Education

3. Capacity Building Implementation Teams Organization Change
System Reinvention
Capacity
Funding
AMOUNTS
YEARS

4. 1 person in 5 in the USA is in school
Education
60 million students
6 million teachers and staff
90,000 school buildings
3,147 counties
58 federal jurisdictions
1 person in 5 in the USA is in school
(c) Dean Fixsen and Karen Blase, 2010

5. All Schools Left Behind
5 Years of turnaround work
10% out of improvement status
90% still “in improvement”
Stuit (2011; Are bad schools immortal?)
Secretary Duncan predicts 82% of all schools in the US will not meet AYP standards in 2011
Today’s Highlights
Vol. 44, No. 12 · Thursday, January 20, 2011
Prefer to get Ed Daily® by ?
Just call LRP customer service
at (800)
School improvement
Researcher: Inattention poses threat
to SIG outcomes Page 2
STEM education
Report: Lowest-performing
schools often fail to improve
By Adam Dolge
A recent report from the Thomas B. Fordham
Institute and Basis Policy Research finds
that only 1 percent of the lowest-performing K-8
schools improved over a five-year period.
The study, Are Bad Schools Immortal?, examined
more than 2,000 low-performing schools
(1,768 district-operated and 257 charters) in
10 states from SY to SY The
purpose of the study, released last month, was to
determine how many schools improved beyond
the 50th percentile within the five-year period.
The study also examined school closures.
“Turnarounds are not easy; we show that
with data” said David Stuit, author of the study
and an analyst with Basis. “Going through the
motions and just doing what is expected based on
federal guidelines is not going to get the job done.
It will work for 1 percent of the schools.”
Bleak outlook
The report offers a bleak outlook on school
improvement initiatives, as only 1.4 percent of
the district schools and less than 1 percent of the
charter schools met the study’s definition of “turnaround.”
The study defined a turnaround school as
a school in its state’s lowest decile (proficiency at or
below the 10th percentile) at the beginning of the review
period that had to surpass the 50th percentile
within five years.

6. Food for Thought
"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get.” …R. Spencer Darling 6

7. National Assessment of Education Progress (NAEP)
WHAT IS IT GOING TO TAKE?

8. Challenges
Students cannot benefit from instruction they do not experience
Teachers and staff have to actually use effective instructional methods skillfully if students are to benefit
Verbal advocacy ≠ Actual use
Dobson, L., & Cook, T. (1980). Avoiding Type III error in program evaluation: results from a field experiment. Evaluation and Program Planning, 3,
Dobson & Cook (1980)

9. Teachers and Staff In education, the TEACHER IS THE INTERVENTION
Everyone / everything else needs to be aligned to provide effective supports so all teachers can produce desired education outcomes for all students
(c) Dean Fixsen and Karen Blase, 2004 9

10. Consider This To meet the individualized learning needs of students …
Educators need to “forgo uniqueness” in favor of consistent use of effective methods …
That are well supported by district and building staff.
(c) Dean Fixsen and Karen Blase, 2004

11. Evidence-based Implementation Research: A Synthesis of the Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:   11

12. Implementation Gap
Interventions are not used as intended and with good outcomes
Interventions are not sustained for a useful period of time
Interventions are not used on a scale sufficient to impact social problems

13. Longitudinal Studies of a Variety of Comprehensive School Reforms
Implementation Science
Longitudinal Studies of a Variety of Comprehensive School Reforms
Evidence-base
Actual Supports
Years 1-3
Outcomes
Years 4-5
Every Teacher Trained
Fewer than 50% of the teachers received some training
Fewer than 10% of the schools used the CSR as intended
Every Teacher Continually Supported
Fewer than 25% of those teachers received support
Vast majority of students did not benefit
Aladjem, D. K., & Borman, K. M. (2006, April). Summary of Findings from the National Longitudinal
Evaluation of Comprehensive School Reform. Paper presented at the Annual meeting of the American
Educational Research Association, San Francisco, CA.
Vernez, G., Karam, R., Mariano, L. T., & DeMartini, C. (2006). Evaluating comprehensive school reform models at scale:
Focus on implementation. Santa Monica, CA: RAND Corporation.
Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

14. Implementation Science
Letting it happen
Recipients are accountable
Helping it happen
Making it happen
Implementation Teams are accountable
Fixsen, D. L., Blase, K., Duda, M., Naoom, S., & Van Dyke, M. (2010). Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. In J. Weisz & A. Kazdin (Eds.), Implementation and dissemination: Extending treatments to new populations and new settings (2nd ed., pp ). New York: Guilford Press.
Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4),
Hall, G., & Hord, S. M. (1987). Change in schools: Facilitating the process. Albany, NY: SUNY Press.
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010) 14

15. Implementation Science
Best Data Show These Methods, When Used Alone, Do Not Result In Uses of Innovations As Intended:
Diffusion/ Dissemination of information
Training
Passing laws/ mandates/ regulations
Providing funding/ incentives
Organization change/ reorganization
About 5% to 20% Return on Investment
Paul Nutt (2002). Why Decisions Fail

16. Formula for Success X = Intended outcomes
Effective intervention practices X
Effective implementation practices =
Intended outcomes
Disproportional Impact: Students benefit 8 to 12 times more 16

17. Implementation Science
An intervention is one thing
Implementation is something else altogether
Like serum and a syringe
Very different evidence bases
Each is necessary
Neither one is useful without the other

18. 2013

19. Implementation Capacity
It is not enough to have high standards, the teachers must be supported if they are to teach consistently and effectively
Shifts accountability for outcomes from the teachers to the staff in buildings, districts, and supporting regional and State education systems

20. Literacy Improvement Improve student outcomes
Improve teacher performance
Improve school supports for teachers
Improve district supports for schools
Improve regional supports for districts
Improve State supports for outcomes
(c) Dean Fixsen and Karen Blase, 2004

21. Consistent uses of Innovations Implementation Drivers
Reliable Benefits
Consistent uses of Innovations
Interventions meet
Implementation
Implementation Drivers
Performance Assessment (fidelity)
Systems
Intervention
Facilitative
Administration
Decision Support
Data System
Organization Drivers
Coaching
Training
Competency Drivers
Integrated & Compensatory
Selection
Adaptive
Technical
Leadership Drivers
© Fixsen & Blase, 2008 21

22. Staff Training & Coaching
 OUTCOMES
(% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting,
and Use new Skills in the Classroom)
TRAINING
COMPONENTS
Knowledge
Skill
Demonstration
Use in the Classroom
Theory and Discussion
10% 5% 0%
..+Demonstration in Training
30%
20%
…+ Practice & Feedback in Training
60%
…+ Coaching in Classroom
95%
The 2002 meta-analysis of training and coaching data by Joyce and Showers makes a compelling case for the need for skillful coaching. Only when training was accompanied by coaching in the service setting – in this case a classroom, was there substantial implementation in the practice setting.
These findings move supervision from systems that monitor units of service, react to crises and advise around case specifics to active coaching systems that monitor adherence to evidence-based practices, are purposeful in developing practitioner skills and offer support in trying out new approaches during that “awkward stage” just after training.
Joyce and Showers, 2002 22

23. Literacy Improvement Improve student outcomes
Improve teacher performance
Improve school supports for teachers
Improve district supports for schools
Improve regional supports for districts
Improve State supports for outcomes
(c) Dean Fixsen and Karen Blase, 2004

24. Effective Instructional
Literacy
Outcomes
for Students
Teachers
Staff
Administrators
Boards
Effective Instructional
Practices
WHAT
Intervention
Performance
Assessments
(fidelity)
Implementation
Teams
Processes
HOW
There may be value in discriminating implementation fidelity from intervention fidelity
Implementation fidelity can be thought of as the HOW for installing the WHAT (e.g. active ingredients, interventions)
Synthesis findings:
A number of studies reported a mix of intervention fidelity measures and implementation fidelity measures
Most valuable found links from implementation measures (e.g. amount and type of coaching or training) and intervention fidelity measures (e.g. Therapist Adherence Measures – MST) with subsequent links to client outcomes.
Studies were rare
WHO
Implementation
Performance
Assessments
(fidelity)
Exploration
Installation
Initial Implementation
Full Implementation 24

25. Implementation Science
Letting it happen
Recipients are accountable
Helping it happen
Making it happen
Implementation Teams are accountable: THEY DO THE WORK 25

26. Implementation Team
Minimum of three people (four or five preferred) to promote effective, efficient, and sustainable implementation, organization change, and system transformation work
Tolerate turnover; teams are sustainable even when the players come and go
An advantage of having a well organized and persistent approach to implementation of evidence-based practices and programs may be that the purveyor can accumulate knowledge over time (Fixsen & Blase, 1993; Fixsen, Phillips, & Wolf, 1978; Winter & Szulanski, 2001). Each attempted implementation of the program reveals barriers that need to be overcome and their (eventual) solutions. Problems encountered later on may be preventable with different actions earlier in the implementation process. The Toyota Supplier and Support Center (TSSC) is a purveyor of the Toyota Production Systems for manufacturing automobiles. MST Services, Inc. is the purveyor of the Multisystemic Therapy (MST) program for serious and chronic juvenile offenders. These are clear-cut examples of purveyors and each has a set of activities designed to help new organizations ("implementation sites") implement their respective programs. In other cases, the "purveyor" is not so readily identified nor are the activities well described. For example, the Assertive Community Treatment program and the Wraparound approach seem to have several individuals who act as consultants to communities and agencies interested in adopting those programs. The Wraparound group has recognized the problem of multiple definitions of their approach being used by different purveyors and have formed a national association to develop a common definition of the approach and a common set of processes for assessing the fidelity of new implementation sites (Bruns, Suter, Leverentz-Brady, & Burchard, 2004). The literature is not always clear about the activities of a purveyor. For example, the Quantum Opportunity Program (Maxfield, Schirm, & Rodriguez-Planas, 2003) was implemented in several sites in a major, multi-state test of the program. The report of the findings simply noted that the originators of the program had received funding to provide technical assistance to the implementation sites. Given the uneven results, it is unfortunate that there was no link back to purveyor activities. 26

27. Implementation Team Simultaneous, Multi-Level Interventions
Building/District Supports
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Teacher/Staff Competence
State and Community Supports
Regional Authority Supports
Implementation Team
Purveyors also quickly learn that the sphere of influence is critical to the success of the implementation effort and over time take on a very active, simultaneous and multi-level intervention role to help increase the likelihood that such meta-contingencies as funding, licensing, referral mechanisms, regulations, and reporting requirements are aligned to support the new way of work

28. Assure Implementation
Implementation Team
Prepare Teachers and Staff
Prepare Buildings & Districts
Prepare Regions
Implementation Team
Assure Student Benefits
Blase, K. A., Fixsen, D. L., & Phillips, E. L. (1984). Residential treatment for troubled children: Developing service delivery systems. In S. C. Paine, G. T. Bellamy & B. Wilcox (Eds.), Human services that work: From innovation to standard practice (pp ). Baltimore, MD: Paul H. Brookes Publishing.
Slavin, R. E., & Madden, N. A. (1999). Disseminating Success for All: Lessons for Policy and Practice (No. 30). Baltimore, MD: Center for Research on the Education of Students Placed at Risk (CRESPAR)- Johns Hopkins University.
Leonard-Barton, D., & Kraus, W. A. (1985). Implementing new technology. Harvard Business Review, 6,
Glennan Jr., T. K., Bodilly, S. J., Galegher, J. R., & Kerr, K. A. (2004). Expanding the reach of education reforms. Santa Monica, CA: RAND Corporation.
Work with Researchers
Parents and Stakeholders
20%
80%
Create Readiness
Assure Implementation
© Fixsen & Blase, 2009 28 28 28

29. Implementation Science
Impl. Team
NO Impl. Team
80%, 3 Yrs
14%, 17 Yrs
Effective
INTERVENTION
Balas & Boren, 2000
Effective use of Implementation Science & Practice
Letting it Happen Helping it Happen
It takes an estimated average of 17 years for only 14% of new scientific discoveries to enter day-to-day clinical practice (Balas & Boren, 2000)
Balas EA, Boren SA. Yearbook of Medical Informatics: Managing Clinical Knowledge for Health Care Improvement. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH; 2000.
With the use of competent Implementation Teams, over 80% of the implementation sites were sustained for 6 years or more (up from 30%) and the time for them to achieve Certification was reduced to 3.6 years.
Fixsen, D. L., Blase, K. A., Timbers, G. D., & Wolf, M. M. (2001). In search of program implementation: 792 replications of the Teaching-Family Model. In G. A. Bernfeld, D. P. Farrington & A. W. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp ). London: Wiley.
Fixsen, Blase, Timbers, & Wolf, 2001
Balas & Boren, 2000
3X to 12X Return on Investment

30. Costs and Savings

31. Statewide Change
Innovative practices do not fare well in existing organizational structures and systems
Organizational and system changes are essential to successful use of innovations
Expect it
Plan for it

32. Foundational Polices & Regulations
Compliance and Crises, Urgent, Time Sensitive!!
Services not meeting Standards
Deal with urgent and high profile issues
Best Practices
Implemented Fully
With Good Outcomes
System Supports & Stability
Regulatory roles
Basic Data Systems
Financing and Fiscal Accountability
Accreditation/ Licensing Standards
HR rules and regulations
Safety Standards
Work with Legislature
Inclusion of Stakeholders
System Supports
& Stability
Mandates,
System Supports,
Foundational Polices & Regulations
Leadership Responsibilities and Leverage Points
Thanks to Tom Bellamy

33. System Reinvention State Management Team System Change
“External” System Change Support
State
Management Team
Adaptive Challenges
Duplication
Fragmentation
Hiring criteria
Salaries
Credentialing
Licensing
Time/ scheduling
Union contracts
RFP methods
Federal/ State laws
Policy Enabled Practice
Practice Informed Policy
System Change
♦Look for Faulty Assumptions & Errors; ♦Make Needed Changes; ♦Invite System to Respond
Implementation
Team
Teachers
Innovations
Students
(c) Dean Fixsen and Karen Blase, 2004
(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 33 33

34. Capacity Building Implementation Teams Organization Change
System Reinvention
Capacity
Funding
AMOUNTS
YEARS

35. For More Information Karen A. Blase, Ph.D. Dean L. Fixsen, Ph.D.
Dean L. Fixsen, Ph.D.
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC 35

36. Implementation Science
Implementation Research:
A Synthesis of the Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:   36

37. Thank You for your Support
Annie E. Casey Foundation (EBPs and cultural competence)
William T. Grant Foundation (implementation literature review)
Substance Abuse and Mental Health Services Administration (implementation strategies grants; national implementation awards)
Centers for Disease Control & Prevention (implementation research)
National Institute of Mental Health (research and training grants)
Juvenile Justice and Delinquency Prevention (program development and evaluation grants
Office of Special Education Programs (Scaling up Capacity Development Center)
Administration for Children and Families (Child Welfare Leadership; Capacity Development)
Duke Endowment (Child Welfare Reform)