Some Emerging Characteristics of Sustainable Practices Thanks to Wing Institute, NIRN and Kent McIntosh

Nature of the Problem In education innovations come and go in 18-48 months (Latham, 1988). Alderman & Taylor (2003) Optimally, sustainability should be a focus from the day a project is implemented. With most projects, the pressure of just becoming operational often postpones such a focus until well into the 2nd year.

Memo To: School Administrators From: District Administrators In keeping with the new state initiative, this fall we will be implementing an exciting new district initiative of SNI in place of LYI. All in-service days previously scheduled for LYI will be rescheduled as staff development for SNI. The $500 for release time and materials for LYI will be discontinued and provided instead for SNI. By the way, you will need to create local SNI teams that meet weekly. The former members of your LYI team would be perfect for this new team. Your new SNI binders will be coming next week. Have a great year!!!

Lot’s of times people have great ideas and have their eye on a goal, but don’t know how to put the pieces in place to make it happen. We have to figure out how to put the meat on the bones.

Why Such a Short Life Span? High Effort Innovation more difficult than expected. Causes too much change. Takes too much time.

Why Such a Short Life Span? Poor system design Supporters leave. Personnel lack training. External funds run out. Inadequate supervision. No accountability. No consequences for early termination.

Even Well Tested Programs Often Fail to Sustain Elliott & Mihalic (2004) review Blueprint Model Programs (violence prevention and drug prevention programs) replication in community settings. Programs reviewed across 5 dimensions Site selection Training Technical assistance Fidelity Sustainability.

Even Well Tested Programs Often Fail to Sustain Critical elements in site readiness Well connected local champion Strong administrative support Formal organizational commitments Formal organizational staffing stability Up front commitment of necessary resources Program credibility within the community Program sustained by the existing operational budget.

Even Well Tested Programs Often Fail to Sustain Critical elements of training Adhere to requirements (planning phase) for training, skills, and education. Natural Implementers identified before scheduling training. Encourage administrators to attend training- A MUST! Plan and budget for staff turnover. Implement program immediately after training.

Even Well Tested Programs Often Fail to Sustain Critical elements of Technical Assistance Proactive plan for technical assistance. Critical elements of Fidelity Monitor fidelity- TIC, IPI Critical elements of Sustainability Function of how well other dimensions are implemented.

Why Such a Short Life Span? Those responsible for developing effective interventions do not necessarily have the skills to effect large systems change. Systems change is different level of intervention. Adults are the target of change rather than student behavior.

Ineffective Methods Excellent evidence for what does not work Implementation by edict does not work Implementation by “following the money” does not work Implementation without changing supporting roles and functions does not work Paul Nutt (2002) Why Decisions Fail

Emerging Features of Sustainable Programs Maintain over time. Maintain across generations of practitioners. Operate within existing financial and staffing resources.

Implications of Current Measures Current accountability measures of NCLB may reflect a change in emphasis. Problem is that NCLB specifies outcomes but does not specify behaviors to accomplish outcome. Consequence may be that test scores improve but student learning does not. Multiple instances of cheating reported Many schools spend great deal of time “teaching to the test.” Remains to be seen if these accountability measures result in more effective practices that sustain.

Do you have a deep understanding of the principles of sustainability? Common perception that sustainability is a ethereal, theoretical concept (Vaughn et al, 2000) We all have experiences with it The same principles of individual behavior still apply to systems…

Consequence Behavior Antecedent Student Outcomes Select Practices & Implement with Fidelity Ongoing Challenge

Research on Sustainability Descriptive information about what we think promotes sustainability Based on theory Based on some anecdotal observations Clear descriptions of examples of non-sustainability (Gersten & Chard, 2000; Sindelar et al., 2006; Vaughn et al., 2000)

Barriers to Sustainability: The Three Cs Consequence Behavior Antecedent Student Outcomes Select Practices & Implement with Fidelity Ongoing Challenge Changes in Context - Lack of contextual fit - New challenges exist - Competing initiatives Changes in Capacity - Loss of funding - Attrition of key personnel Changes in Consequences - Diminished effectiveness due to poor fidelity - Outcomes no longer perceived as important

A SWPBS Sustainability Study (Doolittle, 2006) Sample: 285 schools with SET scores Differences between schools that implemented and those that did not: Expectations Taught Monitoring and Decision Making Differences between schools that sustained and those that did not: On-going Behavioral Reward System Management (Administrator)

Consequence Behavior Antecedent Student Outcomes Identifying & Modifying Practices Select Practices & Implement with Fidelity Ongoing Challenge

Identifying Valued & Modifying Outcomes Practices Practice Antecedent Consequence Behavior Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Measurement Practice Implementation Behavior

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Importance in comparison to other practices PRIORITY Importance in comparison to other practices Connection to other initiatives Incorporation into core system components Behavioral Principle: Competing Schedules of Reinforcement Priority Valued Outcomes

Braid project into other initiatives ENHANCING PRIORITY Braid project into other initiatives Show how practice can lead to other outcomes of new initiatives Get into written policy Advocate for improved visibility Present data to people with resources Describe effects of abandoning support for the practice Priority Valued Outcomes

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Extent to which the practice results in desired outcomes EFFECTIVENESS Extent to which the practice results in desired outcomes Choice of practices should be based on proven effectiveness Effects must be attributed to the practice Behavioral Principle: Reinforcement Effectiveness Identifying & Modifying Practices

Share data that show how adoption is related to effects ENHANCING EFFECTIVENESS Select practices that are likely to produce the desired outcomes (i.e. Evidence-Based Practices) Share data that show how adoption is related to effects Effectiveness Identifying & Modifying Practices

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Relationship between continued effort and continued effectiveness EFFICIENCY Relationship between continued effort and continued effectiveness Weighed against other potential practices Behavioral Principle: Maintenance Practice Implementation Efficiency

A durable practice should become more efficient over time ENHANCING EFFICIENCY A durable practice should become more efficient over time Easier on implementers Repetition builds fluency Easier to modify materials than create them Easier on resources Fewer visits from external consultants Fewer release days Practice Implementation Efficiency

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Iterative monitoring of fidelity and outcomes CONTINUOUS REGENERATION Iterative monitoring of fidelity and outcomes Adaptation and re-adaptation over time while keeping critical features intact Ongoing investment in the practice Behavioral Principle: Generalization Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement

DATA-BASED DECISION MAKING ENHANCING CONTINUOUS REGENERATION DATA-BASED DECISION MAKING Fidelity Outcomes Use in problem-solving Expand to new areas Adjust practices for a changing environment Cultivate local expertise Connect with a community of practice Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement

Identifying Valued & Modifying Outcomes Practices Practice Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

Create a Plan to Sustain from the Start “Train and Hope” Not an effective approach to implement a practice “Implement and Hope” Not an effective way to sustain a practice “Implementation are where good ideas go to die” 4 Big Ideas to Plan for Sustainability…

1. Start with the Ending Let the outcomes drive the selection of practices Identify the valued outcomes for everyone No one has ever been bullied or nagged into long-term sustainability Measure and use data in decision making

2. Death, Taxes, and… …Attrition If the fidelity drops, the effects stop Plan for your champions to move on/up Focus on POSITIONS, not PERSONS Create positions tied to the practice Titles Job Descriptions FTE

3. If you keep doing what you’re doing, you MAY NOT keep getting what you’re getting Environments change – Continuous Improvement adjust to changes (e.g. OISM, MISI) New ideas keep the practice novel Spread the practice To new settings To new systems

Were there 4 Big Ideas? How Can We Increase Sustainability of Practices? Establish “Communities of Practice” at all levels Executive Coaching, Team Leaders, Student Support Team-e.g. BISCC Does it get easier? School level v. LEA level Tipping Point

How Can We Increase Sustainability of Practices? Continue Pro-active technical assistance- Coaches Help solve the real problems of implementation. Continue to Monitor integrity of implementation. Without monitoring, the system likely to drift back to previous practices. Recognition Program, SWIS Anticipate 3-5 years before system is fully operational. Emphasizes the need to plan for multigenerational support.

Future Research We need it Descriptive Experimental Case studies of successes/failures Experimental Test the sustainability model

References Alderman, H. S. & Taylor, L. (2003). On sustainability of project innovations as systemic change. Journal of Education and Psychological Consultation, 14 (1), 1-25. Baum, W. M. (2000). Being concrete about culture and cultural evolution. In N. Thompson and F. Tonneau (Eds.) Perspectives in Ethology (Vol. 13, pp. 181-212). New York: Kluwer Academic/Plenum Doolittle (2006) Elliott, D. S. & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5(1), 47-53. Glenn, S. S. (2003). Operant contingencies and the origin of cultures. In K. A. Lattal and P. N. Chase (Eds.) Behavior Theory and Philosophy. New York: Kluwer Academic/Plenum. Harris, M. (1979). Cultural Materialism: The struggle for a science of culture. New York: Simon and Shuster. Latham, G. (1988). The birth and death cycles of educational innovations. Principal, 68(1), p41 -43. McIntosh, K., Horner, R. H., & Sugai, G. (in preparation). Sustainability of systems-level evidence-based practices in schools: Current knowledge and future directions. Rogers, E. M. (2003). Diffusion of Innovations (5th Edition). New York: Free Press.