1. Some Emerging Characteristics of Sustainable Practices
Thanks to Wing Institute, NIRN and
Kent McIntosh

2. Nature of the Problem
In education innovations come and go in months (Latham, 1988).
Alderman & Taylor (2003) Optimally, sustainability should be a focus from the day a project is implemented. With most projects, the pressure of just becoming operational often postpones such a focus until well into the 2nd year.

3. Memo To: School Administrators From: District Administrators
In keeping with the new state initiative, this fall we will be implementing an exciting new district initiative of SNI in place of LYI. All in-service days previously scheduled for LYI will be rescheduled as staff development for SNI. The $500 for release time and materials for LYI will be discontinued and provided instead for SNI. By the way, you will need to create local SNI teams that meet weekly. The former members of your LYI team would be perfect for this new team. Your new SNI binders will be coming next week. Have a great year!!!

4. Lot’s of times people have great ideas and have their eye on a goal, but don’t know how to put the pieces in place to make it happen. We have to figure out how to put the meat on the bones.

5. Why Such a Short Life Span?
High Effort
Innovation more difficult than expected.
Causes too much change.
Takes too much time.

6. Why Such a Short Life Span?
Poor system design
Supporters leave.
Personnel lack training.
External funds run out.
Inadequate supervision.
No accountability.
No consequences for early termination.

7. Even Well Tested Programs Often Fail to Sustain
Elliott & Mihalic (2004) review Blueprint Model Programs (violence prevention and drug prevention programs) replication in community settings.
Programs reviewed across 5 dimensions
Site selection
Training
Technical assistance
Fidelity
Sustainability.

8. Even Well Tested Programs Often Fail to Sustain
Critical elements in site readiness
Well connected local champion
Strong administrative support
Formal organizational commitments
Formal organizational staffing stability
Up front commitment of necessary resources
Program credibility within the community
Program sustained by the existing operational budget.

9. Even Well Tested Programs Often Fail to Sustain
Critical elements of training
Adhere to requirements (planning phase) for training, skills, and education.
Natural Implementers identified before scheduling training.
Encourage administrators to attend training- A MUST!
Plan and budget for staff turnover.
Implement program immediately after training.

10. Even Well Tested Programs Often Fail to Sustain
Critical elements of Technical Assistance
Proactive plan for technical assistance.
Critical elements of Fidelity
Monitor fidelity- TIC, IPI
Critical elements of Sustainability
Function of how well other dimensions are implemented.

11. Why Such a Short Life Span?
Those responsible for developing effective interventions do not necessarily have the skills to effect large systems change.
Systems change is different level of intervention.
Adults are the target of change rather than student behavior.

12. Ineffective Methods Excellent evidence for what does not work
Implementation by edict does not work
Implementation by “following the money” does not work
Implementation without changing supporting roles and functions does not work
Paul Nutt (2002) Why Decisions Fail

13. Emerging Features of Sustainable Programs
Maintain over time.
Maintain across generations of practitioners.
Operate within existing financial and staffing resources.

14. Implications of Current Measures
Current accountability measures of NCLB may reflect a change in emphasis.
Problem is that NCLB specifies outcomes but does not specify behaviors to accomplish outcome.
Consequence may be that test scores improve but student learning does not.
Multiple instances of cheating reported
Many schools spend great deal of time “teaching to the test.”
Remains to be seen if these accountability measures result in more effective practices that sustain.

15. Do you have a deep understanding of the principles of sustainability?
Common perception that sustainability is a ethereal, theoretical concept (Vaughn et al, 2000)
We all have experiences with it
The same principles of individual behavior still apply to systems…

16. Consequence
Behavior
Antecedent
Student
Outcomes
Select Practices
& Implement
with Fidelity
Ongoing
Challenge

17. Research on Sustainability
Descriptive information about what we think promotes sustainability
Based on theory
Based on some anecdotal observations
Clear descriptions of examples of non-sustainability
(Gersten & Chard, 2000; Sindelar et al., 2006; Vaughn et al., 2000)

18. Barriers to Sustainability: The Three Cs
Consequence
Behavior
Antecedent
Student
Outcomes
Select Practices
& Implement
with Fidelity
Ongoing
Challenge
Changes in Context
- Lack of contextual fit
- New challenges exist
- Competing initiatives
Changes in Capacity
- Loss of funding
- Attrition of key personnel
Changes in Consequences
- Diminished effectiveness due to poor fidelity
- Outcomes no longer perceived as important

19. A SWPBS Sustainability Study (Doolittle, 2006)
Sample: 285 schools with SET scores
Differences between schools that implemented and those that did not:
Expectations Taught
Monitoring and Decision Making
Differences between schools that sustained and those that did not:
On-going Behavioral Reward System
Management (Administrator)

20. Consequence
Behavior
Antecedent
Student
Outcomes
Identifying
& Modifying
Practices
Select Practices
& Implement
with Fidelity
Ongoing
Challenge

21. Identifying Valued & Modifying Outcomes Practices Practice
Antecedent
Consequence
Behavior
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Measurement
Practice
Implementation
Behavior

22. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

23. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

24. Importance in comparison to other practices
PRIORITY
Importance in comparison to other practices
Connection to other initiatives
Incorporation into core system components
Behavioral Principle:
Competing Schedules
of Reinforcement
Priority
Valued
Outcomes

25. Braid project into other initiatives
ENHANCING PRIORITY
Braid project into other initiatives
Show how practice can lead to other outcomes of new initiatives
Get into written policy
Advocate for improved visibility
Present data to people with resources
Describe effects of abandoning support for the practice
Priority
Valued
Outcomes

26. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

27. Extent to which the practice results in desired outcomes
EFFECTIVENESS
Extent to which the practice results in desired outcomes
Choice of practices should be based on proven effectiveness
Effects must be attributed to the practice
Behavioral Principle:
Reinforcement
Effectiveness
Identifying
& Modifying
Practices

28. Share data that show how adoption is related to effects
ENHANCING
EFFECTIVENESS
Select practices that are likely to produce the desired outcomes (i.e. Evidence-Based Practices)
Share data that show how adoption is related to effects
Effectiveness
Identifying
& Modifying
Practices

29. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

30. Relationship between continued effort and continued effectiveness
EFFICIENCY
Relationship between continued effort and continued effectiveness
Weighed against other potential practices
Behavioral Principle:
Maintenance
Practice
Implementation
Efficiency

31. A durable practice should become more efficient over time
ENHANCING EFFICIENCY
A durable practice should become more efficient over time
Easier on implementers
Repetition builds fluency
Easier to modify materials than create them
Easier on resources
Fewer visits from external consultants
Fewer release days
Practice
Implementation
Efficiency

32. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

33. Iterative monitoring of fidelity and outcomes
CONTINUOUS
REGENERATION
Iterative monitoring of fidelity and outcomes
Adaptation and re-adaptation over time while keeping critical features intact
Ongoing investment in the practice
Behavioral Principle:
Generalization
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement

34. DATA-BASED DECISION MAKING
ENHANCING
CONTINUOUS
REGENERATION
DATA-BASED DECISION MAKING
Fidelity
Outcomes
Use in problem-solving
Expand to new areas
Adjust practices for a changing environment
Cultivate local expertise
Connect with a community of practice
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement

35. Identifying Valued & Modifying Outcomes Practices Practice
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Practice
Implementation
Efficiency
School Context

36. Create a Plan to Sustain from the Start
“Train and Hope”
Not an effective approach to implement a practice
“Implement and Hope”
Not an effective way to sustain a practice
“Implementation are where good ideas go to die”
4 Big Ideas to Plan for Sustainability…

37. 1. Start with the Ending
Let the outcomes drive the selection of practices
Identify the valued outcomes for everyone
No one has ever been bullied or nagged into long-term sustainability
Measure and use data in decision making

38. 2. Death, Taxes, and… …Attrition
If the fidelity drops, the effects stop
Plan for your champions to move on/up
Focus on POSITIONS, not PERSONS
Create positions tied to the practice
Titles
Job Descriptions
FTE

39. 3. If you keep doing what you’re doing, you MAY NOT keep getting what you’re getting
Environments change – Continuous Improvement
adjust to changes (e.g. OISM, MISI)
New ideas keep the practice novel
Spread the practice
To new settings
To new systems

40. Were there 4 Big Ideas? How Can We Increase Sustainability of Practices?
Establish “Communities of Practice” at all levels
Executive Coaching, Team Leaders, Student Support Team-e.g. BISCC
Does it get easier? School level v. LEA level
Tipping Point

41. How Can We Increase Sustainability of Practices?
Continue Pro-active technical assistance- Coaches
Help solve the real problems of implementation.
Continue to Monitor integrity of implementation.
Without monitoring, the system likely to drift back to previous practices.
Recognition Program, SWIS
Anticipate 3-5 years before system is fully operational.
Emphasizes the need to plan for multigenerational support.

42. Future Research We need it Descriptive Experimental
Case studies of successes/failures
Experimental
Test the sustainability model

43. References
Alderman, H. S. & Taylor, L. (2003). On sustainability of project innovations as systemic change. Journal of Education and Psychological Consultation, 14 (1), 1-25.
Baum, W. M. (2000). Being concrete about culture and cultural evolution. In N. Thompson and F. Tonneau (Eds.) Perspectives in Ethology (Vol. 13, pp ). New York: Kluwer Academic/Plenum
Doolittle (2006)
Elliott, D. S. & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5(1),
Glenn, S. S. (2003). Operant contingencies and the origin of cultures. In K. A. Lattal and P. N. Chase (Eds.) Behavior Theory and Philosophy. New York: Kluwer Academic/Plenum.
Harris, M. (1979). Cultural Materialism: The struggle for a science of culture. New York: Simon and Shuster.
Latham, G. (1988). The birth and death cycles of educational innovations. Principal, 68(1), p
McIntosh, K., Horner, R. H., & Sugai, G. (in preparation). Sustainability of systems-level evidence-based practices in schools: Current knowledge and future directions.
Rogers, E. M. (2003). Diffusion of Innovations (5th Edition). New York: Free Press.