1. Scaling and Sustaining PBIS: State, District, School Roles Rob Horner University of Oregon www.pbis.org

2. Goals Specific suggestions for state, district and school personnel implementing PBIS (or any MTSS) ----------------------------------------------------------------------- School/ District Self-Assessment State/District (District Capacity Assessment)

3. Assumptions Knowledgeable about PBIS Demonstrations of effective implementation Read to focus on sustainability, scaling and building PBIS at all three tiers.

4. Why SWPBIS? The fundamental purpose of SWPBIS is to make schools more effective learning environments. Predictable Consistent Positive Safe

5. Current Status Multiple Models for Scaling

6. Leadership Team Funding Visibility Political Support TrainingCoachingEvaluation Local School Demonstrtions Active Coordination Technical Expertise Policy

7. Implementation Science Frameworks WHO Teams WHEN Stages HOW Drivers HOW Cycles WHAT Interventions

8. © Fixsen & Blase, 2008 Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Implementation Drivers Competency Drivers Organization Drivers Leadership AdaptiveTechnical Successful Student Outcomes Program/Initiative/Framework (e.g. RtI)

9. Implementation Stages Exploration Installation Initial Implementation Full Implementation Implementation occurs in stages: Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 2 – 3 Years

10. Stages of Implementation FocusStageDescription Exploration/ Adoption Decision regarding commitment to adopting the program/practices and supporting successful implementation. InstallationSet up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan. Initial Implementation Try out the practices, work out details, learn and improve before expanding to other contexts. Full Implementation Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation. Continuous Improvement/ Regeneration Make it easier, more efficient. Embed within current practices. Work to do it right! Work to do it better! Should we do it! Steve Goodman

11. Scaling up School-wide Positive Behavioral Interventions and Supports: The Experiences of Seven States with Documented Success Rob Horner, Don Kincaid, George Sugai, Tim Lewis, Lucille Eber, Susan Barrett, Celeste Rossetto Dickey, Mary Richter, Erin Sullivan, Cyndi Boezio, Nancy Johnson, (2014 ), JPBI ExplorationInstallationInitial ImpFull Imp Leadership Team Funding Visibility Political Support Policy Training Coaching Expertise Evaluation Demos Interviews and Data Reviews with the PBIS implementers from Seven States that had at least 500 schools using PBIS.

12. Exploration and Adoption InstallationInitial Implementation Full Implementation Innovation and sustainability Leadership Team (coordination) Do you have a state leadership team? If you do, how was your first leadership team developed? Who were members? Who supported/lead the team through the exploration process? Was any sort of self- assessment completed (e.g. the PBIS Implementation Blueprint Assessment)? What was the role of State agency personnel in the exploration phase? What were critical issues that confronted the team as it began to install systems changes? What were specific activities the team did to ensure success of the initial implementation efforts? Did the team change personnel or functioning as the # of schools/districts increased? What has the Leadership team done to insure sustainability? In what areas is the State “innovating” and contributing to the research and practice of PBIS (e.g. linking PBIS with literacy or math)?

13. Descriptive Summary: Oregon Exploration / Installation / Initial Imp /Full Imp & Innovate

14. Descriptive Summary: Missouri Exploration / Installation /Initial Imp / Full Imp & Innovate

15. Descriptive Summary: North Carolina Exploration / Installation / Initial & Full Imp / Innovate

16. Descriptive Summary: Colorado Exploration / Installation / Initial & Full Imp / Innovate

17. Descriptive Summary: Florida Exploration/ Installation/ Initial Imp / Full Imp / Innovate

18. Descriptive Summary: Maryland Exploration / Installation / Initial Imp / Full Imp / Innovate

19. Descriptive Summary: Illinois Exploration / Installation / Initial Imp /Full Imp & Innovate

20. Lessons Learned Multiple approaches to achieving scaled implementation Colorado: Started with Leadership Team Illinois: Started with Leadership Advocates and built team only after implementation expanded. Missouri: Strong initial demonstrations led to strong state support All states began with small “demonstrations ” that documented the feasibility and impact of SWPBIS. Only when states reached 100-200 demonstrations did scaling occur. Four core features needed for scaling: Administrative Leadership / Support/ Funding Technical capacity (Local training, coaching, evaluation and behavioral expertise) Local Demonstrations of feasibility and impact (100-200) Evaluation data system (to support continuous improvement) Essential role of Data : Fidelity data AND Outcome data

21. Lessons Learned Scaling is NOT linear Sustained scaling requires continuous regeneration Both Bottom Up and Top Down Threats to Scaling : o Competing initiatives o The seductive lure of the “new idea” o Leadership turnover o Legislative mandates o Fiscal constraint Regular Dissemination of Fidelity and Impact data is the best “protective factor” for threats to scaling

22. Lessons Learned Scaling requires planned efficiency o The unit cost of implementation must decrease as the number of adoptions increases. Shift from external trainers to within state/district trainers Use local demonstrations as exemplars Increased coaching capacity can decrease investment in training Improved “selection” of personnel decreases turnover and development costs Use existing professional development and evaluation resources differently Basic Message : The implementation practices that are needed to establish initial exemplars may be different from the practices used to establish large scale adoption. o Jennifer Coffey, 2008

23. Sustainability Most educational innovations do not endure beyond 9 months The likelihood of sustaining is NOT related to effectiveness Achieving Sustainability is essential for cost effectiveness and scaling up to levels of social importance. The variables that affect sustaining implementation are not necessarily the variables that affect initial adoption.

24. (Latham, 1988)

25. Research on PBIS Sustainability Coffey & Horner, 2012 Exceptional Children, 78, 407-422.

26. Sample: 285 schools with SET scores -------------------------------------------------------- What predicted INITIAL Adopt of PBIS What predicted SUSTAINED use of PBIS A PBIS Sustainability Study (Coffey & Horner, 2012)

27. Never met SET (<80%) Met SET (≥80%) Implementers vs. Non-implementers SET SubscaleSET Overall Expectations Defined Reward System Responding to Violations Management (team and admin) District-Level Support Monitoring and Decision-Making Expectations Taught

28. Met and lost SET in 5 years Sustained SET for ≥5 years Sustainers vs. Non- sustainers SET SubscaleSET Overall Expectations Defined Responding to Violations District-Level Support Monitoring and Decision-Making Expectations Taught Reward System Management (team and admin)

29. Perceived Importance of Contextual Features for Sustainability of PBIS McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of School-wide Positive Behavior Support. Journal of Positive behavior Interventions, 16, 29-41.

30. 1.School administrators actively support PBIS 2.School administrators describes PBIS as a top priority for the school 3.A school administrator regularly attends and participates in PBIS team meetings 4.The PBIS school team is well organized and operates efficiently 5.The school administrators ensure that the PBIS team has regularly scheduled time to meet Most Important Features for Sustainability

31. Recent studies on sustainability of PBS o Perceptions of critical features for sustainability (McIntosh, Predy, Hume, Turri, & Mathews, in press) o Factors predicting sustainability (McIntosh et al., in press) o Critical features of PBS systems (Mathews, McIntosh, Frank, & May, under review) Best predictor of sustained implementation o Collection, Reporting and Use of Data Research on Sustainability

32. Building Capacity State District School Classroom Traditional Behavior/ Classroom Management Initial PBIS Current PBIS SISEP and Scaling UP

33. Districts Coherent District Policy o Social behavior is a priority in district improvement plan o District commitment to selecting practices that are evidence-based o District process for aligning multiple initiatives. Evaluation Capacity o Data systems that inform decision-making and provide policy feedback ** Fidelity and Impact Recruitment, Hiring, Evaluation o “Preference will be given to individuals with knowledge and experience in implementation of multi-tiered academic and behavior supports.” WHAT Interventions HOW Cycles Team Based Implementation WHO Teams

34. Districts Annual Faculty/Staff Orientation o Defines PBIS as a priority o Defines what to expect in a school using PBIS. o 30-60 min of annual orientation Professional Development (Training) o PD is always tied to core improvement goals o PD typically involves distributed training (multiple events) o PD is always linked to on-site coaching. o PD is always linked to fidelity measure Coaching o Prompt learned skills o Build fluency through opportunities to practice o Performance feedback to build precision o Adaptation to fit local context HOW Drivers

35. Reporting Out What is our district doing well? What are areas that would be most helpful to improve? What surprise or “insight” did the process provide?

36. Districts Annual Faculty/Staff Evaluations o Evaluations include assessment of “implementation of multi-tiered systems of academic and behavior support.” Build Technical Capacity to Implement Tier II and Tier III supports ----------------------------------------------------------- Assessing District Capacity o The District Capacity Assessment (DCA)

37. Leadership Team

38. Report to Board

39. Team 1.1 Team Authority: Tier I planning team exists, and meets at least monthly to use implementation fidelity and student outcome data to manage universal, school-wide support systems.  School organizational chart  Tier I systems planning team meeting minutes  Fidelity tool(s) 0 = No team identified to manage and monitor Tier I supports 1 = PBIS Team in place but does review data regularly 2 = PBIS Team exists with the authority to implement at least Tier I PBIS, and has at least monthly meetings with access to fidelity and student outcome data to guide action planning FeatureData Sources Scoring Criteria 0 = Not implemented 1 = Partially implemented 2 = Fully implemented

40. Selecting Practices

41. Implementation 1.4 Behavioral Expectations: School has five or fewer positively stated behavioral expectations for student and staff behaviors (i.e., teaching matrix) defined and in place.  Staff handbook  Student handbook  Walk through reports 0 = No clearly stated behavioral expectations have been identified and posted 1 = Behavioral expectations identified but no matrix 2 = Five or fewer behavioral expectations exist that are positive, posted, and identified for specific settings FeatureData Sources Scoring Criteria 0 = Not implemented 1 = Partially implemented 2 = Fully implemented

42. Subscale and ItemsActionWhoWhenNext Update 1.Commitment and Leadership 1.Systems Alignment 1.Action Planning 1.Performance Feedback 1.Selection 1.Training 1.Coaching 1.Decision Support System 1.Facilitative Administration 1.Systems Intervention

46. Next Steps State o Use SCA to define state capacity o Invest in state leadership team o Build scaling plan with both “bottom up” and “top down” strategies Fuchs et al., 2014 District o Use DCA to define district capacity o Use District Self-Assessment for targeted content areas. o Invest in Leadership Team o Build capacity to Train, Coach, Evaluate, Sustain o Collect school “fidelity” data regularly (TFI) o Collect student outcome data regularly Tier IODR Tier IICICO points Tier III % students “Progressing” ISIS