Presentation on theme: "Enhancing Sustainability at the District and State Levels"— Presentation transcript:
1 Enhancing Sustainability at the District and State Levels Kent McIntosh, University of OregonBridget Drobac, Bethel School DistrictEric Kloos, Minnesota Dept. of Ed.2014 PBIS Implementers ForumHandouts:
2 Who are we? Who are we? Who are you? Kent Bridget Eric Levels? School, district, region, state, nation?PBIS implementation experience?How many have attended previous sessions in this strand?Who are you? – school personnel, school admin, district admin, district support personnel, state admin
3 Support for these projects: IES: NCSER (R324A120278)OSEP: TA Center on PBIS (H326S03002)Social Sciences and Humanities Council of Canada (SRG F )Hampton Endowment Fund (J )
4 Thanks and Acknowledgments Participants in these studiesState NetworksJerry Bloom, Susan Barrett and PBIS MarylandCristy Clouse, Barbara Kelley and CalTACEric Kloos and Minnesota DOEMike Lombardo, Celeste Rossetto Dickey, and Placer COENanci Johnson and MO SW-PBSJustyn Poulos, Wisconsin PBISCayce McCamish, NC DOECo-authors
5 Handouts: http://www.pbis.org Session OverviewResearch on strategies to support implementation and sustainability of PBISKentSystems for sustainability at the district levelBridget, Bethel School District (OR)Systems for sustainability at the state levelEric, Minnesota Department of EducationHandouts:
6 Perceived Importance of Contextual Features for Sustainability of PBIS McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of School-wide Positive Behavior Support. Journal of Positive Behavior Interventions, 16,
7 Perceived Factors Related to Sustainability of PBIS Sample: 257 schools from 14 US states49% Elementary16% Middle5% High SchoolAverage implementation: 6 years (1 to 15)Open-ended question:“What is the most important enabler of sustainability of PBIS?”
8 Enablers of Sustainability How would you answer?
9 District and state systems are the keel in the school’s boat (McIntosh & Goodman, in press)
10 Most Important Single Perceived Factor in Sustainability? School Administrator SupportCan districts play a role in increasing school administrator support?
11 Sustaining PBIS through Administrator Turnover (Strickland-Cohen, McIntosh, & Horner, 2014) School TeamMaintain the PBIS handbookDocument support among staff and stakeholdersCollect and share outcomes dataMeet with the new administratorDistrict TeamBuild PBIS into written policyBuild PBIS competencies into hiring criteriaDevelop district coaching capacity
12 How Do Principals Go From Skeptics to PBIS Champions? McIntosh, K., Kelm, J. L., & Canizal Delabra, A. (2014). In search of how principals change: Critical incidents in enhancing administrator support for school-wide prevention. Manuscript submitted for publication.
13 Research into Enhancing Principal Support Qualitative interviews with 10 principals initially opposed or lukewarm to PBIS but now championsInterview questions:What helped your active support for PBIS?What hindered your active support for PBIS?What would have made you support PBIS from the onset?
14 Strategies for Enhancing Principal Support District Training and SupportProvide “Principal PBIS Academies” for new administratorsBasics of PBISRole of administratorsProvide coaching to schoolsDistrict NetworkingArrange informal conversations with other principals supportive of PBISArrange site visits at nearby PBIS schoolsAt the SchoolHelp school staff demonstrate support
15 What is the strongest predictor of PBIS sustainability? McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of School-wide Positive Behavior Support. Exceptional Children, 79,
16 Results: Predictive Model Model fit indices acceptable (except χ2)χ2 (731) = , p < .001, CFI = .96, TLI = .96, RMSEA = .03R 2 = .45FactorsPriority (B = .14, SE = .39, p > .05)Team Use of Data (B = .61, SE = .24, p < .05)District Priority (B = -1.14, SE = .66, p > .05)Capacity Building (B = .98, SE = .43, p < .05)45% of the variance explained
17 5.38** .07 .47 -.34 .41 School Priority Sustained PBIS Fidelity Team Use of Data8.47888888Priority (20 items, reliability = .94)Staff support, administrator support, perceived effectiveness, perceived efficiency, integration into new initiativesImplementation (11 items, reliability = .94)School team/staff skill, functioning, regular meetings, data collection, use of data for decision making, presenting data to staff and community88District Priority8-.34888Sustained PBIS Fidelity8Capacity Building88.418
18 Four Factors School Priority (20 items) Team Use of Data (11 items) Administrator support, staff support, perceived effectiveness, perceived efficiency, integration into new initiativesTeam Use of Data (11 items)School team/staff skill, functioning, regular meetings, data collection, use of data for decision making, presenting data to staff and communityDistrict Priority (5 items)District support, state support, funding, district policy, promoted to external organizationsCapacity Building (3 items)Access to district coaching, yearly professional development, connection to a community of practiceBelow the close fit hypothesis – close yet failing model13.522.502 items loaded on both factors (effective for a large proportion of students; all school personnel have a basic understanding of PBIS) largest residual correlation -.25nested
19 TakeawaysSchool teams can benefit from training in running meetings and using dataDistrict coaching, professional development, and connection to a community of practice were effective district supportsNo significant independent contribution of active support, general funding, policyNote that they are correlated at .78 and .58Simplistic notion to stop doing non-significant factors
20 What predicts sustained PBIS implementation at 3 and 5 years after training? McIntosh, K., Mercer, S. H., Nese, R. N. T., Strickland-Cohen, M. K., & Hoselton, R. (2014). Predictors of sustained implementation of School-wide PBIS at 3 and 5 years after initial implementation. Manuscript in preparation.Sneak peek into what we’re up to now…
21 Sample1242 schools submitting fidelity data in PBIS Assessment (starting from to 12-13)530 districts29 statesGrade levels:70% elementary schools21% middle schools9% high schoolsPerceived impact
23 What were successful states (>50% at criterion) doing? State PBIS leadership teamsState-level trainers, trainings, and training curriculaState recognition systems (for schools with strong implementation and/or outcomes)Regular use of the SWPBIS Implementer’s Blueprint (Sugai et al., 2010) to assess and inform implementation
24 Bethel School District PBIS Team Approach Be the guiding/driving force for Bethel’s Systems Approach to improving behavior and achievement for all the students in the district.NoneAdapted from Horner, Sugai and Bethel Staff
25 Bethel School District Demographics Bethel is a school district in Eugene, Oregon that is home to approximately 5,700 students.60% of students qualify for free & reduced lunches.17% of students identified as having a special education eligibility.5% of students are ELDFive Elementary SchoolsTwo K-8 SchoolsTwo traditional middle schoolsOne alternative high school (grades 10-12)One traditional high schoolNone
26 Bethel Behavior Data : 10 year span I’m proud to share some of Bethel’s data accrued over time. This graph shows overall improvement in behavior data (referrals per 100 students) over the last ten years of PBIS implementation. While there are a variety of ways to implement effective PBIS, I will share more about Bethel’s process
27 Bethel PBS StructureMonthly District PBS Leadership Team reviews data, process, plans training and provides support for successful implementation. This team made up of a variety of building and district level staff help guide the direction for District PBIS.School Teams drive a building’s PBIS program. Their focus is to maintain the integrity of the school-wide PBIS systems, look at classroom PBIS and the individual student PBIS system (Tiers 1-3).Data-based decision making plays a big role. Schools use SWIS-School-wide Information System to make timely adjustments or preventative interventions on a monthly basis or more often with Individual student system.Of course, keeping everyone trained in the model is key to successful implementation. As new PBIS changes come forward Bethel has a high level of training involvement to ensure student success.Durable, and adaptable school-wide PBIS in a school requires systemic support that extends beyond an individual school. It is important to organize multiple schools (e.g., cluster, complex, district, county, state) so that a common vision, language, and experience are established. This approach allows districts and states to improve the efficiency of resource use, implementation efforts, and organizational management. An expanded infrastructure also enhances the district and state level support (e.g., policy, resources, competence) and provides a supportive context for implementation at the local level.
28 Practices for Student Success: Linking of Academic and Behavioral Interventions Social BehaviorIndividualIndividually Design InstructionIndividualBehavior Support Plan (BSP)Safety Plan1-5%1-5%Targeted: Some Students (at-risk)Core PlusPre-teach, Re-teachSupplemental ProgramsAlter Group Size5-10%5-10%Targeted: Some Students (at-risk)Advance CICOSocial SkillsLunch BuddiesBoy/Girls GroupsCheck-in, Check-out (CICO)This version of the triangle demonstrates how we work to tie in behavior & academics. These efforts come from Bethel’s district policy and our Tiered interventions managed by our school teams.Universal Screening:All StudentsCore Program80-90%Universal Screening:All StudentsSchoolwide expectationstaught explicitlyreinforced frequently80-90%Adapted from: Horner & Sugai28
29 What is involved in being the “guiding force” behind Bethel’s Systems Approach to improving behavior and achievement for all the students in the district?District Coordination of PBIS Systems for the past 13 yearsPBIS in hiring practicesPBIS grades K-8 across the districtUniformed behavior referrals and minorsPBIS incorporation with transportationPBIS District-wide eases transitionsDistrict Coordination of PBIS systems present and futureDistrict “On Track” reportsPBIS practices and SWIS at the high schoolUniformity in behavior form for transportationDistrict PBIS team blog.Coordination past-PBIS has become district policy and plays a role in hiring for licensed staffPBIS is implemented across the district in grades K-8 and has recently been implemented at the high schoolPBIS lead team action prompted consistent expectations and uniformed referrals and minor K-8PBIS incorporation on Bethel and First Student TransportationPBIS District-wide and these efforts ease transition for students and staff with movement between buildings or for families with students in multiple buildingsCoordination futureDistrict “On Track” reports reviewing academics, attendance and behavior in progression toward graduation. Look at whole studentImplementation of PBIS practices and use of Student Wide Information Systems data collection at the high school levelInclude transportation in use of uniform behavior forms for data continuity and clarityDevelopment of PBIS team blog. Create a forum for building level teams to share information and successes.
30 Bethel Triangle DataComparison of behavior data with PBIS implementation in Elementary 2000/2001 compared to
31 Bethel Triangle DataComparison of behavior data with PBIS implementation in Middle 2003/2004 compared to
32 Comparison of behavior data with PBIS implementation K-8 Bethel Triangle DataComparison of behavior data with PBIS implementation K-8to presentFrom our second year of full implementation, Bethel School District’s behavior data has improved in all Tiers, but most notably, Tier I (Gains of 5-7%) The gains indicated are across grades K-8 and over time from
33 Coordination of PBIS : the team approach Ensuring coordination and implementation is a District Lead Team member responsibilityThe DLT includes:PBIS CoordinatorBehavior SpecialistBuilding AdministratorsDirector of Special ServicesSchool Counselor/PBIS FacilitatorsDistrict Equity CoordinatorFirst Student transportation representativeBuilding administrators from all levels- K-5, K-8, Middle and now HS
34 District Leadership Team Purpose To support, improve and sustain PBIS in the Bethel School District. The DLT’s toolbox includes:District-wide data Stable funding to support schoolsPBIS as a district focus Coordination of PBIS across schoolsProfessional development Building team evaluation measuresCapacity for all students Coordinate systems (EBISS, On-Track)Effective practices Partner (U of O, ORI, First Student)Analyze district-wide data and maintain PBIS as a district focus. Provide stable funding to support schools including determination of need and provision of resources for professional developmentEnsure coordination of PBIS across schools and levels and build capacity to meet the needs of all students. Measures to ensure coordination across systems including (EBISS, On-Track)Review research for effective practices and partner with organizations (U of O, First Student, ORI, etc.)Review team and building evaluation measures to insure fidelity and effective support
35 Schools Implementing PBIS in Bethel This chart indicates the level of implementation district wide. Last year, we added the WHS to the schools implementing PBIS in Bethel. 10 of Bethel’s 11 schools currently implement and sustain PBIS practice and use SWIS data for decision making
36 Review of Team Assessment Data Buildings complete annual assessments. Recent assessments include:Team Implementation Checklist (TIC)Benchmarks of Quality (BOQ)Tiered Fidelity Inventory (TFI)Results are indicator of possible growth for building teamResults provide information to the PBIS District Lead TeamDistrict and building level teams review the results of assessment surveys. Building teams use the survey results to develop actions plans for the next year, as well as, long-term goals.The District Lead team reviews survey data to maintain district focus on PBIS practices. DLT provides training and support to buildings based on building fidelity, commitment, implementation and need.
37 Intensive Interventions Specialized Individualized Systems for Students with High-Risk Behavior~5%Targeted InterventionsSpecialized GroupSystems for Students with At-Risk BehaviorUniversal InterventionsSchool-/Classroom-Wide Systems forAll Students,Staff, & Settings~15%Many people place all their emphasis on interventions—what will we do, what kind of targeted interventions do we need?These factors are important BUT we have learned from SWPBS that it is every bit as important to focus on key features to guide intervention implementation. Important aspects can be contextual fit for student population and staff (“buy in”), feasibility of intervention and fidelity~80% of Students
38 BETHEL PBS IMPLEMENTATION PROCESS Administrative SupportDistrict PBIS TeamSchool PBS TeamRepresentative StaffBETHEL PBS IMPLEMENTATION PROCESSAgreementsEstablish PBS GoalsDevelop Data-basedAction PlanIn general, the implementation of a school-wide PBIS approach at the school level is built around five main implementation steps. Teams use data and information gleaned from surveys including the TIC and BOQ to draft action plan. Action plans are submitted to the district level team. Action plans guide the building team efforts throughout the year.EvaluationImplementation
39 Snapshot of District Team Survey Data 2013-2014 A snapshot of the Team Implementation Checklist shows Bethel Team scores on indicators for team sustainability. At all levels K-8, teams identify over 90% commitment from building administration and staff. Teams show at least 80% representation of all stake-holders including administration, certified/classified staff, students and families. Finally, Teams relate 100% coordination between all three Tiers of Intervention. This number translates to clear communication and transition between PBIS and IPBS/Data-teams.
40 Recent Barriers and Challenges Counseling FTE reduced by .5Administrative changes/reductionStaff changes/new staffMaintaining contextual fit within district-wide systemCompeting fociPartial implementation or “drift” from best practiceReduction of counseling created a challenge in that many district counselors participated on the building PBIS team (Tier I) and facilitated IPBS/Data-teams (Tiers II & III).In conjunction with reduction of administration, changes in staff can disrupt continuity and require more intensive training.Changes in curriculum, testing and information sharing with families has provided district and building level competition for attention.Most important nugget of Bethel’s presentation: Awareness of potential for drift from best practice. Avoid complacency (this is how we’ve always done it…does it still fit?)
41 Provision of support for Bethel’s District PBIS Attend building level PBIS meetingsMonitor transitions between PBIS Tier I, II, III teamsMeet as a team with building level Tier I & Tier II/III facilitators each trimesterReview building level dataProvide district, local, state and national training informationMaintain connection and engagement with Bethel transportationAttend building level PBIS meetings for coordination with District Lead TeamMonitor transitions between PBIS Tier I, II, III teams to ensure clear communication between building PBIS team and IPBS/Data TeamsMeet as a team with building level Tier I & Tier II/III facilitators each trimester to maintain momentum, review outcomes, celebrate success and respond to needsReview building level data to note gains and identify areas that may require more intensive focus or interventionProvide district training, information about and access to(funding when possible) local, state and national trainings/conferences. Bethel, thus far, sends at least one representative per building to the state PBIS conference.Visit Bethel transportation offices and ride different bus routes, when possible and provide training and ongoing support
42 From power-point to practice District-wide data indicates increase in percent of ODRs for physical aggression.DLT response: PBIS coach meets with building level teams to review data and increase school wide interventions to address percentage changeRevitalize focus on safety and Stop/Walk/Talk programsIncrease pro-social lessonsUse of peer mediation programsProvide retraining, as neededWhile I am happy to identify the many strengths of Bethel’s PBIS program, I also want to be realistic about some of the challenges Bethel faces. In reviewing year end SWIS data, we noted an increase in the category of physical aggressions across the district. The DLT developed a plan to address this concerning trendTrainings implemented thus far include: Focus on safety, Second Steps, Steps to Respect, Peer Mediation (on-going) and Active Supervision (for adults)
43 District Level Implementation: What to do to be effective? Investment and Commitment to School-Wide PreventionProvision of District Training for PersonnelUse of Data SystemsTeam-Based Decision Making ModelInvestment and Commitment to School-Wide Prevention including coordination of positive intervention and supportDistrict Training for Personnel-follow up and on-site capacity building and district and school based coachingData Systems provide access to effective, clear and “user friendly” data systems. Data is reviewed at building and district level. Data is used to track Tier I, II & III for intervention outcomes and trends.Team-Based Decision Making Model: teams engage in regularly scheduled meetings with representative teams (stake-holders). Data is used as a tool in decision making
44 Bridget Drobac- PBIS Coordinator Contact InformationBridget Drobac- PBIS CoordinatorBethel School District
45 Enhancing Sustainability at the State Level Eric KloosMinnesota Department of Educationeducation.state.mn.us
46 Building Capacity of Effective Implementation of SW-PBIS Team-based training9 training days over two yearsDistributed, team-based implementation of PBISIntended to build capacity, skills, competency and beliefs to sustain implementation beyond initial trainingeducation.state.mn.us
47 Creating Implementation Informed Expectations at a School Level In Minnesota, baselines are rising (average baseline SET = 69), but there are still predictable differences between schools starting training and sustaining schools (average SET = 90, BoQ = 84).What features are similar between baseline schools and sustaining schools?Administrator is an active PBIS team member (96% baseline schools/97% sustaining schools)Administrator reports that team meetings occur (98%/98%)What features are different between baseline schools and sustaining schools?Documented system of teaching expectations (46%/83%)Teaching expectations has occurred this year (74%/94%)SW behavior program has been taught/reviewed with staff this year (78%/97%)Team provides discipline data summary to staff at least 3 times per year (50%/91%)90% of team members report that discipline data is used for decision-making (57%/97%)Why do we measure implementation across time in a school?Because it varies!education.state.mn.us
48 2014 School SET Profile “A Snapshot over Time” Illustrates normal implementation variation over time within in a MN school that has been implementing PBIS 7 yearseducation.state.mn.us
49 Creating Implementation Informed Expectations at a District-Level At a district-level, it is often a challenge to accurately track which schools:have been trained,are in training, andhave yet to participate in training.Differentiate outcome expectations for schools by what we know about their implementation.Get the right information to the right people at the right time to inform district decisions.Support patience and focus to get to results.Why do we measure implementation across a District?- Because it varies!education.state.mn.us
50 2014 District SET Profile “A Snapshot in Time” Illustrates normal implementation variation over time within in a MN district that has been implementing PBIS 6 yearseducation.state.mn.us
51 Example of a District Data Dashboard: Effort, Fidelity and Outcome Data education.state.mn.us
52 District Capacity Assessment (DCA) education.state.mn.us
53 District Calendar for Implementation education.state.mn.us
54 Implementation Informed Expectations for States When do we expect to see state-level outcome changes?How many schools and districts need to be implementing?At what standard?As many variables change, can we continue to produce good outcomesChange in team membersNew trainersNew coachesNew evaluatorsWhy do we measure implementation across the state?It varies across schools, districts, regions and over timeeducation.state.mn.us
55 Cohort 8 SET Results Fall 12-Spring 14 education.state.mn.us
56 Sharing Data and Outcomes: Disciplinary Reductions for District and State
57 Closing ThoughtsWe are learning a lot by studying schools and districts that have been doing PBIS well over time.Tools and processes that synthesize data for school and district teams support sustained implementation.Watch for larger data sets and outcomes to change when at least 25% are effectively implementing.education.state.mn.us
58 The Minnesota Department of Education: The Active Implementation Hub: ResourcesMinnesota PBIS:The Minnesota Department of Education:The Active Implementation Hub:education.state.mn.us
60 Selected ReferencesMcIntosh, K., & Goodman, S. (in press). Multi-tiered systems of support: Integrating academic RTI and school-wide PBIS. New York: Guilford Press. McIntosh, K., Kelm, J. L., & Canizal Delabra, A. (2014). In search of how principals change: Critical incidents in enhancing administrator support for school-wide prevention. Manuscript submitted for publication. McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of school-wide positive behaviour support. Exceptional Children, 79, McIntosh, K., Mercer, S. H., Nese, R. N. T., Strickland-Cohen, M. K., & Hoselton, R. (2014). Predictors of sustained implementation of School-wide PBIS at 3 and 5 years after initial implementation. Manuscript in preparation. McIntosh, K., Predy, L. K., Upreti, G., Hume, A. E., Turri, M. G., & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of school-wide positive behaviour support. Journal of Positive Behavior Interventions, 16, Strickland-Cohen, M. K., McIntosh, K., & Horner, R. H. (2014). Sustaining effective practices in the face of principal turnover. Teaching Exceptional Children, 46(3), Sugai, G., Horner, R. H., Algozzine, R., Barrett, S., Lewis, T., Anderson, C., Simonsen, B. (2010). School-wide positive behavior support: Implementation blueprint and self-assessment (2nd ed.). Eugene, OR: University of Oregon. Available at