Presentation on theme: "Enhancing Sustainability at the District and State Levels Kent McIntosh, University of Oregon Bridget Drobac, Bethel School District Eric Kloos, Minnesota."— Presentation transcript:
Enhancing Sustainability at the District and State Levels Kent McIntosh, University of Oregon Bridget Drobac, Bethel School District Eric Kloos, Minnesota Dept. of Ed. 2014 PBIS Implementers Forum Handouts: http://pbis.org
Who are we? Kent Bridget Eric Who are you? Levels? School, district, region, state, nation? PBIS implementation experience? Who are we?
Support for these projects: IES: NCSER (R324A120278) OSEP: TA Center on PBIS (H326S03002) Social Sciences and Humanities Council of Canada (SRG F09-05052) Hampton Endowment Fund (J07-0038)
Participants in these studies State Networks Jerry Bloom, Susan Barrett and PBIS Maryland Cristy Clouse, Barbara Kelley and CalTAC Eric Kloos and Minnesota DOE Mike Lombardo, Celeste Rossetto Dickey, and Placer COE Nanci Johnson and MO SW-PBS Justyn Poulos, Wisconsin PBIS Cayce McCamish, NC DOE Co-authors Thanks and Acknowledgments
1. Research on strategies to support implementation and sustainability of PBIS Kent 2. Systems for sustainability at the district level Bridget, Bethel School District (OR) 3. Systems for sustainability at the state level Eric, Minnesota Department of Education Session Overview Handouts: http://www.pbis.orghttp://www.pbis.org
Perceived Importance of Contextual Features for Sustainability of PBIS McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of School-wide Positive Behavior Support. Journal of Positive Behavior Interventions, 16, 29-41.
Sample: 257 schools from 14 US states 49% Elementary 16% Middle 5% High School Average implementation: 6 years (1 to 15) Open-ended question: “What is the most important enabler of sustainability of PBIS?” Perceived Factors Related to Sustainability of PBIS
District and state systems are the keel in the school’s boat (McIntosh & Goodman, in press)
School Administrator Support Can districts play a role in increasing school administrator support? Most Important Single Perceived Factor in Sustainability?
School Team Maintain the PBIS handbook Document support among staff and stakeholders Collect and share outcomes data Meet with the new administrator District Team Build PBIS into written policy Build PBIS competencies into hiring criteria Develop district coaching capacity Sustaining PBIS through Administrator Turnover (Strickland-Cohen, McIntosh, & Horner, 2014)
How Do Principals Go From Skeptics to PBIS Champions? McIntosh, K., Kelm, J. L., & Canizal Delabra, A. (2014). In search of how principals change: Critical incidents in enhancing administrator support for school-wide prevention. Manuscript submitted for publication.
Qualitative interviews with 10 principals initially opposed or lukewarm to PBIS but now champions Interview questions: 1. What helped your active support for PBIS? 2. What hindered your active support for PBIS? 3. What would have made you support PBIS from the onset? Research into Enhancing Principal Support
District Training and Support Provide “Principal PBIS Academies” for new administrators Basics of PBIS Role of administrators Provide coaching to schools District Networking Arrange informal conversations with other principals supportive of PBIS Arrange site visits at nearby PBIS schools At the School Help school staff demonstrate support Strategies for Enhancing Principal Support
What is the strongest predictor of PBIS sustainability? McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of School-wide Positive Behavior Support. Exceptional Children, 79, 293-311.
Model fit indices acceptable (except χ 2 ) χ 2 (731) = 881.55, p <.001, CFI =.96, TLI =.96, RMSEA =.03 R 2 =.45 Factors Priority (B =.14, SE =.39, p >.05) Team Use of Data (B =.61, SE =.24, p <.05) District Priority (B = -1.14, SE =.66, p >.05) Capacity Building (B =.98, SE =.43, p <.05) Results: Predictive Model
Sustained PBIS Fidelity 5.38 **.07 Team Use of Data School Priority.47 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 District Priority Capacity Building Sustained PBIS Fidelity -.34.41 8 8 8 8 8 8 8 8
School Priority (20 items) Administrator support, staff support, perceived effectiveness, perceived efficiency, integration into new initiatives Team Use of Data (11 items) School team/staff skill, functioning, regular meetings, data collection, use of data for decision making, presenting data to staff and community District Priority (5 items) District support, state support, funding, district policy, promoted to external organizations Capacity Building (3 items) Access to district coaching, yearly professional development, connection to a community of practice Four Factors
School teams can benefit from training in running meetings and using data District coaching, professional development, and connection to a community of practice were effective district supports No significant independent contribution of active support, general funding, policy Takeaways
What predicts sustained PBIS implementation at 3 and 5 years after training? McIntosh, K., Mercer, S. H., Nese, R. N. T., Strickland-Cohen, M. K., & Hoselton, R. (2014). Predictors of sustained implementation of School-wide PBIS at 3 and 5 years after initial implementation. Manuscript in preparation.
1242 schools submitting fidelity data in PBIS Assessment (starting from 05-06 to 12-13) 530 districts 29 states Grade levels: 70% elementary schools 21% middle schools 9% high schools Sample
State PBIS leadership teams State-level trainers, trainings, and training curricula State recognition systems (for schools with strong implementation and/or outcomes) Regular use of the SWPBIS Implementer’s Blueprint (Sugai et al., 2010) to assess and inform implementation What were successful states (>50% at criterion) doing?
Bethel School District PBIS Team Approach Be the guiding/driving force for Bethel’s Systems Approach to improving behavior and achievement for all the students in the district. Adapted from Horner, Sugai and Bethel Staff
Bethel School District Demographics Bethel is a school district in Eugene, Oregon that is home to approximately 5,700 students. 60% of students qualify for free & reduced lunches. 17% of students identified as having a special education eligibility. 5% of students are ELD Five Elementary Schools Two K-8 Schools Two traditional middle schools One alternative high school (grades 10-12) One traditional high school
28 Academic Social Behavior 1-5% 5-10% 80-90% Individual Individually Design Instruction Individual Behavior Support Plan (BSP) Safety Plan Targeted: Some Students (at-risk) Core Plus Pre-teach, Re-teach Supplemental Programs Alter Group Size Targeted: Some Students (at-risk) Advance CICO Social Skills Lunch Buddies Boy/Girls Groups Check-in, Check-out (CICO) Universal Screening: All Students Core Program Universal Screening: All Students Schoolwide expectations taught explicitly reinforced frequently Practices for Student Success: Linking of Academic and Behavioral Interventions Adapted from: Horner & Sugai
What is involved in being the “guiding force” behind Bethel’s Systems Approach to improving behavior and achievement for all the students in the district? District Coordination of PBIS Systems for the past 13 years PBIS in hiring practices PBIS grades K-8 across the district Uniformed behavior referrals and minors PBIS incorporation with transportation PBIS District-wide eases transitions District Coordination of PBIS systems present and future District “On Track” reports PBIS practices and SWIS at the high school Uniformity in behavior form for transportation District PBIS team blog.
Bethel Triangle Data Comparison of behavior data with PBIS implementation in Elementary 2000/2001 compared to 2013-2014
Bethel Triangle D ata Comparison of behavior data with PBIS implementation in Middle 2003/2004 compared to 2013-2014
Bethel Triangle Data Comparison of behavior data with PBIS implementation K-8 2000-2001 to present 2013-2014
Coordination of PBIS : the team approach Ensuring coordination and implementation is a District Lead Team member responsibility The DLT includes: PBIS Coordinator Behavior Specialist Building Administrators Director of Special Services School Counselor/PBIS Facilitators District Equity Coordinator First Student transportation representative
District Leadership Team Purpose To support, improve and sustain PBIS in the Bethel School District. The DLT’s toolbox includes: District-wide dataStable funding to support schools PBIS as a district focusCoordination of PBIS across schools Professional developmentBuilding team evaluation measures Capacity for all studentsCoordinate systems (EBISS, On-Track) Effective practicesPartner (U of O, ORI, First Student)
Review of Team Assessment Data Buildings complete annual assessments. Recent assessments include: Team Implementation Checklist (TIC) Benchmarks of Quality (BOQ) Tiered Fidelity Inventory (TFI) Results are indicator of possible growth for building team Results provide information to the PBIS District Lead Team
Intensive Interventions Specialized Individualized Systems for Students with High-Risk Behavior Targeted Interventions Specialized Group Systems for Students with At-Risk Behavior Universal Interventions School-/Classroom- Wide Systems for All Students, Staff, & Settings ~80% of Students ~15% ~5%
Agreements Establish PBS Goals Administrative Support District PBIS Team School PBS Team Representative Staff Develop Data-based Action Plan ImplementationEvaluation BETHEL PBS IMPLEMENTATION PROCESS
Snapshot of District Team Survey Data 2013-2014
Recent Barriers and Challenges Counseling FTE reduced by.5 Administrative changes/reduction Staff changes/new staff Maintaining contextual fit within district-wide system Competing foci Partial implementation or “drift” from best practice
Provision of support for Bethel’s District PBIS Attend building level PBIS meetings Monitor transitions between PBIS Tier I, II, III teams Meet as a team with building level Tier I & Tier II/III facilitators each trimester Review building level data Provide district, local, state and national training information Maintain connection and engagement with Bethel transportation
From power-point to practice District-wide data indicates increase in percent of ODRs for physical aggression. DLT response: PBIS coach meets with building level teams to review data and increase school wide interventions to address percentage change Revitalize focus on safety and Stop/Walk/Talk programs Increase pro-social lessons Use of peer mediation programs Provide retraining, as needed
District Level Implementation: What to do to be effective? Investment and Commitment to School-Wide Prevention Provision of District Training for Personnel Use of Data Systems Team-Based Decision Making Model 43
Contact Information Bridget Drobac- PBIS Coordinator Bethel School District firstname.lastname@example.org
Enhancing Sustainability at the State Level Eric Kloos Minnesota Department of Education education.state.mn.us
Building Capacity of Effective Implementation of SW-PBIS Team-based training 9 training days over two years Distributed, team-based implementation of PBIS Intended to build capacity, skills, competency and beliefs to sustain implementation beyond initial training education.state.mn.us
Creating Implementation Informed Expectations at a School Level In Minnesota, baselines are rising (average baseline SET = 69), but there are still predictable differences between schools starting training and sustaining schools (average SET = 90, BoQ = 84). What features are similar between baseline schools and sustaining schools? Administrator is an active PBIS team member (96% baseline schools/97% sustaining schools) Administrator reports that team meetings occur (98%/98%) What features are different between baseline schools and sustaining schools? Documented system of teaching expectations (46%/83%) Teaching expectations has occurred this year (74%/94%) SW behavior program has been taught/reviewed with staff this year (78%/97%) Team provides discipline data summary to staff at least 3 times per year (50%/91%) 90% of team members report that discipline data is used for decision-making (57%/97%) Why do we measure implementation across time in a school? Because it varies! education.state.mn.us
2014 School SET Profile “A Snapshot over Time” education.state.mn.us
Creating Implementation Informed Expectations at a District-Level At a district-level, it is often a challenge to accurately track which schools: have been trained, are in training, and have yet to participate in training. Differentiate outcome expectations for schools by what we know about their implementation. Get the right information to the right people at the right time to inform district decisions. Support patience and focus to get to results. Why do we measure implementation across a District? - Because it varies! education.state.mn.us
2014 District SET Profile “A Snapshot in Time” education.state.mn.us
Example of a District Data Dashboard: Effort, Fidelity and Outcome Data education.state.mn.us
District Capacity Assessment (DCA) education.state.mn.us
District Calendar for Implementation education.state.mn.us
Implementation Informed Expectations for States When do we expect to see state-level outcome changes? How many schools and districts need to be implementing? At what standard? As many variables change, can we continue to produce good outcomes Change in team members New trainers New coaches New evaluators Why do we measure implementation across the state? It varies across schools, districts, regions and over time education.state.mn.us
Cohort 8 SET Results Fall 12-Spring 14 education.state.mn.us
Sharing Data and Outcomes: Disciplinary Reductions for District and State
Closing Thoughts We are learning a lot by studying schools and districts that have been doing PBIS well over time. Tools and processes that synthesize data for school and district teams support sustained implementation. Watch for larger data sets and outcomes to change when at least 25% are effectively implementing. education.state.mn.us
Minnesota PBIS: http://pbismn.org The Minnesota Department of Education : http://education.state.mn.us The Active Implementation Hub : http://implementation.fpg.unc.edu/about-the-ai-hub Eric.Kloos@state.mn.us Resources education.state.mn.us
McIntosh, K., & Goodman, S. (in press). Multi-tiered systems of support: Integrating academic RTI and school-wide PBIS. New York: Guilford Press. McIntosh, K., Kelm, J. L., & Canizal Delabra, A. (2014). In search of how principals change: Critical incidents in enhancing administrator support for school-wide prevention. Manuscript submitted for publication. McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of school-wide positive behaviour support. Exceptional Children, 79, 293-311. McIntosh, K., Mercer, S. H., Nese, R. N. T., Strickland-Cohen, M. K., & Hoselton, R. (2014). Predictors of sustained implementation of School-wide PBIS at 3 and 5 years after initial implementation. Manuscript in preparation. McIntosh, K., Predy, L. K., Upreti, G., Hume, A. E., Turri, M. G., & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of school-wide positive behaviour support. Journal of Positive Behavior Interventions, 16, 29-41. Strickland-Cohen, M. K., McIntosh, K., & Horner, R. H. (2014). Sustaining effective practices in the face of principal turnover. Teaching Exceptional Children, 46(3), 18-24. Sugai, G., Horner, R. H., Algozzine, R., Barrett, S., Lewis, T., Anderson, C.,... Simonsen, B. (2010). School-wide positive behavior support: Implementation blueprint and self-assessment (2nd ed.). Eugene, OR: University of Oregon. Available at http://pbis.org/resource/216.http://pbis.org/resource/216 Selected References