School-wide PBIS: Using Data for Effective Coaching Rob Horner University of Oregon

Goals Address questions you face as a PBIS Coach Building Decision-Systems  Basic Foundations  Access to core data sources  Rubric for using data to make decisions  On-going review and improvement using data Data needed for effective SWPBIS coaching  Using Fidelity data to assist teams in SWPBIS implementation Team problem solving processes  Establishing the Foundations  Using data for action planning  Using data for implementation assessment and adaptation

Using Data for Decision-making What decisions are needed? Who needs what information at what time in what form to make effective decisions?

Action Planning Improving Decision-Making Problem Solution From To Problem Solving Information Solution

School-Team Decisions Are we implementing SWPBIS?  Are we implementing at all three tiers?  What would be the next smallest change that would make the biggest effect? If we are implementing, are the procedures benefiting students?  Are there problems?  Define problem with precision:  What, Where, When, Who, Why  What proven actions (interventions, practices, packages) would address the problem yet fit with current strengths?  Are selected actions being implemented and effective?  How do they need to be adapted?  Are selection actions producing desired outcomes?

Fidelity of Implementation Measuring fidelity is more than accountability.  Measuring fidelity and regular action planning are key procedures for “getting it right”  Help teams use their strengths to find the path that best helps them achieve the SWPBIS core features. Be careful about adoption too many strategies/ practices.  Do a small number of things well.

~80% of Students ~15% ~5% ESTABLISHING CONTINUUM of SWPBS SECONDARY PREVENTION Check in/out Targeted social skills instruction Peer-based supports Social skills club TERTIARY PREVENTION Function-based support Wraparound Person-centered planning PRIMARY PREVENTION Teach SW expectations Proactive SW discipline Positive reinforcement Effective instruction Parent engagement School-wide Bully Prevention SECONDARY PREVENTION TERTIARY PREVENTION PRIMARY PREVENTION

Level of SupportResearch Measures (2-4 hours) Annual Self- Assessment Measures (45-60 min) Progress Monitoring Measures (15 min) Universal (Tier I)School-wide Evaluation Tool (SET) Self-Assessment Survey (SAS) Benchmarks of Quality (BoQ) Team Implementation Checklist (TIC) Secondary and Tertiary (Tier II, Tier III) Individual Student School-wide Evaluation Tool (ISSET) Benchmarks of Advanced Tiers (BAT) Measure of Advanced Tiers Tool (MATT) Overall Implementation Implementation Phases Inventory (IPI) Phases of Implementation (POI) Phases of Implementation Checklist (PIC) SWPBIS Measures of Fidelity ( See PBIS Evaluation Blueprint; Measures available at How Knowledgeable/Comfortable are you with ( 1= low; 5 = high): School-wide Evaluation ToolSET Self-Assessment Survey SAS Team Implementation Checklist TIC Benchmark of Quality BoQ Individual Student SETISSET Benchmark of Advanced TiersBAT Measure of Advanced Tiers ToolMATT

Schedule of SWPBIS Measures (For shaded measures, select one) LevelMeasurePreYear 1Year 2Year 3 SFWSSFWSSFWS Universal SWPBS Progress Monitoring: TIC XXX XXX Annual Self- Assessment: BoQ, X X X Research Measure: SET X X X Self- Assessment Survey: SAS X X X X Secondary/Tertiary Progress Monitoring: TBA XXX XXX Annual Self- Assessment: BAT X X Research Measure: I- SSET X X

Match Coaching to Stage of Implementation

School Level Implementation Takes Time: 2 – 4 Years EXPLORATION INSTALLATION INITIAL IMPLEMENTATION FULL IMPLEMENTATION Sages of Implementation How long at a district level? How long at a state level? How long at a national level? Fixsen & Blase 2012

Coaching Across Stages of Implementation Phase of Implementation Coaching ChallengeMeasures Exploration  Knowledge of SWPBIS  Self-Assessment  Readiness for adoption  Commitment Self-Assessment Survey Team Implementation Checklist Benchmarks of Quality SET Installation  Readiness for adoption  Team Implementation Checklist Initial Implementation  Fidelity Assessment  Action Planning  Adaptation to Local Context/Strengths/ Culture Team Implementation Checklist Full Implementation  Sustain SWPBIS Implementation  Expand to Tier II, and Tier III Benchmark of Quality Measure of Advanced Tiers Benchmark of Advanced Tiers

Coaching for Implementation Are we implementing SWPBIS?  Team Implementation Checklist (15 min)  Benchmark of Quality (45-60 min)  School-wide Evaluation Tool (2-4 hours) How Often Should Data be Collected?  Initial implementation (every 3 rd or 4 th meeting)  On-going (annually) Total Score  Are we improving? Sub-scale score  What is working where do we focus next? Item score  Action planning

Elementary Middle

Team Checklist: Subscale Scores Percentage of Total Points

Your Turn: What should team focus on ?

Time 1 versus Time 2

Demonstration School Exemplar NCES ID: Zenith, Winnemac Demonstration District NCES ID : School YearNumber of Responses Date CollectedActionWho/When /30/2009 FeatureScore (0, 1, 2) Establish Commitment 1. Administrator's Support & Active Involvement.1 2. Faculty/Staff Support.1 Establish & Maintain Team 3. Team Established (Representative).1 4. Team has regular meeting schedule, effective operating procedures Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support. 0 Conduct Self-Assessment 6. Team completes the Team Implementation Checklist (TIC) Team summarizes existing school discipline data.1 8. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus). 0

Demonstration School Exemplar NCES ID: Zenith, Winnemac Demonstration District NCES ID : School YearNumber of Responses Date CollectedActionWho/ When /15/2011 FeatureScore (0, 1, 2) Establish Commitment 1. Administrator's Support & Active Involvement.2 2. Faculty/Staff Support.2 Establish & Maintain Team 3. Team Established (Representative).1 4. Team has regular meeting schedule, effective operating procedures Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support. 0 Conduct Self-Assessment 6. Team completes the Team Implementation Checklist (TIC) Team summarizes existing school discipline data.2 8. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus). 2

Using Student Impact Data Universal Screening Progress Monitoring Team Meetings and Use of Data for Problem Solving

Universal Screening and Progress Monitoring. Data SourcesDecisions Universal Screening Behavior : Oct, Feb Reading: Sept, Nov, Mar SWPBIS Fidelity Measures Teacher report SSBD/ SSRS/ SSIS Office Discipline Referrals Oral Reading Fluency (etc) Are Tier I supports in place? What students need more intensive support? Initiate early intervention Progress Monitoring Fidelity: Match to level of intensity of support Impact: Match to level of intensity of support CICO/ ISIS fidelity measures Office Discipline Referrals Check-in Check-out points ISIS intensive support Oral Reading Fluency Phonemic Segmentation Comprehension Are Tier II and Tier III supports implemented as planned? Are supports effective? Are more intensive/ individualized supports needed?

~80% of Students ~15% ~5% 0-1 office discipline referral 6+ office discipline referrals 2-5 office discipline referrals Using office discipline referrals as a metric for universal screening of student social behavior Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual. 32

Using the Referrals by Student report as a Universal Screening Tool 33

Cumulative Mean ODRs Cumulative Mean ODRs Per Month for 325+ Elementary Schools Jennifer Frank, Kent McIntosh, Seth May

Helping Teams Use Data at All Three Tiers Tier IOffice discipline referrals

~80% of Students ~15% ~5% ESTABLISHING a CONTINUUM of Data Options within SWPBS SECONDARY PREVENTION Fidelity: BAT, MATT * Student Outcomes: CICO-SWIS ODRs TERTIARY PREVENTION Fidelity: ISSET, BAT, MATT ISIS-SWIS Student Outcomes: ISIS-SWIS PRIMARY PREVENTION Fidelity: TIC, BoQ, SET Student Outcomes: Office Discipline Referrals SECONDARY PREVENTION TERTIARY PREVENTION PRIMARY PREVENTION

Coaching Tier III Data Use Needs ◦ System to assist coordination of individualized Team ◦ System for managing (a) assessments, (b) plan development/modification, and (c) team meeting minutes ◦ System for building data collection tailored to individualized plan  Collect Fidelity Data: Are we doing plan?  Collect Impact Data: Is plan benefiting student? ◦ System for on-going collection, summarizing and reporting of data for team decision-making.

Helping Teams use Data for Decision-Making Build team foundation  Roles, Agenda, Schedule Use data to define problems with precision Use data to build “Action Plans” that fit the local context and will work. Use data to assess if action plans are being implemented with effect

Problem-Solving Meeting Foundations Structure of meetings lays foundation for efficiency & effectiveness

Meeting Foundations Elements Four features of effective meetings  Predictability  Participation  Accountability  Communication Define roles & responsibilities  Facilitator, Minute Taker, Data Analyst Use electronic meeting minutes format 43 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

A. Predictability 1.Defined roles, responsibilities and expectations for the meeting 2.Start & end on time, if meeting needs to be extended, get agreement from all members 3.Agenda is used to guide meeting topics 4.Data are reviewed in first 5 minutes of the meeting 5.Next meeting is scheduled B. Participation 5.75% of team members present & engaged in topic(s) 6.Decision makers are present when needed What makes a successful meeting?

C. Accountability 7. Facilitator, Minute Taker & Data Analyst come prepared for meeting & complete during the meeting responsibilities 8.System is used for monitoring progress of implemented solutions (review previous meeting minutes, goal setting) 9.System is used for documenting decisions 10. Efforts are making a difference in the lives of children/students. D. Communication 11.All regular team members (absent or present) get access to the meeting minutes within 24 hours of the meeting 12.Team member support to practice team meeting norms/agreements

Define roles for effective meetings Core roles ◦ Facilitator ◦ Minute taker ◦ Data analyst ◦ Active team member ◦ Administrator Backup for each role Can one person serve multiple roles? Are there other roles needed? Typically NOT the administrator 46 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

47

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual. 48

Who is Responsible? ActionPerson Responsible Reserve RoomFacilitator Recruit items for AgendaFacilitator Review data prior to the meetingData Analyst Reserve projector and computer for meeting Minute Taker Keep discussion focusedFacilitator Record Topics and Decisions on agenda/minutes Minute taker Ensure that problems are defined with precision Facilitator Ensure that solutions have action plansFacilitator Provide “drill down” data during discussion Data Analyst End on timeFacilitator Prepare minutes and send to all members Minute taker

Activity # 1 (7 min) Select ◦ Facilitator ◦ Data Analyst ◦ Minute Taker Back up for each 50 RolePrimaryBackup Facilitator Data Analyst Minute Taker Next role review date: Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

Organizing for an effective problem solving conversation Problem Solution Out of Time Use Data A key to collective problem solving is to provide a visual context that allows everyone to follow and contribute 51 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

Newton, J. S., Todd, A. W., Horner, R. H., Algozzine, B., & Algozzine, K. (2012) (Version 1.2)

Where in the Form would you place: 1.Planning for next PTA meeting? 2.There have been five fights on the playground in the past 3 weeks. 3.Update on CICO implementation 4.Increasing gang recruitment as an agenda topic for today. 5.Next meeting report on lunch-room status.

Activity Examine the Langley minutes: ◦ 1. What is one agenda item for next meeting? ◦ 2. Who will do what by when to get the video system working? ◦ 3. For the problem of “disrespectful behavior,” how will they know if they achieved their goal?

What needs to be documented? Meeting demographics – Date, time, who is present, who is absent – Agenda – Next meeting date/time/location/roles Administrative/ general Information/Planning items – Topic of discussion, decisions made, who will do what, by when Problem-Solving items – Problem statement, data used for problem solving, determined solutions, who will do what by when, goal, how/how often will progress toward goal be measured, how/how often will fidelity of implementation be measured 57 Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

Summary: Team-based Problem Solving Team Foundations  Roles  Minute form Defining Problems Building Solutions Action Planning/ Follow up / Adaptation