1 Positive Behavioral Interventions and Supports Maryland’s State And School System Implementation Strategies, Successes, and Outcomes rd International Conference on PBS March 23, 2006

2 Presentation Team Milt McKenna Milt McKenna Susan Barrett Susan Barrett Jerry Bloom Jerry Bloom Catherine Bradshaw Catherine Bradshaw Tim McCormick Tim McCormick

3 ACKNOWLWEDGEMENTS Maryland State Department of Education Maryland State Department of Education Sheppard Pratt Health System Sheppard Pratt Health System Johns Hopkins University Johns Hopkins University 24 Local School Systems 24 Local School Systems University of Oregon University of Oregon University of Connecticut University of Connecticut

4 Advanced Organizer Evolution of PBIS in MD Evolution of PBIS in MD Organization and Implementation Activities Organization and Implementation Activities Evaluation Structure Evaluation Structure Lessons Learned Years 1-5 Lessons Learned Years 1-5 Years 5-10 Years 5-10

5 Pennsylvania Delaware D.C. Virginia West Virginia

6 Big picture ….. How do you change individual behavior in schools? How do you create school environments that sustain change? What system-level mechanisms and infrastructure are necessary to sustain and bring to scale interventions that create desired change?

7 Coordination/ Collaboration PBIS MSDE Sheppard Pratt Johns Hopkins University Local School Systems

8 Participating PBIS Schools Total Public Schools: 1380 Elementary: 829 Middle: 233 Senior: 236 Combined: 40 Other 42 Nonpublic 1157 PBIS Schools trained Total: 368 ES: 189 K-8: 8 MS: 101 HS: 35 Special: 10 Alternative:24 Participating: 301

9 Number of MD Schools Teams and Behavior Support Coaches Trained By Year

10 Schools Trained and Active

11 MD Implementation Model State Leadership Team State Leadership Team Local School System (LSS) Contact Local School System (LSS) Contact Coach Facilitators/ Coordinators Coach Facilitators/ Coordinators Behavior Support Coaches Behavior Support Coaches School Teams School Teams

12 State Leadership Team: Diversified MSDE Division of Special Education/ Early Intervention Services MSDE Division of Special Education/ Early Intervention Services MSDE Division of Student and School Services MSDE Division of Student and School Services Sheppard Pratt Health System Sheppard Pratt Health System Johns Hopkins University Johns Hopkins University LSS Behavior Support Coaches LSS Behavior Support Coaches Mental Health Partners Mental Health Partners

13 State Leadership Team Functions Set policy Set policy Provide support for local leadership Provide support for local leadership Influence System of Change at District Level Influence System of Change at District Level Assess Training Needs Assess Training Needs Event Coordination Event Coordination Provide Training and Technical Assistance Provide Training and Technical Assistance Monitor Outcomes Monitor Outcomes Features of implementation Features of implementation Referrals Referrals Other indicators Other indicators

14 Local School System (LSS) 24 Local School Systems 24 Local School Systems Designated Point of Contact Designated Point of Contact –Director of Student Services –Director of Special Education

15 Behavior Support Coaches 183 Behavior Support Coaches 183 Behavior Support Coaches – Itinerant positions/funded by LSS Meet 5 times/year Meet 5 times/year – Networking – Regional Meetings – Workgroups Coaches Reception Coaches Reception Coaches Newsletter/Coaches Calendar Coaches Newsletter/Coaches Calendar Coach = Recruiter Coach = Recruiter 5 LSS Coach Facilitators

16 Behavior Support Coach Activities FTE allocated to complete tasks FTE allocated to complete tasks Consistently attend team meetings Consistently attend team meetings Assist team with data-based decision-making, planning, and implementation Assist team with data-based decision-making, planning, and implementation Attend Regional/State Coaches meetings/trainings Attend Regional/State Coaches meetings/trainings Send information to PBIS State/District Coordinator (e.g., checklists, action plans, etc.) Send information to PBIS State/District Coordinator (e.g., checklists, action plans, etc.) Assist with dissemination activities (e.g., presentations, case studies, articles, etc.) Assist with dissemination activities (e.g., presentations, case studies, articles, etc.)

17 School Commitments/Assurances One of top SIT Goals One of top SIT Goals Active Role By Administration Active Role By Administration 3 year timeline 3 year timeline Use DATA for decision making - Use DATA for decision making - –And to INFORM

18 School Team Activities Administrator is active and present for meetings. Administrator is active and present for meetings. Team is making progress on PBIS “Getting Started” checklist (Form A). Team is making progress on PBIS “Getting Started” checklist (Form A). Team uses school discipline & related DATA to discuss monthly progress. Team uses school discipline & related DATA to discuss monthly progress. Team uses annual action plan to discuss monthly progress Team uses annual action plan to discuss monthly progress Team provides monthly updates/data summaries to entire school staff Team provides monthly updates/data summaries to entire school staff Team meetings are efficiently run (e.g., clear objectives, tasks, goals) Team meetings are efficiently run (e.g., clear objectives, tasks, goals) Team activities are coordinated with other school initiatives/committees Team activities are coordinated with other school initiatives/committees

19 Maryland Annual Events Spring Forum (April 6, 2006) Spring Forum (April 6, 2006) July Institute (July 10, July 17) July Institute (July 10, July 17) Coaches Meetings (5/year) Coaches Meetings (5/year) Regional Team Leader/Coach Meetings (2/year) Regional Team Leader/Coach Meetings (2/year) Schools serving students with special needs - MANSEF (2/year) Schools serving students with special needs - MANSEF (2/year) High Schools – (2/year) High Schools – (2/year)

20 Start Up Activities Commitment and Application Process Commitment and Application Process Planning Phases Planning Phases Self Assessment Self Assessment Profile Profile

21 Spring Forum Purpose: Recruitment Purpose: Recruitment Date: April (set one year in advance) Date: April (set one year in advance) Overview & School Presentations Overview & School Presentations Participants: Participants: – Administrators from “potential new” schools – “Key” system personnel – Potential coaches Planning Phase I Begins

22 Summer Training New Coaches New Coaches New Teams New Teams Returning Teams Returning Teams Exemplar Schools Exemplar Schools Receptions and Poster Sessions Receptions and Poster Sessions

23 Other Training Events Local Coordinators Local Coordinators State Coaches Meetings State Coaches Meetings Regional Meetings Regional Meetings Targeted Group Targeted Group Behavioral Basics Behavioral Basics School Wide Information System (SWIS) School Wide Information System (SWIS) SWIS facilitator SWIS facilitator School-wide Evaluation Tool (SET) School-wide Evaluation Tool (SET)

24 Evaluation Capacity Establishing Measurable Outcomes 1. What schools have been trained and are active? 2. How well are schools implementing PBIS? 3. What impact does PBIS have on student behavior? Achievement?

25 Monitoring Outcomes Team Implementation Checklist Team Implementation Checklist SWIS SWIS SET SET Coaches Checklist Coaches Checklist Staff Survey Staff Survey Satisfaction Surveys Satisfaction Surveys Implementation Phases Inventory (IPI) Implementation Phases Inventory (IPI)

26 Evaluation Tools Access 2003 Database Access 2003 Database Data entry/storage Data entry/storage Report Generation Report Generation

27 Evaluation Tools Maryland website Various levels:Any user Team/coach Team/coach LSS Point of Contact State Team Maryland Forms Matrix Access Database SWIS PBS surveys (

28

29 How Well are Schools Implementing? Systems-wide Evaluation Tool (SET) Systems-wide Evaluation Tool (SET) –Annually –7 Features of SW Implementation Implementation Phases Inventory (IPI) Implementation Phases Inventory (IPI) –Semi-annually –Levels of SW: Preparation, Initiation, Implementation, and Maintenance

30 School-Wide Evaluation Tool 35 Coaches trained as SET assessors 15 Contractual SET assessors 97 SETs completed SETs completed SETs completed SETs completed schools have at least two SET scores 104 schools have at least two SET scores 80% Total score is considered Maintenance Phase (IPI) 80% Total score is considered Maintenance Phase (IPI) All regions met 80% criterion across schools All regions met 80% criterion across schools 69% increase after one year of implementation 69% increase after one year of implementation

31 SET Scores by Region

32

33 PROJECT TARGET Evaluating PBIS in Maryland Dr. Catherine Bradshaw Center for the Prevention of Youth Violence Johns Hopkins University Project Target

34 Randomized Trial of PBIS: Project Target Sample 37 Elementary Schools 37 Elementary Schools – Allegany (7), Anne Arundel (4), Baltimore (13), Charles(11), & Washington(2) – 3,057 Staff & 20,246 Students Design Randomly Assigned Randomly Assigned – 21 PBIS – 16 Comparison Baseline plus 4 years Baseline plus 4 years Funding : NIMH & CDC to the Johns Hopkins Center for the Prevention of Youth Violence Project Target

35 Data Collected Disruptive behavior Disruptive behavior – Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson et al., 1991) – Student Interactions in Specific Settings (SISS; Cushing & Horner, 2002) – Office discipline referrals (SWIS; School-Wide Information System) Academic information Academic information – Attendance – Academic performance School climate School climate – School Climate Survey (Haynes, Emmons, & Comer, 1994) Organizational health Organizational health – Organizational Health Inventory (Hoy et al., 1990) Implementation fidelity Implementation fidelity – System-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001) – Staff Survey (Sugai, Todd, & Horner, 2000) Project Target

36 Implementation of PBIS Notes. No significant differences between groups at baseline, but differences at all other years, p<.05. * *

37 Impact of PBIS on Organizational Health Mean OHI Note. Adjusted means from 3-level model in Mplus. Intervention effect on slope of overall OHI significant at p<.05.

38 Summary of Preliminary Findings from Project Target PBIS schools have high program fidelity PBIS schools have high program fidelity PBIS increased organizational health PBIS increased organizational health Some positive effects on student outcomes Some positive effects on student outcomes – Behavior problems in classroom – Reduced office discipline referrals – Reduced suspensions Project Target

39 Lessons Learned Establish 3-5 Year Commitment Establish 3-5 Year Commitment Designate State Leader(s) Designate State Leader(s) Establish Diversified State Leadership Team Establish Diversified State Leadership Team Establish Coaching Model Establish Coaching Model Establish Accountability Model Establish Accountability Model –Monthly Reports –Site Visits –Data Collection –Exemplars

40 Anticipated Growth 50% of MD Schools will be trained by 2010

41 MARYLAND Years 5-10 Expanded State Leadership Team Expanded State Leadership Team State-Level Advisory Committee State-Level Advisory Committee Management Team Management Team Mental Health Integration Grant Mental Health Integration Grant

42 5 YEAR GOALS Goal 1:Increase state coordination by identifying regional coordinators for Southern, Central, Eastern and Western Maryland. Goal 1:Increase state coordination by identifying regional coordinators for Southern, Central, Eastern and Western Maryland. Goal 2:Increase evaluation capacity by expanding web site functionality i.e. online tutorials, data submission and retrieval. Goal 2:Increase evaluation capacity by expanding web site functionality i.e. online tutorials, data submission and retrieval. Goal 3:Increase local training capacity by identifying training coordinator(s). Goal 3:Increase local training capacity by identifying training coordinator(s). Goal 4:Increase visibility by identifying an individual to write grants, develop marketing material and interface with outside agencies. Goal 4:Increase visibility by identifying an individual to write grants, develop marketing material and interface with outside agencies.

43 Useful Web Links Maryland Maryland – Oregon Oregon –

44 CONTACT INFORMATION Milt McKenna – Maryland State Department of Education, Susan Barrett - Sheppard Pratt Health System, Jerry Bloom – Sheppard Pratt Health System, Catherine Bradshaw – Center for the Prevention of Youth Violence, Johns Hopkins University, 410 Tim McCormick – Maryland State Department of Education,