1 Maryland’s State Implementation Strategies, Successes, and Outcomes Susan Barrett

2 ACKNOWLWEDGEMENTS Maryland State Department of Education Maryland State Department of Education Sheppard Pratt Health System Sheppard Pratt Health System Johns Hopkins University Johns Hopkins University 24 Local School Systems 24 Local School Systems University of Oregon University of Oregon University of Connecticut University of Connecticut

3 Pennsylvania Delaware D.C. Virginia West Virginia

4

5 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

6 Political Support PBIS was part of the top ten MSDE state supplemental budget request for FY’ 00-FY’ 07. Section Of the Annotated Code of Maryland In this section, “Positive Behavioral Interventions and Support Program” means the research-based, systems approach method adopted by the State Board to build capacity among school staff to adopt and sustain the use of positive, effective practices to create learning environments where teachers can teach and students can learn. Each county board of education and the Board of School Commissioners of Baltimore City shall require an elementary school that has a suspension rate that exceeds 18 percent of the elementary school’s enrollment to implement: – –A Positive Behavioral Interventions and Support Program; or – –An alternative behavioral modification program in collaboration with the Department. The State Board shall adopt regulations to implement the provisions of this section.

7 Section Of the Annotated Code of Maryland (a)In this section, “Positive Behavioral Interventions and Support Program” means the research-based, systems approach method adopted by the State Board to build capacity among school staff to adopt and sustain the use of positive, effective practices to create learning environments where teachers can teach and students can learn. (b)Each county board of education and the Board of School Commissioners of Baltimore City shall require an elementary school that has a suspension rate that exceeds 18 percent of the elementary school’s enrollment to implement: (1)A Positive Behavioral Interventions and Support Program; or (2)An alternative behavioral modification program in collaboration with the Department. (c)The State Board shall adopt regulations to implement the provisions of this section.

8 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

9 State Leadership Team Visibility: MSDE Marketing Department is developing a tiered marketing/advertising plan targeting key PBIS stakeholders to generate more awareness for the initiative. MSDE Marketing Department is developing a tiered marketing/advertising plan targeting key PBIS stakeholders to generate more awareness for the initiative. Target Audiences: Parents and Community Members/Leaders, District Leaders, Instructional Leaders, State Leaders, Teachers, Students Politicians

10 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

11 State Leadership Team Funding: As the PBIS initiative prepares for a large- scale expansion, we are in progress on a framework to identify and secure a broad base of funding support to ensure the infrastructure required to maintain the consistency and quality of technical assistance and evaluation support to districts.

12 Funding PBIS has been one of the top ten line items of the Maryland State Department of Education supplemental budget request FY’ 00- FY’ 07. PBIS has been one of the top ten line items of the Maryland State Department of Education supplemental budget request FY’ 00- FY’ 07.

13 Funding -Reality Title IV-Safe and Drug Free Schools Title IV-Safe and Drug Free Schools Special Education Special Education HIV prevention HIV prevention Drop Out Prevention Drop Out Prevention Sheppard Pratt Sheppard Pratt

14 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

15 Schools Trained and Active

16 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

17 State Leadership Team: Diversified MSDE Division of Special Education/ Early Intervention Services MSDE Division of Student and School Services Sheppard Pratt Health System Johns Hopkins University LSS Behavior Support Coaches Juvenile Justice Governor’s Office of Children University of Maryland Department of Health and Mental Hygiene Management Team-Core Group MSDE, JHU, Sheppard Pratt

18 Mental Health Integration Grant MARYLAND STATE DEPARTMENT OF EDUCATION MARYLAND STATE DEPARTMENT OF EDUCATION THE UNIVERSITY OF MARYLAND, BALTIMORE, ON BEHALF OF ITS CENTER FOR SCHOOL MENTAL HEALTH ANALYSIS AND ACTION THE UNIVERSITY OF MARYLAND, BALTIMORE, ON BEHALF OF ITS CENTER FOR SCHOOL MENTAL HEALTH ANALYSIS AND ACTION DEPARTMENT OF JUVENILE SERVICES DEPARTMENT OF JUVENILE SERVICES GOVERNOR’S OFFICE FOR CHILDREN GOVERNOR’S OFFICE FOR CHILDREN MENTAL HYGIENE ADMINISTRATION MENTAL HYGIENE ADMINISTRATION THE MENTAL HEALTH ASSOCIATION OF MARYLAND THE MENTAL HEALTH ASSOCIATION OF MARYLAND THE MARYLAND COALITION OF FAMILIES FOR CHILDREN’S MENTAL HEALTH THE MARYLAND COALITION OF FAMILIES FOR CHILDREN’S MENTAL HEALTH MARYLAND ASSEMBLY ON SCHOOL-BASED HEALTHCARE MARYLAND ASSEMBLY ON SCHOOL-BASED HEALTHCARE SHEPPARD PRATT HEALTH SYSTEM SHEPPARD PRATT HEALTH SYSTEM

19 State Leadership Team Functions Set policy Set policy Provide support for local leadership Provide support for local leadership Influence System of Change at District Level Influence System of Change at District Level Assess Training Needs Assess Training Needs Event Coordination Event Coordination Provide Training and Technical Assistance Provide Training and Technical Assistance Monitor Outcomes Monitor Outcomes Features of implementation Features of implementation Referrals Referrals Other indicators Other indicators

20 Building Local Capacity Roles and Responsibilities Roles and Responsibilities

21 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

22 Behavior Support Coaches 283 Behavior Support Coaches 283 Behavior Support Coaches – Itinerant positions/funded by LSS Meet 5 times/year Meet 5 times/year – Networking – Regional Meetings – Workgroups Coaches Reception Coaches Reception Coaches Newsletter/Coaches Calendar Coaches Newsletter/Coaches Calendar Coach = Recruiter Coach = Recruiter 8 LSS Coordinators

23 Roles of Coach Sustainability & Accountability Sustainability & Accountability Hands-on technical assistance Hands-on technical assistance Guide problem solving Guide problem solving Local training Local training Team start-up & sustainability Team start-up & sustainability Public relations/communications Public relations/communications Support local leadership Support local leadership Local coordination of resources Local coordination of resources Provide prompts & reinforcers Provide prompts & reinforcers

24 Behavior Support Coach Activities FTE allocated to complete tasks FTE allocated to complete tasks Consistently attend team meetings Consistently attend team meetings Assist team with data-based decision-making, planning, and implementation Assist team with data-based decision-making, planning, and implementation Attend Regional/State Coaches meetings/trainings Attend Regional/State Coaches meetings/trainings Send information to PBIS State/District Coordinator (e.g., checklists, action plans, etc.) Send information to PBIS State/District Coordinator (e.g., checklists, action plans, etc.) Assist with dissemination activities (e.g., presentations, case studies, articles, etc.) Assist with dissemination activities (e.g., presentations, case studies, articles, etc.)

25 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

26 Maryland Annual Events Spring Forum (April 6, 2006) Spring Forum (April 6, 2006) July Institute (July 10, July 17) July Institute (July 10, July 17) Coaches Meetings (5/year) Coaches Meetings (5/year) Regional Team Leader/Coach Meetings (2/year) Regional Team Leader/Coach Meetings (2/year) Schools serving students with special needs - MANSEF (2/year) Schools serving students with special needs - MANSEF (2/year) High Schools – (2/year) High Schools – (2/year)

27 Spring Forum Purpose: Recruitment Purpose: Recruitment Date: April (set one year in advance) Date: April (set one year in advance) Overview & School Presentations Overview & School Presentations Participants: Participants: – Administrators from “potential new” schools – “Key” system personnel – Potential coaches Planning Phase I Begins

28 Summer Training New Coaches New Coaches New Teams New Teams Returning Teams Returning Teams Exemplar Schools Exemplar Schools Receptions and Poster Sessions Receptions and Poster Sessions

29 Other Training Events Local Coordinators/Trainers Local Coordinators/Trainers State Coaches Meetings State Coaches Meetings Regional Meetings Regional Meetings Targeted Group Targeted Group Behavioral Basics Behavioral Basics School Wide Information System (SWIS) School Wide Information System (SWIS) SWIS facilitator SWIS facilitator School-wide Evaluation Tool (SET) School-wide Evaluation Tool (SET)

30 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Active Coordination Local School Teams/Demonstrations

31 Evaluation Capacity Establishing Measurable Outcomes 1. What schools have been trained and are active? 2. How well are schools implementing PBIS? 3. What impact does PBIS have on student behavior? Achievement?

32 Monitoring Outcomes Team Implementation Checklist Team Implementation Checklist SWIS SWIS SET SET Coaches Checklist Coaches Checklist Staff Survey Staff Survey Satisfaction Surveys Satisfaction Surveys Implementation Phases Inventory (IPI) Implementation Phases Inventory (IPI)

33 Evaluation Tools Access 2003 Database Access 2003 Database Data entry/storage Data entry/storage Report Generation Report Generation

34 Evaluation Tools Maryland website Various levels:Any user Team/coach Team/coach LSS Point of Contact LSS Point of Contact State Team State Team Maryland Forms Matrix Access Database SWIS PBS surveys (

35

36 How Well are Schools Implementing? Systems-wide Evaluation Tool (SET) Systems-wide Evaluation Tool (SET) –Annually –7 Features of SW Implementation Implementation Phases Inventory (IPI) Implementation Phases Inventory (IPI) –Semi-annually –Levels of SW: Preparation, Initiation, Implementation, and Maintenance

37 School-Wide Evaluation Tool 35 Coaches trained as SET assessors 15 Contractual SET assessors 97 SETs completed SETs completed SETs completed SETs completed schools have at least two SET scores 104 schools have at least two SET scores 80% Total score is considered Maintenance Phase (IPI) 80% Total score is considered Maintenance Phase (IPI) All regions met 80% criterion across schools All regions met 80% criterion across schools 69% increase after one year of implementation 69% increase after one year of implementation

38 SET Scores by Region

39

40 PROJECT TARGET Evaluating PBIS in Maryland Dr. Catherine Bradshaw Center for the Prevention of Youth Violence Johns Hopkins University Project Target

41 Randomized Trial of PBIS: Project Target Sample 37 Elementary Schools 37 Elementary Schools – Allegany (7), Anne Arundel (4), Baltimore (13), Charles(11), & Washington(2) – 3,057 Staff & 20,246 Students Design Randomly Assigned Randomly Assigned – 21 PBIS – 16 Comparison Baseline plus 4 years Baseline plus 4 years Funding : NIMH & CDC to the Johns Hopkins Center for the Prevention of Youth Violence Project Target

42 Data Collected Disruptive behavior Disruptive behavior – Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson et al., 1991) – Student Interactions in Specific Settings (SISS; Cushing & Horner, 2002) – Office discipline referrals (SWIS; School-Wide Information System) Academic information Academic information – Attendance – Academic performance School climate School climate – School Climate Survey (Haynes, Emmons, & Comer, 1994) Organizational health Organizational health – Organizational Health Inventory (Hoy et al., 1990) Implementation fidelity Implementation fidelity – System-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001) – Staff Survey (Sugai, Todd, & Horner, 2000) Project Target

43 Implementation of PBIS Notes. No significant differences between groups at baseline, but differences at all other years, p<.05. * *

44 Impact of PBIS on Organizational Health Mean OHI Note. Adjusted means from 3-level model in Mplus. Intervention effect on slope of overall OHI significant at p<.05.

45 Summary of Preliminary Findings from Project Target PBIS schools have high program fidelity PBIS schools have high program fidelity PBIS increased organizational health PBIS increased organizational health Some positive effects on student outcomes Some positive effects on student outcomes – Behavior problems in classroom – Reduced office discipline referrals – Reduced suspensions Project Target

46 PBS Systems Implementation Logic Leadership Team Funding VisibilityPolitical Support TrainingCoachingEvaluation Action Plan Local School Teams/Demonstrations

47 Anticipated Growth 50% of MD Schools will be trained by 2010

48 5 YEAR GOALS Goal 1:Increase state coordination by identifying regional coordinators for Southern, Central, Eastern and Western Maryland. Goal 1:Increase state coordination by identifying regional coordinators for Southern, Central, Eastern and Western Maryland. Goal 2:Increase evaluation capacity by expanding web site functionality i.e. online tutorials, data submission and retrieval. Goal 2:Increase evaluation capacity by expanding web site functionality i.e. online tutorials, data submission and retrieval. Goal 3:Increase local training capacity by identifying training coordinator(s). Goal 3:Increase local training capacity by identifying training coordinator(s). Goal 4:Increase visibility by identifying an individual to write grants, develop marketing material and interface with outside agencies. Goal 4:Increase visibility by identifying an individual to write grants, develop marketing material and interface with outside agencies.

49 Lessons Learned Pacing Pacing Be Patient Be Patient Exemplar Exemplar Feedback Feedback Social Marketing-target audiences Social Marketing-target audiences Role of Coach Role of Coach Process Process