Implementing Applied Behavior Analysis at Scales of Social Importance Rob Horner University of Oregon

Main Messages Behavior Analysis has the potential to improve the quality of life for children and families in our society The benefits of ABA, however, will not be realized until we become much better at implementing our technology at scales of social importance

Goals Lessons Learned from Efforts to Implement School-wide Positive Behavioral Interventions and Supports (PBIS) Participants will define three themes that are shaping implementation of ABA   Participants will define the role of “core features” in the definition of effective practices, and the importance of defining core features for large scale implementation Participants will define the importance of measuring “implementation fidelity” Participants will define the importance of “efficiency” and “sustainability” in the large-scale adoption of new practices.

Effective Implementation Themes Affecting Adoption of Behavior Analysis Multi-tiered Systems, Evidence-based Practices, Implementation Science Evidence-based Practices Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Competency Drivers Effective Implementation Organization Drivers Adaptive Technical Leadership Drivers Multi-tiered Systems of Support Implementation Science

PBIS is a practical application of ABA What is School-wide Positive Behavior Interventions and Supports (PBIS)? School-wide PBIS is: A multi-tiered framework for establishing the social culture and behavioral supports needed for a school achieve behavioral and academic outcomes for all students. Evidence-based features of SWPBIS Prevention Define and teach positive social expectations Acknowledge positive behavior Arrange consistent consequences for problem behavior On-going collection and use of data for decision-making Continuum of intensive, individual intervention supports. Implementation of the systems that support effective practices PBIS is a practical application of ABA

School-wide Positive Behavioral Interventions and Supports (PBIS) The social culture of a school matters. A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families. Effective practices with the systems needed for high fidelity and sustainability Multiple tiers of intensity

Why PBIS? The fundamental purpose of PBIS is to make schools more effective, efficient and equitable learning environments. Predictable Positive Consistent Safe

Invest in prevention first Multiple tiers of support intensity SCHOOL-WIDE POSITIVE BEHAVIOR SUPPORT Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior ~5% Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior ~15% Primary Prevention: School-/Classroom- Wide Systems for All Students, Staff, & Settings Main Ideas: Invest in prevention first Multiple tiers of support intensity Early/rapid access to support ~80% of Students 27

Experimental Research on PBIS SWPBIS Experimentally Related to: Reduction in problem behavior Increased academic performance Improved perception of safety Reduction in bullying behaviors Improved organizational efficiency Reduction in staff turnover Increased perception of teacher efficacy Improved Social Emotional competence Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115 Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26. Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14. Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128. Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156

Schools Using PBIS Jan, 2017 3138 High Schools 24,312 Schools Implementing PBIS --------------------------- 11,958,000 Students 3138 High Schools

Schools Implementing PBIS Jan, 2017 21 States with over 500 schools using PBIS California Massachusetts

Proportion of Schools Implementing PBIS by State, Jan, 2017 11 States with over 40% of schools using PBIS

Define and distinguish between Practices Core features Valued outcomes Lessons Learned Define and distinguish between Practices Core features Valued outcomes Functional Communication Training Your Favorite Pivotal Response Training Discrete Trial Training

Defining a “Practice/ Program/ Intervention” A “practice” is a procedure, or set of procedures, designed for use in a specific context, by individuals with certain skills/features, to produce specific changes in context or performance patterns that result in valued outcomes for specific individuals. ------------------------------------------------- Operationally defined procedures (Core Features) What you do Target population For whom Setting/Context Where Implementer Characteristics By whom Defined outcomes Valued impact (Evidence of functional relation) Practice  Core Features  Valued outcome Flay et al., 2005

Practices  Core Features  Outcomes Effective Practice Effective Practice Effective Practice Effective Practice Technology Be Firm about the Values and Science Be Flexible about the Technology Core Features Science Values Valued Outcomes

Anticipate implementation error patterns Lessons Learned Anticipate implementation error patterns

Primary Prevention: School-/Classroom- Wide Systems for All Students, SCHOOL-WIDE POSITIVE BEHAVIOR SUPPORT Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior ~5% Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior ~15% Primary Prevention: School-/Classroom- Wide Systems for All Students, Staff, & Settings ~80% of Students 27

Systems for Students with High-Risk Behavior SCHOOL-WIDE Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior SCHOOL-WIDE POSITIVE BEHAVIOR SUPPORT Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior Primary Prevention: School-/Classroom- Wide Systems for All Students, Staff, & Settings 27

School-Wide Positive Behavior Support Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior ~5% Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior Primary Prevention: School-/Classroom- Wide Systems for All Students, Staff, & Settings ~15% This is the same model used by RTI for academics—the two systems are the same; within IPBS we are building on this logic to support all students. ~80% of Students

Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center Multi-tier Model Academic Systems Behavioral Systems Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures 1-5% 1-5% 5-10% 5-10% Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Although three tiers are the ones most often seen, an RtI model can have any number of tiers. One misinterpretation to guard against is that tier 1 is general education, tier 2 is Title I and tier 3 is Special Education. This is a common misunderstanding and could lead to simply keeping the historical system and calling it RtI. General ed., title I and special education are resources for providing Universal interventions, supplemental interventions and intensive interventions. There are students, for example, who need intensive intervention who do not qualify for special education ( ELL, gifted and talented, students who have missed a lot of school). The focus of this model is primarily on the Nature and Intensity of instruction that students need. 80-90% Universal Interventions All students Preventive, proactive All settings, all students Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center (c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

Is Integrated Continuum 23 Behavior Continuum Academic Continuum PBIS Is Integrated Continuum NOTICE GREEN GOES IS FOR “ALL” Mar 10 2010

+ Where is the students performing now? Where do we want him to be? Tier III For Approx 5% of Students Core + Supplemental Intensive Individual Instruction …to achieve benchmarks Where is the students performing now? Where do we want him to be? How long do we have to get him there? What supports has he received? What resources will move him at that rate? Tier III Effective if there is progress (i.e., gap closing) towards benchmark and/or progress monitoring goals. District facilitates discussions among schools on curriculum standards preschool through12th grade. Systematic process for monitoring, evaluating, reviewing curriculum. 23 23 23

School Data Teams School Data Teams Bethel’s Comprehensive Secondary Counseling Program Outcomes Individualized Targeted Universal Engagement School Data Teams Behavior (PBS) School Data Teams Academics

Positive Behavior Support Universal School-Wide Data Collection and Analyses School-Wide Prevention Systems (rules, routines, arrangements) Targeted Intensive Group Interventions Analyze Student Data Interviews, Questionnaires, etc. Simple Student Interventions Intervention Assessment Observations and ABC Analysis Complex Individualized Interventions Multi-Disciplinary Assessment & Analysis Team-Based Wraparound Interventions Dr. Terry Scott: Adapted from George Sugai, 1996 © Terrance M. Scott, 2001

Tier I: Universal/Prevention for All Coordinated Systems, Data, Practices for Promoting Healthy Social and Emotional Development for ALL Students Tier 2: Early Intervention for Some Coordinated Systems for Early Detection, Identification, and Response to Mental Health Concerns Tier 3: Intensive Interventions for Few Individual Student and Family Supports Adapted from the ICMHP Interconnected Systems Model for School Mental Health, which was originally adapted from Minnesota Children’s Mental Health Task Force, Minnesota Framework for a Coordinated System to Promote Mental Health in Minnesota; center for Mental Health in Schools, Interconnected Systems for Meeting the Needs of All Youngsters.

Aurora, CO, EBD, all schools Two messages: 1. high rates of problem behaviors & reactive “get tough” management 2. teaching to the corner

Math Remember that the multiple tiers of support refer to our SUPPORT not Students. Avoid creating a new disability labeling system. Behavior Health Reading

Measure “fidelity of implementation” Lesson Learned Measure “fidelity of implementation” As a DV to assess implementation practices As an IV to improve level of adoption.

Fidelity Measures within SWPBIS ~5% ~15% ~80% of Students

Sub-scale report

Sub-subscale report Tier I Teams Implementation Evaluation Tier II Interventions Evaluation Tier III Teams Resources Assessment Support plan Monitoring and adaptation

Item Report

Action Planning 1. Team to propose teaching template and Fall teaching schedule at Feb 16 Faculty meeting. Alan Feb 16

Tier I Fidelity by School by State TFI, BoQ, SET, TIC Count of Schools Using PBIS 10,705 schools with measured Tier I fidelity Count of Schools Measuring Fidelity Count of Schools Achieving Fidelity

The Role of Implementation Fidelity at Tier III

Improving Fidelity by Incorporating “Contextual fit” The extent to which the procedures required by a plan of support are consistent with the values, skills, resources and administrative support of those who implement the plan

Use Implementation Science Lesson Learned Use Implementation Science Selection of Evidence-based Practices Teams Implementation Drivers Stages of Implementation Improvement Cycles

Implementation Science Frameworks WHO Teams WHEN Stages WHAT Interventions HOW Cycles HOW Drivers Full set of icons with headers

Successful Student Outcomes Program/Initiative/Framework (e.g. RtI) Performance Assessment (Fidelity) Coaching Systems Intervention Training Facilitative Administration Competency Drivers Organization Drivers Implementation Drivers There are two categories of Implementation Drivers: Competency and Organization. When these core components are in place they provide the support to a successful implementation that will be sustained. Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention PD is not a panacea to address every problem PD must be housed in a systems to support this effective practices resulting in successful sustainable student outcomes. Selection Decision Support Data System Leadership Adaptive Technical © Fixsen & Blase, 2008

Stages of Implementation Implementation occurs in stages: Exploration Installation Initial Implementation Full Implementation 2 – 4 Years Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

Leadership Team Funding Visibility Political Support Policy Training Scaling up School-wide Positive Behavioral Interventions and Supports: The Experiences of Seven States with Documented Success Rob Horner, Don Kincaid, George Sugai, Tim Lewis, Lucille Eber, Susan Barrett, Celeste Rossetto Dickey, Mary Richter, Erin Sullivan, Cyndi Boezio, Nancy Johnson Exploration Installation Initial Imp Full Imp Innovation Sustainability Leadership Team Funding Visibility Political Support Policy Training Coaching Expertise Evaluation Demos

Descriptive Summary: Oregon Exploration / Installation / Initial Imp /Full Imp & Innovate

Descriptive Summary: Missouri Exploration / Installation /Initial Imp / Full Imp & Innovate

Descriptive Summary: North Carolina Exploration / Installation / Initial & Full Imp / Innovate

Descriptive Summary: Colorado Exploration / Installation / Initial & Full Imp / Innovate

Descriptive Summary: Florida Exploration/ Installation/ Initial Imp / Full Imp / Innovate

Descriptive Summary: Maryland Exploration / Installation / Initial Imp / Full Imp / Innovate

Descriptive Summary: Illinois Exploration / Installation / Initial Imp /Full Imp & Innovate

Lessons Learned Resource Leveraging ----------------- Multiple approaches to achieving scaled implementation Colorado: Started with Leadership Team Illinois: Started with Leadership Advocates and built team only after implementation expanded. Missouri: Strong initial demonstrations led to strong state support All states began with small “demonstrations” that documented the feasibility and impact of SWPBIS. Only when states reached 100-200 demonstrations did scaling occur. Four core features needed for scaling: Administrative Leadership / Support/ Funding Technical capacity (Local training, coaching, evaluation and behavioral expertise) Local Demonstrations of feasibility and impact (100-200) Evaluation data system (to support continuous improvement) Essential role of Data: Fidelity data AND Outcome data Resource Leveraging ----------------- The use of small initial pilot investments to increase likelihood of larger, scaling investment

Resource Leveraging Initial Investment Step 1 Step 2 Step 3 Step 4 Investment of New Funds Fidelity of Adoption -------------- Effect Reasonable Cost Large-scale Implementation with fidelity and impact Initial Investment Shift in Policies, Procedures Reallocation of Existing Resources Resource Leveraging: Initial investment provides proof of concept that is used to leverage (a) new (larger) funding, (b) policy shifts needed for efficient implementation, and (c) reallocation of existing resources (e.g. FTE). The result is a large-scale adoption of new practices with fidelity and impact.

Lesson Learned Implementing at scale requires sustainability Sustainability requires the use of data for continuous regeneration.

Identifying Valued & Modifying Outcomes Practices Implementation Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Implementation Cleanest model in B=W with animations Efficiency McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention, 40(3), 184-191. School Context

Summary Implementation at scale is possible Consider the cluster of core features needed for scaling Admin support, Technical capacity, 100-200 demonstrations Small demonstrations may be necessary but insufficient Build in system for adapting the program to fit the local context while retaining the core features. Measure fidelity of implementation as a part of effective implementation. Fidelity is affected by Contextual Fit Sustained implementation requires continuous regeneration Always emphasize, measure and report on valued outcomes