1. Implementing Applied Behavior Analysis at Scales of Social Importance
Rob Horner
University of Oregon

2. Main Messages
Behavior Analysis has the potential to improve the quality of life for children and families in our society
The benefits of ABA, however, will not be realized until we become much better at implementing our technology at scales of social importance

3. Goals
Lessons Learned from Efforts to Implement School-wide Positive Behavioral Interventions and Supports (PBIS)
Participants will define three themes that are shaping implementation of ABA
Participants will define the role of “core features” in the definition of effective practices, and the importance of defining core features for large scale implementation
Participants will define the importance of measuring “implementation fidelity”
Participants will define the importance of “efficiency” and “sustainability” in the large-scale adoption of new practices.

4. Effective Implementation
Themes Affecting Adoption of Behavior Analysis Multi-tiered Systems, Evidence-based Practices, Implementation Science
Evidence-based Practices
Performance Assessment (Fidelity)
Coaching
Training
Selection
Systems Intervention
Facilitative Administration
Decision Support Data System
Competency Drivers
Effective Implementation
Organization Drivers
Adaptive
Technical
Leadership Drivers
Multi-tiered Systems of Support
Implementation Science

5. PBIS is a practical application of ABA
What is School-wide Positive Behavior Interventions and Supports (PBIS)?
School-wide PBIS is:
A multi-tiered framework for establishing the social culture and behavioral supports needed for a school achieve behavioral and academic outcomes for all students.
Evidence-based features of SWPBIS
Prevention
Define and teach positive social expectations
Acknowledge positive behavior
Arrange consistent consequences for problem behavior
On-going collection and use of data for decision-making
Continuum of intensive, individual intervention supports.
Implementation of the systems that support effective practices
PBIS is a practical application of ABA

6. School-wide Positive Behavioral Interventions and Supports (PBIS)
The social culture of a school matters.
A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families.
Effective practices with the systems needed for high fidelity and sustainability
Multiple tiers of intensity

7. Why PBIS?
The fundamental purpose of PBIS is to make schools more effective, efficient and equitable learning environments.
Predictable
Positive
Consistent
Safe

8. Invest in prevention first Multiple tiers of support intensity
SCHOOL-WIDE
POSITIVE BEHAVIOR
SUPPORT
Tertiary Prevention:
Specialized
Individualized
Systems for Students with High-Risk Behavior
~5%
Secondary Prevention:
Specialized Group
Systems for Students with At-Risk Behavior
~15%
Primary Prevention:
School-/Classroom-
Wide Systems for
All Students,
Staff, & Settings
Main Ideas:
Invest in prevention first
Multiple tiers of support intensity
Early/rapid access to support
~80% of Students 27

9. Experimental Research on PBIS
SWPBIS Experimentally Related to:
Reduction in problem behavior
Increased academic performance
Improved perception of safety
Reduction in bullying behaviors
Improved organizational efficiency
Reduction in staff turnover
Increased perception of teacher efficacy
Improved Social Emotional competence
Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):

10. Schools Using PBIS Jan, 2017 3138 High Schools
24,312 Schools Implementing PBIS
11,958,000 Students
3138 High Schools

11. Schools Implementing PBIS Jan, 2017
21 States with over 500 schools using PBIS
California
Massachusetts

12. Proportion of Schools Implementing PBIS by State, Jan, 2017
11 States with over 40% of schools using PBIS

13. Define and distinguish between Practices Core features Valued outcomes
Lessons Learned
Define and distinguish between
Practices
Core features
Valued outcomes
Functional Communication Training
Your Favorite
Pivotal Response Training
Discrete Trial Training

14. Defining a “Practice/ Program/ Intervention”
A “practice” is a procedure, or set of procedures, designed for use in a specific context, by individuals with certain skills/features, to produce specific changes in context or performance patterns that result in valued outcomes for specific individuals.
Operationally defined procedures (Core Features)
What you do
Target population
For whom
Setting/Context
Where
Implementer Characteristics
By whom
Defined outcomes
Valued impact
(Evidence of functional relation)
Practice  Core Features  Valued outcome
Flay et al., 2005

15. Practices  Core Features  Outcomes
Effective Practice
Effective Practice
Effective Practice
Effective Practice
Technology
Be Firm about the Values and Science
Be Flexible about the Technology
Core Features
Science
Values
Valued Outcomes

16. Anticipate implementation error patterns
Lessons Learned
Anticipate implementation error patterns

17. Primary Prevention: School-/Classroom- Wide Systems for All Students,
SCHOOL-WIDE
POSITIVE BEHAVIOR
SUPPORT
Tertiary Prevention:
Specialized
Individualized
Systems for Students with High-Risk Behavior
~5%
Secondary Prevention:
Specialized Group
Systems for Students with At-Risk Behavior
~15%
Primary Prevention:
School-/Classroom-
Wide Systems for
All Students,
Staff, & Settings
~80% of Students 27

18. Systems for Students with High-Risk Behavior SCHOOL-WIDE
Tertiary Prevention:
Specialized
Individualized
Systems for Students with High-Risk Behavior
SCHOOL-WIDE
POSITIVE BEHAVIOR
SUPPORT
Secondary Prevention:
Specialized Group
Systems for Students with At-Risk Behavior
Primary Prevention:
School-/Classroom-
Wide Systems for
All Students,
Staff, & Settings 27

19. School-Wide Positive Behavior Support Tertiary Prevention: Specialized
Individualized
Systems for Students with High-Risk Behavior
~5%
Secondary Prevention:
Specialized Group
Systems for Students with At-Risk Behavior
Primary Prevention:
School-/Classroom-
Wide Systems for
All Students,
Staff, & Settings
~15%
This is the same model used by RTI for academics—the two systems are the same; within IPBS we are building on this logic to support all students.
~80% of Students

20. Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center
Multi-tier Model
Academic Systems
Behavioral Systems
Intensive, Individual Interventions
Individual Students
Assessment-based
High Intensity
Of longer duration
Intensive, Individual Interventions
Individual Students
Assessment-based
Intense, durable procedures
1-5%
1-5%
5-10%
5-10%
Targeted Group Interventions
Some students (at-risk)
High efficiency
Rapid response
Targeted Group Interventions
Some students (at-risk)
High efficiency
Rapid response
Although three tiers are the ones most often seen, an RtI model can have any number of tiers.
One misinterpretation to guard against is that tier 1 is general education, tier 2 is Title I and tier 3 is Special Education. This is a common misunderstanding and could lead to simply keeping the historical system and calling it RtI. General ed., title I and special education are resources for providing Universal interventions, supplemental interventions and intensive interventions. There are students, for example, who need intensive intervention who do not qualify for special education ( ELL, gifted and talented, students who have missed a lot of school). The focus of this model is primarily on the Nature and Intensity of instruction that students need.
80-90%
Universal Interventions
All students
Preventive, proactive
All settings, all students
Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center
(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

22. Is Integrated Continuum23
Behavior Continuum
Academic Continuum
PBIS
Is Integrated Continuum
NOTICE GREEN GOES IS FOR “ALL”
Mar

23. + Where is the students performing now? Where do we want him to be?
Tier III
For Approx 5% of Students
Core +
Supplemental
Intensive Individual Instruction
…to achieve benchmarks
Where is the students performing now?
Where do we want him to be?
How long do we have to get him there?
What supports has he received?
What resources will move him at that rate?
Tier III Effective if there is progress (i.e., gap closing) towards benchmark and/or progress monitoring goals.
District facilitates discussions among schools on curriculum standards preschool through12th grade.
Systematic process for monitoring, evaluating, reviewing curriculum. 23 23 23

24. School Data Teams School Data Teams
Bethel’s Comprehensive Secondary Counseling Program
Outcomes
Individualized
Targeted
Universal
Engagement
School Data Teams
Behavior (PBS)
School Data Teams
Academics

25. Positive Behavior Support
Universal
School-Wide Data Collection and Analyses
School-Wide Prevention Systems (rules, routines, arrangements)
Targeted
Intensive
Group Interventions
Analyze Student Data
Interviews, Questionnaires, etc.
Simple Student Interventions
Intervention
Assessment
Observations and ABC Analysis
Complex Individualized Interventions
Multi-Disciplinary Assessment & Analysis
Team-Based Wraparound Interventions
Dr. Terry Scott: Adapted from George Sugai, 1996
© Terrance M. Scott, 2001

26. Tier I: Universal/Prevention for All
Coordinated Systems, Data, Practices for Promoting Healthy Social
and Emotional Development for ALL Students
Tier 2: Early Intervention for Some
Coordinated Systems for Early Detection, Identification,
and Response to Mental Health Concerns
Tier 3: Intensive Interventions for Few
Individual Student and Family Supports
Adapted from the ICMHP Interconnected Systems Model for School Mental Health, which was originally adapted from Minnesota Children’s Mental Health Task Force, Minnesota
Framework for a Coordinated System to Promote Mental Health in Minnesota; center for Mental Health in Schools, Interconnected Systems for Meeting the Needs of All Youngsters.

27. Aurora, CO, EBD, all schools
Two messages:
1. high rates of problem behaviors & reactive “get tough” management
2. teaching to the corner

28. Math
Remember that the multiple tiers of support refer to our SUPPORT not Students.
Avoid creating a new disability labeling system.
Behavior
Health
Reading

29. Measure “fidelity of implementation”
Lesson Learned
Measure “fidelity of implementation”
As a DV to assess implementation practices
As an IV to improve level of adoption.

30. Fidelity Measures within SWPBIS
~5%
~15%
~80% of Students

31. Sub-scale report

32. Sub-subscale report Tier I Teams Implementation Evaluation Tier II
Interventions
Evaluation
Tier III
Teams
Resources
Assessment
Support plan
Monitoring and adaptation

33. Item Report

34. Action Planning
1. Team to propose teaching template and Fall teaching schedule at Feb 16 Faculty meeting.
Alan
Feb 16

35. Tier I Fidelity by School by State TFI, BoQ, SET, TIC
Count of Schools Using PBIS
10,705 schools with measured Tier I fidelity
Count of Schools Measuring Fidelity
Count of Schools Achieving Fidelity

36. The Role of Implementation Fidelity at Tier III

37. Improving Fidelity by Incorporating “Contextual fit”
The extent to which the procedures required by a plan of support are consistent with the values, skills, resources and administrative support of those who implement the plan

38. Use Implementation Science
Lesson Learned
Use Implementation Science
Selection of Evidence-based Practices
Teams
Implementation Drivers
Stages of Implementation
Improvement Cycles

39. Implementation Science Frameworks
WHO
Teams
WHEN
Stages
WHAT
Interventions
HOW
Cycles
HOW
Drivers
Full set of icons with headers

40. Successful Student Outcomes
Program/Initiative/Framework (e.g. RtI)
Performance Assessment (Fidelity)
Coaching
Systems
Intervention
Training
Facilitative
Administration
Competency Drivers
Organization Drivers
Implementation
Drivers
There are two categories of Implementation Drivers: Competency and Organization.
When these core components are in place they provide the support to a successful implementation that will be sustained.
Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment
Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention
PD is not a panacea to address every problem
PD must be housed in a systems to support this effective practices resulting in successful sustainable student outcomes.
Selection
Decision Support Data System
Leadership
Adaptive
Technical
© Fixsen & Blase, 2008

41. Stages of Implementation
Implementation occurs in stages:
Exploration
Installation
Initial Implementation
Full Implementation
2 – 4 Years
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

42. Leadership Team Funding Visibility Political Support Policy Training
Scaling up School-wide Positive Behavioral Interventions and Supports: The Experiences of Seven States with Documented Success Rob Horner, Don Kincaid, George Sugai, Tim Lewis, Lucille Eber, Susan Barrett, Celeste Rossetto Dickey, Mary Richter, Erin Sullivan, Cyndi Boezio, Nancy Johnson
Exploration
Installation
Initial Imp
Full Imp
Innovation
Sustainability
Leadership Team
Funding
Visibility
Political Support
Policy
Training
Coaching
Expertise
Evaluation
Demos

43. Descriptive Summary: Oregon
Exploration / Installation / Initial Imp /Full Imp & Innovate

44. Descriptive Summary: Missouri
Exploration / Installation /Initial Imp / Full Imp & Innovate

45. Descriptive Summary: North Carolina
Exploration / Installation / Initial & Full Imp / Innovate

46. Descriptive Summary: Colorado
Exploration / Installation / Initial & Full Imp / Innovate

47. Descriptive Summary: Florida
Exploration/ Installation/ Initial Imp / Full Imp / Innovate

48. Descriptive Summary: Maryland
Exploration / Installation / Initial Imp / Full Imp / Innovate

49. Descriptive Summary: Illinois
Exploration / Installation / Initial Imp /Full Imp & Innovate

50. Lessons Learned Resource Leveraging -----------------
Multiple approaches to achieving scaled implementation
Colorado: Started with Leadership Team
Illinois: Started with Leadership Advocates and built team only after implementation expanded.
Missouri: Strong initial demonstrations led to strong state support
All states began with small “demonstrations” that documented the feasibility and impact of SWPBIS.
Only when states reached demonstrations did scaling occur. Four core features needed for scaling:
Administrative Leadership / Support/ Funding
Technical capacity (Local training, coaching, evaluation and behavioral expertise)
Local Demonstrations of feasibility and impact ( )
Evaluation data system (to support continuous improvement)
Essential role of Data: Fidelity data AND Outcome data
Resource Leveraging
The use of small initial pilot investments to increase likelihood of larger, scaling investment

51. Resource Leveraging Initial Investment Step 1 Step 2 Step 3 Step 4
Investment of New Funds
Fidelity of Adoption
Effect
Reasonable Cost
Large-scale Implementation with fidelity and impact
Initial Investment
Shift in Policies, Procedures
Reallocation of Existing Resources
Resource Leveraging: Initial investment provides proof of concept that is used to leverage (a) new (larger) funding, (b) policy shifts needed for efficient implementation, and (c) reallocation of existing resources (e.g. FTE). The result is a large-scale adoption of new practices with fidelity and impact.

52. Lesson Learned Implementing at scale requires sustainability
Sustainability requires the use of data for continuous regeneration.

53. Identifying Valued & Modifying Outcomes Practices Implementation
Priority
Effectiveness
Valued
Outcomes
Identifying
& Modifying
Practices
Data-
Based
Prob.
Solving
Continuous
Regeneration
Capacity
Building
Continuous
Measurement
Implementation
Cleanest model in B=W with animations
Efficiency
McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention, 40(3),
School Context

54. Summary Implementation at scale is possible
Consider the cluster of core features needed for scaling
Admin support, Technical capacity, demonstrations
Small demonstrations may be necessary but insufficient
Build in system for adapting the program to fit the local context while retaining the core features.
Measure fidelity of implementation as a part of effective implementation.
Fidelity is affected by Contextual Fit
Sustained implementation requires continuous regeneration
Always emphasize, measure and report on valued outcomes