DO WE SAY “ALL” BUT MEAN “SOME”? Data-driven Conversations on Equity & Disproportionality Bert Eliason & Katie Conley PBIS Applications NWPBIS Network Conference 2016 Portland – Red Lion Jantzen Beach

Goal Use a four-step problem-solving model to address discipline disproportionality Resources Data Guide for School Teams (pbis.org, 2015) Four-step problem-solving model SWIS Ethnicity Report Organizing and Analyzing the Data Data Sources Common Metrics Drill Down Process Putting It Into Practice School Example Session Intentions

Educational and Community Supports (ECS) is a research unit within the College of Education at the University of Oregon. ECS focuses on the development and implementation of practices that result in positive, durable, and scientifically substantiated change in the lives of individuals. Federal and state funded projects support research, teaching, dissemination, and technical assistance. PBIS Applications (PBISApps) is a series of educational tools created within ECS and related to the implementation of multi-tiered systems of support (MTSS). The PBIS Application tools have been utilized in 25,000+ schools both domestically and internationally. Educational and Community Supports

School  Equity & PBIS

Data Sources: What is Necessary? Required features: Consistent data collection Discipline referrals (ODRs) Student race/ethnicity School enrollment by race/ethnicity Instantaneous access for school teams—not just district teams Capability to disaggregate ODRs and patterns by race/ethnicity Capability to calculate risk indices and risk ratios by race/ethnicity

Data Sources: What is Recommended? Standardized forms with the 5W’s Who, what, when, where, why Clear definitions of problem behaviors Clear guidance in discipline procedures Office vs. staff-managed Report generation On demand Disaggregated by race/ethnicity Automatic calculation of disproportionality

Problem Solving Model 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

Step 1: Problem Identification 1. Problem Identification Is there a problem? Problem-solving approach Use valid & reliable metrics Quantify the difference between current outcomes and goals 62% of students have 0-1 ODR (at least 80% recommended) 38% of students have 2+ ODRs For disproportionality Quantify outcomes across racial/ethnic groups Compare differences Racial Subgroups vs. White Multiple metrics are recommended

Step 1: Problem Identification Common Metrics Risk Index Percent of a group at risk for an outcome (e.g., receiving an ODR) Number of Black Students with 1 or more ODR Number of Black Students Enrolled =.59 = 41 70

SWIS Ethnicity Reports Referral Risk Index

Step 1: Problem Identification Common Metrics Risk Ratio Risk index for one group divided by the risk index for comparison group Comparison group is usually White students 1.0 is equal risk > 1.0 is overrepresentation < 1.0 is underrepresentation Risk Index of Target Group Risk Index of Comparison Group Risk Index of Latino Students Risk Index of White Students = 1.27 Risk Index of Black Students Risk Index of White Students = 0.91

Step 1: Problem Identification Common Metrics Risk Ratio Free Risk Ratio Calculator from Wisconsin RtI CenterRisk Ratio Calculator

Step 1: Problem Identification Common Metrics Composition – Percent of Students with Referrals Compares subgroup’s percentage of school population to the subgroup’s percentage of just the students with ODRs Is the number students with ODRs for one subgroup proportionate to that subgroup’s size within the school?

SWIS Ethnicity Reports Students with Referrals by Ethnicity

Step 1: Problem Identification Common Metrics Composition – Percent of Total Referrals Compares subgroup’s percentage of school population to the subgroup’s percentage of all ODRs written Is the number of ODRs for one subgroup proportionate to that subgroup’s size within the school?

SWIS Ethnicity Reports Referrals by Ethnicity

Step 1: Problem Identification Procedure 1.Select metrics to use 2.Calculate metrics and compare to goals Previous years from same school Local or national norms U.S. public schools using SWIS with at least 10 African American and 10 White students Median risk ratio (African American to White) = th percentile = 1.38 Logical criteria U.S. Equal Employment Opportunity Commission (EEOC) Disparate impact criterion – “4/5s Rule” Goal risk ratio range between.80 and 1.25

School Example: Rainie Middle School School-wide Information System (SWIS) Calculate risk indices Calculate risk ratios African American = 3.2 (significant) Use US EEOC risk ratio goal range Latino/a = 1.1

Step 2: Problem Analysis 2. Problem Analysis 1. Problem Identification Is there a problem? Why is it happening?

Step 2: Problem Analysis Purpose: Identify underlying causes of the problem Focus: Systems & practices that can be changed Evaluate: Is the disproportionality identified consistent across all situations or more pronounced in some situations? Explicit bias vs. Implicit bias Disproportionality in all settings vs. Specific settings

Implicit Bias Unconscious, automatic Based on stereotypes We all have it (even those affected by it) Generally not an indication of our beliefs and values More likely to influence: Snap decisions Decisions that are ambiguous

A Unidimensional View of Bias Racial Bias Disproportionate Discipline

A Multidimensional View of Bias Racial Bias Disproportionate Discipline Situation Vulnerable Decision Point

What is a Vulnerable Decision Point A specific decision that is more vulnerable to effects of implicit bias Two parts: Elements of the situation The person’s decision state (internal state)

Two Systems for Decision Making  System 1: Fast Decisions Automatic, snap judgments Intuitive, unconscious  System 2: Slow Decisions Deliberate decisions Allows for conscious attention Kahneman, 2011

Subjective problem behavior Defiance, Disrespect, Disruption Major vs. Minor Non-classroom areas Hallways Playgrounds Afternoons VDPs from national ODR data

Step 2: Problem Analysis Purpose: Identify underlying causes of the problem Focus: Systems and practices that can be changed

Step 2: Problem Analysis Defining Disproportionate Discipline with Precision Who is involved? What are the problem behaviors? Where is it happening? When is it happening? Why are these things happening? Perceived function of problem behavior

Step 2: Problem Analysis Identified Subgroup Location Time of Day Problem Behavior Motivation Many sixth grade Latino/a students in the 7 th grade are more likely to receive referrals from the classroom, cafeteria, & commons for inappropriate language and physical aggression. Referrals are perceived to be task avoidance and getting adult attention. Assess PBIS implementation fidelity Achievement gap Vulnerable Decision Point (VDP)

SWIS Ethnicity Reports & Drill Down Total enrollment is approx. 25% Latino, but almost 29% of the total referrals are given to Latino/a students. Total enrollment is approx. 25% Latino, but of total number of students with referrals 30% of the them are Latino/a. There are 123 Latino/a students enrolled and 101 of them have been given a referral. With a risk index of 0.82, 82% of Latino/a students are at risk for or have already received a referral.

SWIS Drill Down Precise Problem Statement

SWIS Drill Down Subgroup: Latino/a Students Who?When?What?Where?Why? 3 rd gradeAfter 12:00 PM M-Disruption, Inappropriate Language, Defiance All settingsAvoid task 4 th grade8:00 AM– 11:30 AMM-DefianceClassroomAvoid task 7 th gradeAfter 12:00 PM Inappropriate Lang., Physical Aggression, M-Inappropriate Lang. Classroom Cafeteria Commons Avoid task Obtain Adult Attention

Precise Problem Statements  Latino/a students in the 3 rd grade are more likely to receive referrals in the afternoon across all settings for disrespect, inappropriate language, and disruption. Referrals are related to task avoidance.  Latino/a students in the 4 th grade are more likely to receive referrals from the classroom during the morning instructional block for defiance. Referrals are related to task avoidance.  Latino/a students in the 7 th grade are more likely to receive referrals from the classroom, cafeteria, & commons for inappropriate language and physical aggression. Referrals are related to task avoidance and getting adult attention.

School Example: Rainie Middle School 1.Use SWIS Drill Down to build precise problem statements 2.Assess PBIS implementation 3.Assess academic gaps RMS PBIS Team Meeting Decisions

Identified Subgroup Location Time of Day Problem Behavior Motivation Referrals are perceived to be related to getting peer and adult attention. Vulnerable Decision Point (VDP) School Example: Rainie Middle School Latino/a 6 th grade students are receiving referrals throughout the day in the classroom, cafeteria, & commons for inappropriate language and dress code violations.

Assess PBIS implementation TFI shows Tiers I, II, and III meet benchmark Team still wants to work on improving ODR process Major vs Minor Office vs Staff Tiered Fidelity Inventory (TFI) School Example: Rainie Middle School

Assess academic gaps African American and Latino/a students lagging in reading proficiency. School Example: Rainie Middle School

Step 3: Plan Implementation 2. Problem Analysis 3. Plan Implementation 1. Problem Identification Is there a problem? Why is it happening? What should be done?

Step 3: Plan Implementation Information from Step 2 is used to select strategies. An action plan is created to ensure adequate implementation of the strategies. Action plans show everyone - WHO will do WHAT by WHEN.

Step 3: Plan Implementation Options All issues Calculate and share disproportionality data regularly Inadequate PBIS implementation Implement core features of PBIS to establish a foundation of support Misunderstandings regarding school-wide expectations Enhance culturally-responsive PBIS with input from the students/families Academic achievement gap Implement effective academic instruction Disproportionality across all settings (indicating explicit bias) Enact strong anti-discrimination policies that include accountability Disproportionality in specific settings (indicating implicit bias) Neutralize vulnerable decision points

School Example: Rainie Middle School

Step 4: Plan Evaluation 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

Step 4: Plan Evaluation Evaluation Time Frame: Identify time periods for evaluating disproportionality data Caution: Disproportionality metrics may not be sensitive to rapid change Consider monthly assessment of implementation & quarterly assessment of disproportionality metrics Avoid using risk indices as they will increase throughout the year

Step 4: Plan Evaluation 1.Assess progress and fidelity of plan implementation 2.Calculate metrics from Step 1 3.Compare to the goal determined in Step 1 4.Share results with relevant stakeholders

School Example: Rainie Middle School 6th grade team may need a refresher on office vs. staff-managed behaviors Revise action plan for next year Continue evaluation cycle

Problem Solving Model 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

Bert Eliason & Katie Conley PBIS Applications NWPBIS Network Conference 2016 Data-driven Conversations on Equity & Disproportionality Use your available tools and resources to make sure you are offering all students their best chance for success.