1. DO WE SAY “ALL” BUT MEAN “SOME”? Data-driven Conversations on Equity & Disproportionality Bert Eliason & Katie Conley PBIS Applications NWPBIS Network Conference 2016 Portland – Red Lion Jantzen Beach

2. Goal Use a four-step problem-solving model to address discipline disproportionality Resources Data Guide for School Teams (pbis.org, 2015) Four-step problem-solving model SWIS Ethnicity Report Organizing and Analyzing the Data Data Sources Common Metrics Drill Down Process Putting It Into Practice School Example Session Intentions

3. Educational and Community Supports (ECS) is a research unit within the College of Education at the University of Oregon. ECS focuses on the development and implementation of practices that result in positive, durable, and scientifically substantiated change in the lives of individuals. Federal and state funded projects support research, teaching, dissemination, and technical assistance. PBIS Applications (PBISApps) is a series of educational tools created within ECS and related to the implementation of multi-tiered systems of support (MTSS). The PBIS Application tools have been utilized in 25,000+ schools both domestically and internationally. Educational and Community Supports

4. www.pbis.org School  Equity & PBIS

5. Data Sources: What is Necessary? Required features: Consistent data collection Discipline referrals (ODRs) Student race/ethnicity School enrollment by race/ethnicity Instantaneous access for school teams—not just district teams Capability to disaggregate ODRs and patterns by race/ethnicity Capability to calculate risk indices and risk ratios by race/ethnicity

6. Data Sources: What is Recommended? Standardized forms with the 5W’s Who, what, when, where, why Clear definitions of problem behaviors Clear guidance in discipline procedures Office vs. staff-managed Report generation On demand Disaggregated by race/ethnicity Automatic calculation of disproportionality

7. Problem Solving Model 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

8. Step 1: Problem Identification 1. Problem Identification Is there a problem? Problem-solving approach Use valid & reliable metrics Quantify the difference between current outcomes and goals 62% of students have 0-1 ODR (at least 80% recommended) 38% of students have 2+ ODRs For disproportionality Quantify outcomes across racial/ethnic groups Compare differences Racial Subgroups vs. White Multiple metrics are recommended

9. Step 1: Problem Identification Common Metrics Risk Index Percent of a group at risk for an outcome (e.g., receiving an ODR) Number of Black Students with 1 or more ODR Number of Black Students Enrolled =.59 = 41 70

10. SWIS Ethnicity Reports Referral Risk Index

11. Step 1: Problem Identification Common Metrics Risk Ratio Risk index for one group divided by the risk index for comparison group Comparison group is usually White students 1.0 is equal risk > 1.0 is overrepresentation < 1.0 is underrepresentation Risk Index of Target Group Risk Index of Comparison Group Risk Index of Latino Students Risk Index of White Students.82.65 = 1.27 Risk Index of Black Students Risk Index of White Students.59.65 = 0.91

12. Step 1: Problem Identification Common Metrics Risk Ratio Free Risk Ratio Calculator from Wisconsin RtI CenterRisk Ratio Calculator www.wisconsinrticenter.org

13. Step 1: Problem Identification Common Metrics Composition – Percent of Students with Referrals Compares subgroup’s percentage of school population to the subgroup’s percentage of just the students with ODRs Is the number students with ODRs for one subgroup proportionate to that subgroup’s size within the school?

14. SWIS Ethnicity Reports Students with Referrals by Ethnicity

15. Step 1: Problem Identification Common Metrics Composition – Percent of Total Referrals Compares subgroup’s percentage of school population to the subgroup’s percentage of all ODRs written Is the number of ODRs for one subgroup proportionate to that subgroup’s size within the school?

16. SWIS Ethnicity Reports Referrals by Ethnicity

17. Step 1: Problem Identification Procedure 1.Select metrics to use 2.Calculate metrics and compare to goals Previous years from same school Local or national norms 2011-2012 U.S. public schools using SWIS with at least 10 African American and 10 White students Median risk ratio (African American to White) = 1.84 25 th percentile = 1.38 Logical criteria U.S. Equal Employment Opportunity Commission (EEOC) Disparate impact criterion – “4/5s Rule” Goal risk ratio range between.80 and 1.25

18. School Example: Rainie Middle School School-wide Information System (SWIS) Calculate risk indices Calculate risk ratios African American = 3.2 (significant) Use US EEOC risk ratio goal range Latino/a = 1.1

19. Step 2: Problem Analysis 2. Problem Analysis 1. Problem Identification Is there a problem? Why is it happening?

20. Step 2: Problem Analysis Purpose: Identify underlying causes of the problem Focus: Systems & practices that can be changed Evaluate: Is the disproportionality identified consistent across all situations or more pronounced in some situations? Explicit bias vs. Implicit bias Disproportionality in all settings vs. Specific settings

21. Implicit Bias Unconscious, automatic Based on stereotypes We all have it (even those affected by it) Generally not an indication of our beliefs and values More likely to influence: Snap decisions Decisions that are ambiguous

22. A Unidimensional View of Bias Racial Bias Disproportionate Discipline

23. A Multidimensional View of Bias Racial Bias Disproportionate Discipline Situation Vulnerable Decision Point

24. What is a Vulnerable Decision Point A specific decision that is more vulnerable to effects of implicit bias Two parts: Elements of the situation The person’s decision state (internal state)

25. Two Systems for Decision Making  System 1: Fast Decisions Automatic, snap judgments Intuitive, unconscious  System 2: Slow Decisions Deliberate decisions Allows for conscious attention Kahneman, 2011

26. Subjective problem behavior Defiance, Disrespect, Disruption Major vs. Minor Non-classroom areas Hallways Playgrounds Afternoons VDPs from national ODR data

27. Step 2: Problem Analysis Purpose: Identify underlying causes of the problem Focus: Systems and practices that can be changed

28. Step 2: Problem Analysis Defining Disproportionate Discipline with Precision Who is involved? What are the problem behaviors? Where is it happening? When is it happening? Why are these things happening? Perceived function of problem behavior

29. Step 2: Problem Analysis Identified Subgroup Location Time of Day Problem Behavior Motivation Many sixth grade Latino/a students in the 7 th grade are more likely to receive referrals from the classroom, cafeteria, & commons for inappropriate language and physical aggression. Referrals are perceived to be task avoidance and getting adult attention. Assess PBIS implementation fidelity Achievement gap Vulnerable Decision Point (VDP)

30. SWIS Ethnicity Reports & Drill Down Total enrollment is approx. 25% Latino, but almost 29% of the total referrals are given to Latino/a students. Total enrollment is approx. 25% Latino, but of total number of students with referrals 30% of the them are Latino/a. There are 123 Latino/a students enrolled and 101 of them have been given a referral. With a risk index of 0.82, 82% of Latino/a students are at risk for or have already received a referral.

31. SWIS Drill Down Precise Problem Statement

32. SWIS Drill Down Subgroup: Latino/a Students Who?When?What?Where?Why? 3 rd gradeAfter 12:00 PM M-Disruption, Inappropriate Language, Defiance All settingsAvoid task 4 th grade8:00 AM– 11:30 AMM-DefianceClassroomAvoid task 7 th gradeAfter 12:00 PM Inappropriate Lang., Physical Aggression, M-Inappropriate Lang. Classroom Cafeteria Commons Avoid task Obtain Adult Attention

33. Precise Problem Statements  Latino/a students in the 3 rd grade are more likely to receive referrals in the afternoon across all settings for disrespect, inappropriate language, and disruption. Referrals are related to task avoidance.  Latino/a students in the 4 th grade are more likely to receive referrals from the classroom during the morning instructional block for defiance. Referrals are related to task avoidance.  Latino/a students in the 7 th grade are more likely to receive referrals from the classroom, cafeteria, & commons for inappropriate language and physical aggression. Referrals are related to task avoidance and getting adult attention.

34. School Example: Rainie Middle School 1.Use SWIS Drill Down to build precise problem statements 2.Assess PBIS implementation 3.Assess academic gaps RMS PBIS Team Meeting Decisions

35. Identified Subgroup Location Time of Day Problem Behavior Motivation Referrals are perceived to be related to getting peer and adult attention. Vulnerable Decision Point (VDP) School Example: Rainie Middle School Latino/a 6 th grade students are receiving referrals throughout the day in the classroom, cafeteria, & commons for inappropriate language and dress code violations.

36. Assess PBIS implementation TFI shows Tiers I, II, and III meet benchmark Team still wants to work on improving ODR process Major vs Minor Office vs Staff Tiered Fidelity Inventory (TFI) School Example: Rainie Middle School

37. Assess academic gaps African American and Latino/a students lagging in reading proficiency. School Example: Rainie Middle School

38. Step 3: Plan Implementation 2. Problem Analysis 3. Plan Implementation 1. Problem Identification Is there a problem? Why is it happening? What should be done?

39. Step 3: Plan Implementation Information from Step 2 is used to select strategies. An action plan is created to ensure adequate implementation of the strategies. Action plans show everyone - WHO will do WHAT by WHEN.

40. Step 3: Plan Implementation Options All issues Calculate and share disproportionality data regularly Inadequate PBIS implementation Implement core features of PBIS to establish a foundation of support Misunderstandings regarding school-wide expectations Enhance culturally-responsive PBIS with input from the students/families Academic achievement gap Implement effective academic instruction Disproportionality across all settings (indicating explicit bias) Enact strong anti-discrimination policies that include accountability Disproportionality in specific settings (indicating implicit bias) Neutralize vulnerable decision points

41. School Example: Rainie Middle School

42. Step 4: Plan Evaluation 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

43. Step 4: Plan Evaluation Evaluation Time Frame: Identify time periods for evaluating disproportionality data Caution: Disproportionality metrics may not be sensitive to rapid change Consider monthly assessment of implementation & quarterly assessment of disproportionality metrics Avoid using risk indices as they will increase throughout the year

44. Step 4: Plan Evaluation 1.Assess progress and fidelity of plan implementation 2.Calculate metrics from Step 1 3.Compare to the goal determined in Step 1 4.Share results with relevant stakeholders

45. School Example: Rainie Middle School 6th grade team may need a refresher on office vs. staff-managed behaviors Revise action plan for next year Continue evaluation cycle

46. Problem Solving Model 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done? Is the plan working?

47. Bert Eliason & Katie Conley PBIS Applications training@pbisapps.org NWPBIS Network Conference 2016 Data-driven Conversations on Equity & Disproportionality Use your available tools and resources to make sure you are offering all students their best chance for success.