Enhancing Equity in School Discipline 1: Using Discipline Data to Assess and Address Disproportionality Kent McIntosh Kelsey Morris University of Oregon Handouts: http://pbis.sccdoe.org

Acknowledgements PBIS Center Disproportionality Workgroup Aaron Barnes Don Kincaid Alondra Canizal Delabra Tim Lewis Yolanda Cargile Kent McIntosh Erin Chaparro Kelsey Morris Tai Collins Rhonda Nese Bert Eliason Vicki Nishioka Erik Girvan Heidi von Ravensberg Milaney Leverson Jen Rose Steve Goodman Russ Skiba Clynita Grafenreed Kent Smith Ambra Green Keith Smolkowski Rob Horner

Who is the audience for the presentations? Equity 1: Data (Session B) District and school teams External coaches Equity 2: Implicit Bias (Session C) School staff Those supporting school staff (coaches, teams, etc.)

Starting Questions: How much do you agree? I am aware of my personal biases. I am concerned about the consequences of bias in education. I have effective strategies for reducing bias in educational decisions. (Devine et al., 2012)

Overview of Today’s Session Describe the challenge of disproportionality in school discipline Share an intervention approach for enhancing equity in school discipline Guide you though a process for using data to: Assess levels of disproportionality Identify causes and solutions Plan to monitor progress Focus on skills/strategies: we in higher Ed are lousy at giving you the tools to manage behavior. Learn on the job. Handouts: http://www.pbis.org

Disproportionality in School Discipline (Losen et al., 2015)

Addressing Common Questions “Isn't it all really about poverty?” Poverty plays a role, but racial disproportionality remains, even when controlling for poverty American Psychological Association, 2008 Skiba et al., 2005 Wallace et al., 2008 Thank dissenters for courage in raising a point that many others may be thinking.

Addressing Common Questions “Aren’t Black boys just more violent?” No evidence of different base rates of behavior for any subgroups Bradshaw et al., 2010 Losen & Skiba, 2010 Skiba et al., 2014

Addressing Common Questions “Are you saying that all teachers are racist?” No! Our research indicates that disproportionality comes from unconscious bias – that we’re not even aware of. Banaji & Greenwald, 2013 Greenwald & Pettigrew, 2014 van den Bergh et al., 2010

A few assumptions… We are aware of the extent of disproportionality We are committed to enhancing equity in school discipline This work is uncomfortable There are a wide range of approaches that could work to enhance equity Raise your hand if you are not committed to enhancing equity? Will share some in the second half

A 5-point Intervention Approach to Enhance Equity in School Discipline There are no silver bullets “For every complex problem, there is a simple solution that won’t work.” (derived from H.L. Menken) Honor what you are already doing – we often throw everything at the problem http://www.pbis.org/school/equity-pbis

5-point Intervention Approach Use engaging academic instruction to reduce the achievement gap Implement a behavior framework that is preventive, multi-tiered, and culturally responsive Collect, use, and report disaggregated student discipline data Develop policies with accountability for disciplinary equity Teach neutralizing routines for vulnerable decision points Quaiity academic instruction Invest in systems that are proactive and flexible to be tailored to values and needs of students and families As we’ll see Not discussing this pm – ask me later Go into detail in this session http://www.pbis.org/school/equity-pbis

1. Why a focus on engaging academic instruction? Teacher presents student with grade level academic task Student’s academic skills do not improve Student engages in problem behavior Achievement gap exacerbates the discipline gap Student escapes academic task Teacher removes academic task or removes student (McIntosh et al., 2008)

What do we mean by engaging academic instruction? Explicit instruction High rates of opportunities to respond Quality performance feedback Progress monitoring and data-based decision making (Hattie, 2009)

Effects of Engaging Instruction on the Achievement Gap Tigard-Tualatin School District (Chaparro, Helton, & Sadler, in press)

2. Why start with a foundation of SWPBIS? Proactive, instructional approach may prevent problem behavior and exposure to biased responses to problem behavior Increasing positive student-teacher interactions may enhance relationships to prevent challenges More objective referral and discipline procedures may reduce subjectivity and influence of cultural bias Professional development may provide teachers with more instructional responses Teach hidden curriculum – avoid assumicide Ambiguity is disproportionality’s best friend “You can’t punish skills into a child” (Greflund et al., 2014)

Effects of SWPBIS on Discipline Disproportionality (Vincent, Swain-Bradway, Tobin & May, 2011) 72 in blue 81 in red Attend to elements of CR: Implement in partnership with students and families Assess areas of cultural differences in expectations Imaging

How inviting are we for all?

Which SWPBIS Features are Most Related to Equity Which SWPBIS Features are Most Related to Equity? (Tobin & Vincent, 2011) Examined change in Black-White Relative Risk Index for suspensions in 46 schools Two key predictors of decreased disproportionality: Regular use of data for decision making Implementation of classroom SWPBIS systems

Which features predicted decreased disproportionality? Expected behaviors defined clearly Problem behaviors defined clearly Expected behaviors taught Expected behaviors acknowledged regularly Consistent consequences CW procedures consistent with SW systems Options exist for instruction Instruction/materials match student ability High rates of academic success Access to assistance and coaching Transitions are efficient and orderly

Culturally Responsive SWPBIS Implementation Ensure equitable access to praise and acknowledgment systems Develop and revise school-wide systems with active involvement of families, students, and the community Use regular student and family surveys to assess acceptability and fit Target shrinking Expectations- where is it that students must be silent/not loud? It's a preference we have for a reason

Student Input & Satisfaction Survey Target shrinking Expectations- where is it that students must be silent/not loud? It's a preference we have for a reason

PBIS Cultural Responsiveness Companion Aligned directly with SWPBIS Tiered Fidelity Inventory (TFI) Tier I Scale Identifies SWPBIS critical feature Identifies cultural responsiveness concept Provides non-examples, examples, activities, and resources

PBIS Cultural Responsiveness Companion http://tinyurl.com/ncn8fmf

3. Using disaggregated data to assess and address equity Disproportionality Data Guide http://www.pbis.org/school/equity-pbis

4. Implement policies with accountability for equity How could policy work fit in to enhancing equity? Could highlight a common priority Could reduce effects of explicit bias Could enable implementation of other aspects of equity interventions Could reduce use of discriminatory practices

What does not work in policy Enacting policies that nobody knows about Enacting policies that don’t change practice Policies without accountability for implementation

Equity Policy Recommendations Include a Specific Commitment to Equity Create mission statements that include equity Enact hiring preferences for equitable discipline Install Effective Practices Require clear, objective school discipline procedures Support implementation of proactive, positive approaches to discipline Replace exclusionary practices w/ instructional ones Create Accountability for Efforts Create teams and procedures to enhance equity Share disproportionality data regularly Build equity outcomes into evaluations

5. How can we reduce implicit bias in our decision making? Stay tuned (next session…) no change to his or her attitudes or beliefs, an individual may selectively show racial bias in different decision situations. For example, a teacher may make more equitable discipline decisions at the start of the day but be more likely to send students of color to the office at the end of the day More accurate predictor of biased decision making AND

Discipline Data Systems Needs Required features: Consistent entry of ODR data and student race/ethnicity School enrollment by race/ethnicity Instantaneous access for school teams (not just district teams) Capability to disaggregate ODRs and patterns by race/ethnicity

Discipline Data Systems Needs Recommended features: Standardized ODR forms with a range of fields (e.g., location, time of day, consequence) Clear definitions of problem behaviors Clear guidance in discipline procedures (e.g., office vs. staff-managed) Instantaneous graphing capability Capability to show graphs by race/ethnicity Automatic calculation of disproportionality data

Discipline Data Systems Needs The School-Wide Information System (SWIS) meets these criteria Available at http://www.pbisapps.org

Worksheet Activity What data sources will you be using? Options: School-level data systems (e.g., SWIS) Which school(s)? State-level data systems Nothing? Use the SWIS demo account data Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Discipline Data System Example SWIS Demo Data: http://www.pbisapps.org

Discipline Data System Example

General Problem Solving Model 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? Is the plan working? What should be done?

Step 1: Problem Identification 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem?

Step 1: Problem Identification General problem-solving approach: Use valid and reliable metrics that assess outcomes of concern Quantify the difference between current outcomes and expected outcomes (goals) For disproportionality: Compare outcomes (e.g., discipline rates) across racial/ethnic groups (e.g., Black vs. White) Quantify these differences Multiple metrics are recommended (IDEA Data Center, 2014)

Step 1: Problem Identification Common Metrics Risk Index Percent of a group that receives an ODR or suspension (i.e., risk for that outcome) # of students with 1+ ODRs # of students in the group # of Enrolled Students # of Students With Referrals % of Students Within Ethnicity With Referrals Risk Index Native 5 2 40.00% 0.4 Asian 21 10 47.62% 0.48 Black 70 42 60.00% 0.6 Latino 123 101 82.11% 0.82 Pacific 3 White 255 165 64.71% 0.65 Unknown 0.00% Not Listed Multi-racial 14 66.67% 0.67 Totals: 500 337 Disproportionality may be hidden if only one metric (i.e., way of counting data) is used. For example, different groups of students may have the same overall risk of receiving ODRs (risk index), but a specific group who receives ODRs may receive many more than others (composition). As a result, it is important to examine multiple metrics instead of just one (IDEA Data Center, 2014). # of Latino/a students with 1+ ODRs # of Latino/a students enrolled # of White students with 1+ ODRs # of White students enrolled

Step 1: Problem Identification Common Metrics Risk Ratio Risk index for one group divided by risk index for comparison group (usually White students) 1.0 is equal risk > 1.0 is overrepresentation < 1.0 is underrepresentation It is possible to use Excel spreadsheets to quickly calculate risk ratios like this example from the Wisconsin PBIS Network. Available for free at http://goo.gl/mNcgVs Risk Index of Target Group Risk Index of Comparison Group Risk Index of Latino/a Students Risk Index of White Students .82 .65 = 1.27

Step 1: Problem Identification Common Metrics Risk Ratio Risk index for one group divided by risk index for comparison group (usually White students) 1.0 is equal risk > 1.0 is overrepresentation < 1.0 is underrepresentation Available for free at http://goo.gl/mNcgVS It is possible to use Excel spreadsheets to quickly calculate risk ratios like this example from the Wisconsin PBIS Network. Available for free at http://goo.gl/mNcgVs Risk Index of Target Group Risk Index of Comparison Group Risk Index of Latino/a Students Risk Index of White Students .82 .65 = 1.27

Step 1: Problem Identification Common Metrics Composition/Difference in Composition Compares proportion of students in a group to the proportion of ODRs from the same group Assesses whether the number of ODRs from one group is proportionate to the group’s size Composition metrics are a useful addition because in some cases, risk indices and ratios may show that a similar percent of each group has received an ODR, but students from a specific group with ODRs may receive many more than students from other groups.

Step 1: Problem Identification Procedure Select metrics to use Calculate metrics and compare to goals Previous years from same school Local or national norms 2011-2012 U.S. public schools using SWIS with at least 10 Black and 10 White students Median Black-White ODR risk ratio = 1.84 (25th percentile = 1.38) Logical criteria U.S. Equal Employment Opportunity Commission (EEOC) Disparate impact criterion (goal risk ratio range between .80 and 1.25) Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

School Example: Rainie Middle School School-wide Information System (SWIS) Metric: risk ratio Goal: All groups with a risk ratio < 1.25 STEPS Calculate risk indices Calculate risk ratios African American = 3.2 (significant) Latino/a = 1.1

Step 1: Worksheet Activity Complete STEP 1 (pp. 1-2) Select metrics to use Calculate metrics Compare to goals Reflect on your data: to what extent is there a problem? How clear are you on these steps? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Step 1: Worksheet Activity Share out… How were the steps? What metrics did you select? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Step 2: Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening?

Step 2: Problem Analysis General problem-solving approach: Identify underlying causes of the problem Focus on variables that can be changed For disproportionality: Identify whether disproportionality is consistent across all situations or more pronounced in some situations Assess other causes, such as: Achievement gap Fidelity of implementation of discipline or equity interventions

Identifying Patterns of Explicit vs. Implicit Bias Explicit Bias (conscious) Pattern: Consistent disproportionality across all situations Implications: Address through strong policy, regular reporting of data, and accountability for change Implicit Bias (unconscious) Pattern: Peaks and valleys of disproportionality depending on the situation Implications: Clarify discipline procedures, provide strategies for decision making

What is a Vulnerable Decision Point (VDP)? A specific decision that is more vulnerable to effects of implicit bias Two parts: Elements of the situation The person’s decision state (internal state)

Options for Identifying VDPs for Intervention Levels of specificity: All ODR/suspension decisions (general self-instruction routine) Identify VDPs through national data Use school or district data

National SWIS Data (2011-12) 3,026,367 ODRs 6,269 schools 47 states, plus DC

Office Referrals by Problem Behavior “Ambiguity is disproportionality’s best friend” 2 kinds of decisions

Office Referrals by Location

Office Referrals by Time of Day

VDPs from national ODR data ambiguity Subjective problem behavior Defiance, Disrespect, Disruption Major vs. minor Non-classroom areas Hallways Afternoons LACK OF contact “Ambiguity is disproportionality’s best friend” fatigue

SWIS Drill Down (www.swis.org) Add demographic group of interest as a filter (click to “Include in Dataset”). Click each graph and compare to overall patterns. Trainer Note: The SWIS Dashboard can help schools/facilities identify if a problem or trend exists. The SWIS Dashboard cannot tell you where the connections are. Users need to use the Drill Down feature to explore and identify what information is connected. This example shows how to take an ethnicity and drill down to better analyze what it is happening, when it’s likely to happen, and who is likely to engage in the problem behavior. This level of analysis can be completed by changing the graph type. Latino students in this example school are more likely to receive a referral in comparison to their peers. The existence of disproportionality is likely caused by multiple and complex factors that are unique to the particular school environment. The data-mining capability that exists within SWIS allows schools to dig deeper in their data analysis to better identify discipline referral patterns within overrepresented racial/ethnic groups. A deeper level of data analysis allows schools to examine if their current behavioral systems, practices, definitions, expectations are culturally relevant. Do students of a particular racial/ethnic group understand the school-wide system. Is it relevant to their culture? Culture is defined more broadly than race alone. An individual’s culture is more than just the group with which an individual identifies. Culture is made up of: Local contextual factors Community values Customs If a school’s PBIS system is not culturally relevant to a particular group, the school can reach out to the larger community for assistance in making it culturally relevant and compatible. By working to include a particular group’s culture within the school-wide culture, the school is being culturally responsive. If a school’s current practices are culturally relevant and disproportionality still exists, schools can collect additional information that evaluate the systems and staff behavior.

School Example: Rainie Middle School Assess PBIS implementation TFI indicates successful implementation Improve office vs. staff-managed systems Improve consequence systems SWIS Drill Down for precise problem statement Precise Problem Statement = African American students are more likely to receive ODRs in the classroom for inappropriate language and dress code violations. Most likely to occur before lunch and are related to peer attention. Tiered Fidelity Inventory (TFI)

Step 2: Worksheet Activity Complete STEP 2 (pp. 3-4) Assess PBIS fidelity Identify vulnerable decision points Assess achievement gap How clear are you on these steps? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Stage 1 Behavior: White Students

Stage 1 Behavior: Black Students

Stage 1 Location: White Students

Stage 1 Location: Black Students

Stage 1 Time: White Students

Stage 1 Time: Black Students

Stage 1 Grade: White Students

Stage 1 Grade: Black Students

Step 2: Worksheet Activity Share out… How were the steps? What did you find? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Step 3: Plan Implementation 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? What should be done?

Step 3: Plan Implementation General problem-solving approach: Use the information from Step 2 (Problem Analysis) to select strategies Create a plan to ensure adequate implementation of the strategies For disproportionality: No differences from general approach

Step 3: Plan Implementation Options All issues Calculate and share disproportionality data regularly Inadequate PBIS implementation Implement core features of PBIS to establish a foundation of support Clarify ODR definitions and procedures Misunderstandings regarding school-wide expectations Enhance cultural responsiveness of PBIS with input from families, students, and community Significant academic achievement gap Use effective academic instruction Disproportionality across all settings (indicating explicit bias) Enact strong equity policies that include accountability Disproportionality in specific settings (indicating implicit bias) Teach neutralizing routines for vulnerable decisions

School Example: Rainie Middle School

Step 3: Worksheet Activity Complete STEP 3 (pp. 5-6) Identify strategies to implement Create a detailed action plan How clear are you on these steps? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Step 3: Worksheet Activity Share out… How were the steps? What are you planning… Now? After that? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

1. Problem Identification Step 4: Plan Evaluation 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? Is the plan working? What should be done?

Step 4: Plan Evaluation General problem-solving approach: Assess whether the plan is implemented Calculate metrics (from Step 1) regularly Compare outcomes to predetermined goals For disproportionality: Disproportionality metrics may not be as sensitive to rapid change as other measures Consider monthly assessment of implementation and quarterly assessment of disproportionality metrics Avoid using risk indices (will rise throughout year)

Step 4: Plan Evaluation Procedure Identify the time periods for evaluating disproportionality data Assess fidelity of plan implementation Calculate metrics selected in Step 1 Compare to the goal determined in Step 1 Share results with relevant stakeholders

School Example: Rainie Middle School 6th grade team may need a refresher on office vs. staff-managed behaviors Revise action plan for next year Continue evaluation cycle

Step 3: Worksheet Activity Complete STEP 4 (p. 7) Identify the time periods for evaluation (complete later) How clear are you on these steps? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Step 4: Worksheet Activity Share out… How were the steps? What are your time periods going to be? Once metrics are calculated, the next step is to compare these numbers to a criterion. This step can be challenging because there is no federal definition of what constitutes disproportionality, so each state sets its own criteria. One important caution when considering comparison is the number of students in each group. When there are fewer than 10 students in a particular group, smaller changes on ODRs or suspensions may inflate the disproportionality metrics (U.S. Government Accountability Office, 2013).

Big Ideas Disproportionality in school discipline is one of the biggest challenges in education today We can use data to assess and monitor how we are doing If you don’t have the data you need at hand, advocate for it The same steps we have for solving discipline problems work for disproportionality

Contact Information Kent McIntosh Special Education Program University of Oregon kentm@uoregon.edu @_kentmc Cannon Beach, Oregon © GoPictures, 2010 Handouts: http://kentmcintosh.wordpress.com

References Greflund, S., McIntosh, K., Mercer, S. H., & May, S. L. (2014). Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS. Canadian Journal of School Psychology, 29, 213-235. McIntosh, K., Barnes, A., Morris, K., & Eliason, B. M. (2014). Using discipline data within SWPBIS to identify and address disproportionality: A guide for school teams. Eugene, OR: Center on Positive Behavioral Interventions and Supports. University of Oregon. McIntosh, K., Girvan, E. J., Horner, R. H., Smolkowski, K., & Sugai, G. (2014). Recommendations for addressing discipline disproportionality in education. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. Vincent, C. G., Swain-Bradway, J., Tobin, T. J., & May, S. (2011). Disciplinary referrals for culturally and linguistically diverse students with and without disabilities: Patterns resulting from school-wide positive behavior support. Exceptionality, 19, 175-190.