enhancing equity in school discipline 1: using discipline data to assess and address...
TRANSCRIPT
Enhancing Equity in School Discipline 1:
Using Discipline Data to Assess and Address Disproportionality
Kent McIntosh
Kelsey Morris
University of Oregon
Handouts:http://pbis.sccdoe.org
PBIS Center Disproportionality Workgroup
Acknowledgements
Aaron Barnes Alondra Canizal Delabra Yolanda Cargile Erin Chaparro Tai Collins Bert Eliason Erik Girvan Milaney Leverson Steve Goodman Clynita Grafenreed Ambra Green Rob Horner
Don Kincaid Tim Lewis Kent McIntosh Kelsey Morris Rhonda Nese Vicki Nishioka Heidi von Ravensberg Jen Rose Russ Skiba Kent Smith Keith Smolkowski
Equity 1: Data (Session B)District and school teamsExternal coaches
Equity 2: Implicit Bias (Session C)School staffThose supporting school staff
(coaches, teams, etc.)
Who is the audience for the presentations?
1. I am aware of my personal biases.
2. I am concerned about the consequences of bias in education.
3. I have effective strategies for reducing bias in educational decisions.
Starting Questions:How much do you agree?
(Devine et al., 2012)
1. Describe the challenge of disproportionality in school discipline
2. Share an intervention approach for enhancing equity in school discipline
3. Guide you though a process for using data to:1. Assess levels of disproportionality
2. Identify causes and solutions
3. Plan to monitor progress
Overview of Today’s Session
Handouts: http://www.pbis.org
Disproportionality in School Discipline (Losen et al., 2015)
Poverty plays a role, but racial disproportionality remains, even when controlling for povertyAmerican Psychological Association, 2008Skiba et al., 2005Wallace et al., 2008
Addressing Common Questions
“Isn't it all really about poverty?”
No evidence of different base rates of behavior for any subgroupsBradshaw et al., 2010Losen & Skiba, 2010Skiba et al., 2014
Addressing Common Questions
“Aren’t Black boys just more violent?”
No! Our research indicates that disproportionality comes from unconscious bias – that we’re not even aware of.Banaji & Greenwald, 2013Greenwald & Pettigrew, 2014van den Bergh et al., 2010
Addressing Common Questions
“Are you saying that all teachers are racist?”
We are aware of the extent of disproportionality
We are committed to enhancing equity in school discipline
This work is uncomfortable There are a wide range of approaches
that could work to enhance equity
A few assumptions…
A 5-point
Intervention
Approach to Enhance Equity in School Discipline
http://www.pbis.org/school/equity-pbis
1. Use engaging academic instruction to reduce the achievement gap
2. Implement a behavior framework that is preventive, multi-tiered, and culturally responsive
3. Collect, use, and report disaggregated student discipline data
4. Develop policies with accountability for disciplinary equity
5. Teach neutralizing routines for vulnerable decision points
5-point Intervention Approach
http://www.pbis.org/school/equity-pbis
Teacher presents student with grade level academic task
Teacher removes academic task or removes student
Student engages in problem behavior
Student escapes academic task
Student’s academic skills do not improve
1. Why a focus on engaging academic instruction?
(McIntosh et al., 2008)
Explicit instruction High rates of opportunities to respond Quality performance feedback Progress monitoring and data-based
decision making
What do we mean by engaging academic instruction?
(Hattie, 2009)
2007-08 2008-09 2009-10 2010-11 2011-12 2012-130%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
43% 47%36%
28% 24%11%
81% 84%88%
94% 91% 94%
38% 37%
52%
66% 67%
83%
White
Latino
Perc
ent M
eetin
g or
Exc
eedi
ng S
tand
ards
Effects of Engaging Instruction on the Achievement Gap
Tigard-Tualatin School District (Chaparro, Helton, & Sadler, in press)
1. Proactive, instructional approach may prevent problem behavior and exposure to biased responses to problem behavior
2. Increasing positive student-teacher interactions may enhance relationships to prevent challenges
3. More objective referral and discipline procedures may reduce subjectivity and influence of cultural bias
4. Professional development may provide teachers with more instructional responses
2. Why start with a foundation of SWPBIS?
(Greflund et al., 2014)
Effects of SWPBIS onDiscipline Disproportionality(Vincent, Swain-Bradway, Tobin & May, 2011)
200506 200607 2007080%
5%
10%
15%
20%
25%
SWPBISNo SWPBIS
Bla
ck-W
hite
Dis
cip
line
Ga
p
How inviting are we for all?
Examined change in Black-White Relative Risk Index for suspensions in 46 schools
Two key predictors of decreased disproportionality:Regular use of data for decision making Implementation of classroom SWPBIS
systems
Which SWPBIS Features are Most Related to Equity? (Tobin & Vincent, 2011)
Expected behaviors defined clearly Problem behaviors defined clearly Expected behaviors taught Expected behaviors acknowledged regularly Consistent consequences CW procedures consistent with SW systems Options exist for instruction Instruction/materials match student ability High rates of academic success Access to assistance and coaching Transitions are efficient and orderly
Which features predicted decreased disproportionality?
Ensure equitable access to praise and acknowledgment systems
Develop and revise school-wide systems with active involvement of families, students, and the community
Use regular student and family surveys to assess acceptability and fit
Culturally Responsive SWPBIS Implementation
Student Input &SatisfactionSurvey
Aligned directly with SWPBIS Tiered Fidelity Inventory (TFI) Tier I Scale Identifies SWPBIS critical feature Identifies cultural responsiveness conceptProvides non-examples, examples, activities,
and resources
PBIS Cultural Responsiveness Companion
Disproportionality Data Guide
3. Using disaggregated data to assess and address equity
http://www.pbis.org/school/equity-pbis
4. Implement policies with accountability for equity How could policy work fit in to enhancing
equity?Could highlight a common priorityCould reduce effects of explicit biasCould enable implementation of other
aspects of equity interventionsCould reduce use of discriminatory practices
Enacting policies that nobody knows about Enacting policies that don’t change
practice Policies without accountability for
implementation
What does not work in policy
Include a Specific Commitment to Equity Create mission statements that include equity Enact hiring preferences for equitable discipline
Install Effective Practices Require clear, objective school discipline procedures Support implementation of proactive, positive
approaches to discipline Replace exclusionary practices w/ instructional ones
Create Accountability for Efforts Create teams and procedures to enhance equity Share disproportionality data regularly Build equity outcomes into evaluations
Equity Policy Recommendations
Stay tuned (next session…)
5. How can we reduce implicit bias in our decision making?
Discipline Data Systems Needs Required features:
Consistent entry of ODR data and student race/ethnicity
School enrollment by race/ethnicity
Instantaneous access for school teams (not just district teams)
Capability to disaggregate ODRs and patterns by race/ethnicity
Discipline Data Systems Needs Recommended features:
Standardized ODR forms with a range of fields (e.g., location, time of day, consequence)
Clear definitions of problem behaviors
Clear guidance in discipline procedures (e.g., office vs. staff-managed)
Instantaneous graphing capability
Capability to show graphs by race/ethnicity
Automatic calculation of disproportionality data
The School-Wide Information System (SWIS) meets these criteriaAvailable at http://www.pbisapps.org
Discipline Data Systems Needs
What data sources will you be using? Options:
School-level data systems (e.g., SWIS) Which school(s)?
State-level data systemsNothing? Use the SWIS demo account data
Worksheet Activity
Discipline Data System Example
General Problem Solving Model
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Is there a problem?
Why is it happening?
What should be done?
Is the plan working?
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Step 1: Problem Identification
Is there a problem?
Step 1: Problem Identification General problem-solving approach:
Use valid and reliable metrics that assess outcomes of concern
Quantify the difference between current outcomes and expected outcomes (goals)
For disproportionality:Compare outcomes (e.g., discipline rates) across
racial/ethnic groups (e.g., Black vs. White)Quantify these differences Multiple metrics are recommended
(IDEA Data Center, 2014)
Step 1: Problem IdentificationCommon Metrics Risk Index
Percent of a group that receives an ODR or suspension (i.e., risk for that outcome)
# of Enrolled Students
# of Students With Referrals
% of Students Within Ethnicity With Referrals
Risk Index
Native 5 2 40.00% 0.4
Asian 21 10 47.62% 0.48
Black 70 42 60.00% 0.6
Latino 123 101 82.11% 0.82
Pacific 5 3 60.00% 0.6
White 255 165 64.71% 0.65
Unknown 0 0 0.00% 0
Not Listed 0 0 0.00% 0
Multi-racial
21 14 66.67% 0.67
Totals: 500 337
# of students with 1+ ODRs
# of students in the group
# of Latino/a students with 1+ ODRs
# of Latino/a students enrolled
# of White students with 1+ ODRs
# of White students enrolled
Step 1: Problem IdentificationCommon Metrics Risk Ratio
Risk index for one group divided by risk index for comparison group (usually White students)
1.0 is equal risk
> 1.0 is overrepresentation
< 1.0 is underrepresentation
Risk Index of Target Group
Risk Index of Comparison Group
Risk Index of Latino/a Students
Risk Index of White Students
.82
.65= 1.27
Step 1: Problem IdentificationCommon Metrics Risk Ratio
Risk index for one group divided by risk index for comparison group (usually White students)
1.0 is equal risk
> 1.0 is overrepresentation
< 1.0 is underrepresentation
Risk Index of Target Group
Risk Index of Comparison Group
Risk Index of Latino/a Students
Risk Index of White Students
.82
.65= 1.27
Available for free at http://goo.gl/mNcgVS
Step 1: Problem IdentificationCommon Metrics Composition/Difference in Composition
Compares proportion of students in a group to the proportion of ODRs from the same group
Assesses whether the number of ODRs from one group is proportionate to the group’s size
Step 1: Problem IdentificationProcedure
1. Select metrics to use2. Calculate metrics and compare to goals
• Previous years from same school• Local or national norms
• 2011-2012 U.S. public schools using SWIS with at least 10 Black and 10 White students
• Median Black-White ODR risk ratio = 1.84 (25th percentile = 1.38)
• Logical criteria• U.S. Equal Employment Opportunity Commission (EEOC)
• Disparate impact criterion (goal risk ratio range between .80 and 1.25)
School Example: Rainie Middle School
White Black Latino/a Asian0%
10%20%30%40%50%60%
48%
31%20%
10%
School Enrollment
School-wide Information System (SWIS)
• Metric: risk ratio• Goal: All groups with a
risk ratio < 1.25
STEPS
• Calculate risk indices
• Calculate risk ratios
• African American = 3.2 (significant)
• Latino/a = 1.1
1. Complete STEP 1 (pp. 1-2)1. Select metrics to use
2. Calculate metrics
3. Compare to goals
2. Reflect on your data: to what extent is there a problem?
How clear are you on these steps?
Step 1: Worksheet Activity
Share out…
How were the steps? What metrics did you select?
Step 1: Worksheet Activity
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Step 2: Problem Analysis
Is there a problem?
Why is it happening?
Step 2: Problem Analysis General problem-solving approach:
Identify underlying causes of the problemFocus on variables that can be changed
For disproportionality: Identify whether disproportionality is consistent
across all situations or more pronounced in some situations
Assess other causes, such as: Achievement gap Fidelity of implementation of discipline or equity
interventions
Explicit Bias (conscious) Pattern: Consistent disproportionality across all
situations Implications: Address through strong policy,
regular reporting of data, and accountability for change
Implicit Bias (unconscious) Pattern: Peaks and valleys of disproportionality
depending on the situation Implications: Clarify discipline procedures,
provide strategies for decision making
Identifying Patterns of Explicit vs. Implicit Bias
A specific decision that is more vulnerable to effects of implicit bias
Two parts:Elements of the situationThe person’s decision state (internal state)
What is a Vulnerable Decision Point (VDP)?
Levels of specificity:
1. All ODR/suspension decisions (general self-instruction routine)
2. Identify VDPs through national data
3. Use school or district data
Options for Identifying VDPs for Intervention
National SWIS Data(2011-12)
3,026,367 ODRs
6,269 schools
47 states, plus DC
Langu
age
Defian
ce/ Disr
espect
Disruption
Fighting
Forge
ry/ Th
eft
Harassm
ent/
bullying
Lying/
chea
ting
M - Defi
ance/
Disresp
ect
M - Disr
uption
M - Dres
s code
M - Lan
guag
e
M - Other
M - Physi
cal Contac
t
M - Pro
perty M
isuse
M - Tard
y0
5
10
15
20
25
5
21
9
51
31
11
7
1 2 3 31
3
Black
Langu
age
Defian
ce/ Disr
espect
Disruption
Fighting
Forge
ry/ Th
eft
Harassm
ent/
bullying
Lying/
chea
ting
M - Defi
ance/
Disresp
ect
M - Disr
uption
M - Dres
s code
M - Lan
guag
e
M - Other
M - Physi
cal Contac
t
M - Pro
perty M
isuse
M - Tard
y02468
1012141618
4
14
5
21
31
15
10
12
6 6
2
4
WhiteOffice Referrals by Problem Behavior
Art Room
Bathro
om/ Rest
room Bus
Bus Load
ing Zone
Cafeter
ia
Classro
om
Commons/ Common Area
s
Computer La
bGym
Hall/ B
reeze
way
Librar
y
Locke
r Room
Music Room
Off-Campus
Office
Other Lo
cation
Parking L
ot
Playgro
und
Speci
al Ev
ent/
Assembly/
Field
trip
Stadium
Unknown Lo
cation
Vocational
Room0
10
20
30
40
50
60
70
1 25
0
5
56
1 03
7
1 0 1 13 2
0
9
0 0 1 0
BlackWhite
Office Referrals by Location
12:00AM
12:30AM
1:00AM
1:30AM
2:00AM
2:30AM
3:00AM
3:30AM
4:00AM
4:30AM
5:00AM
5:30AM
6:00AM
6:30AM
7:00AM
7:30AM
8:00AM
8:30AM
9:00AM
9:30AM
10:00AM
10:30AM
11:00AM
11:30AM
12:00PM
12:30PM
1:00PM
1:30PM
2:00PM
2:30PM
3:00PM
3:30PM
4:00PM
4:30PM
5:00PM
5:30PM
6:00PM
6:30PM
7:00PM
7:30PM
8:00PM
8:30PM
9:00PM
9:30PM
10:00PM
10:30PM
11:00PM
11:30PM
0.01.02.03.04.05.06.07.08.09.0
Black
12:00AM
12:30AM
1:00AM
1:30AM
2:00AM
2:30AM
3:00AM
3:30AM
4:00AM
4:30AM
5:00AM
5:30AM
6:00AM
6:30AM
7:00AM
7:30AM
8:00AM
8:30AM
9:00AM
9:30AM
10:00AM
10:30AM
11:00AM
11:30AM
12:00PM
12:30PM
1:00PM
1:30PM
2:00PM
2:30PM
3:00PM
3:30PM
4:00PM
4:30PM
5:00PM
5:30PM
6:00PM
6:30PM
7:00PM
7:30PM
8:00PM
8:30PM
9:00PM
9:30PM
10:00PM
10:30PM
11:00PM
11:30PM
0.01.02.03.04.05.06.07.08.09.0
White
Office Referrals by Time of Day
Subjective problem behaviorDefiance, Disrespect, DisruptionMajor vs. minor
Non-classroom areasHallways
Afternoons
VDPs from national ODR data
ambiguit
y
LACK OF
contact
fatigue
SWIS Drill Down (www.swis.org)
Add demographic group of interest as a filter (click to
“Include in Dataset”).
Click each graph and compare to overall patterns.
• Assess PBIS implementation• TFI indicates successful
implementation
• Improve office vs. staff-managed systems
• Improve consequence systems
• SWIS Drill Down for precise problem statement
White Black Latino/a Asian0
20406080
100
7144 52
100
Reading
Met
Tiered Fidelity Inventory (TFI)
School Example: Rainie Middle School
1. Complete STEP 2 (pp. 3-4)1. Assess PBIS fidelity
2. Identify vulnerable decision points
3. Assess achievement gap
How clear are you on these steps?
Step 2: Worksheet Activity
Stage 1 Behavior: White Students
Stage 1 Behavior: Black Students
Stage 1 Location: White Students
Stage 1 Location: Black Students
Stage 1 Time: White Students
Stage 1 Time: Black Students
Stage 1 Grade: White Students
Stage 1 Grade: Black Students
Share out…
How were the steps? What did you find?
Step 2: Worksheet Activity
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Step 3: Plan Implementation
Is there a problem?
Why is it happening?
What should be done?
Step 3: Plan Implementation General problem-solving approach:
Use the information from Step 2 (Problem Analysis) to select strategies
Create a plan to ensure adequate implementation of the strategies
For disproportionality:No differences from general approach
Step 3: Plan ImplementationOptions All issues
Calculate and share disproportionality data regularly
Inadequate PBIS implementation Implement core features of PBIS to establish a foundation of support
Clarify ODR definitions and procedures
Misunderstandings regarding school-wide expectations Enhance cultural responsiveness of PBIS with input from families,
students, and community
Significant academic achievement gap Use effective academic instruction
Disproportionality across all settings (indicating explicit bias) Enact strong equity policies that include accountability
Disproportionality in specific settings (indicating implicit bias) Teach neutralizing routines for vulnerable decisions
School Example: Rainie Middle School
1. Complete STEP 3 (pp. 5-6)1. Identify strategies to implement
2. Create a detailed action plan
How clear are you on these steps?
Step 3: Worksheet Activity
Share out…
How were the steps? What are you planning…
Now? After that?
Step 3: Worksheet Activity
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Step 4: Plan Evaluation
Is there a problem?
Why is it happening?
What should be done?
Is the plan working?
Step 4: Plan Evaluation General problem-solving approach:
Assess whether the plan is implemented Calculate metrics (from Step 1) regularlyCompare outcomes to predetermined goals
For disproportionality:Disproportionality metrics may not be as
sensitive to rapid change as other measures Consider monthly assessment of implementation and
quarterly assessment of disproportionality metrics Avoid using risk indices (will rise throughout year)
Step 4: Plan EvaluationProcedure
1. Identify the time periods for evaluating disproportionality data
2. Assess fidelity of plan implementation
3. Calculate metrics selected in Step 1
4. Compare to the goal determined in Step 1
5. Share results with relevant stakeholders
October January March June0
0.51
1.52
2.53
3.54
Black-White ODR Risk Ratio
Risk Ratio
• 6th grade team may need a refresher on office vs. staff-managed behaviors
• Revise action plan for next year
• Continue evaluation cycle
School Example: Rainie Middle School
1. Complete STEP 4 (p. 7)1. Identify the time periods for evaluation
2. (complete later)
3. (complete later)
How clear are you on these steps?
Step 3: Worksheet Activity
Share out…
How were the steps? What are your time periods going to be?
Step 4: Worksheet Activity
Disproportionality in school discipline is one of the biggest challenges in education today
We can use data to assess and monitor how we are doing If you don’t have the data you need at hand,
advocate for it The same steps we have for solving
discipline problems work for disproportionality
Big Ideas
Contact Information
Kent McIntoshSpecial Education Program
University of Oregon
@_kentmc
Handouts: http://kentmcintosh.wordpress.com
Cannon Beach, Oregon © GoPictures, 2010
Greflund, S., McIntosh, K., Mercer, S. H., & May, S. L. (2014). Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS. Canadian Journal of School Psychology, 29, 213-235.
McIntosh, K., Barnes, A., Morris, K., & Eliason, B. M. (2014). Using discipline data within SWPBIS to identify and address disproportionality: A guide for school teams. Eugene, OR: Center on Positive Behavioral Interventions and Supports. University of Oregon.
McIntosh, K., Girvan, E. J., Horner, R. H., Smolkowski, K., & Sugai, G. (2014). Recommendations for addressing discipline disproportionality in education. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports.
Vincent, C. G., Swain-Bradway, J., Tobin, T. J., & May, S. (2011). Disciplinary referrals for culturally and linguistically diverse students with and without disabilities: Patterns resulting from school-wide positive behavior support. Exceptionality, 19, 175-190.
References