flavio figueiredo justiça na aprendizagem de...
TRANSCRIPT
![Page 1: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/1.jpg)
Justiça naAprendizagem de MáquinaEscola de Verão 2020Flavio Figueiredo
1
![Page 2: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/2.jpg)
About Me
Flavio Figueiredo
Whenever someone asks me: What do you research?
![Page 3: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/3.jpg)
About Me
Flavio Figueiredo
Whenever someone asks me: What do you research?
● Distributed Systems○ Dependability, File Sharing, Social Exchanges
● Information Retrieval○ Folksonomies
● Social Networks○ Popularity and evolution
● Human Computer Interaction○ User studies
● Machine Learning○ Learning models from social data
![Page 4: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/4.jpg)
Human Factors in Computer Science
Distributed SystemsWhy and how do people share information?
Information RetrievalHow does unstructured human knowledge grow?
Social DynamicsHow does user actions impact popularity? How do users perceive popularity?
Machine LearningHow to capture complex human behavior?
![Page 5: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/5.jpg)
Interest on Human-Centric Machine Learning
5
![Page 6: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/6.jpg)
Fairness is one of the main issues!
6
![Page 7: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/7.jpg)
Simplified Background
● Mathematically speaking, what is the goal of a supervised learning system?
● The goal is to learn some parameters
● Where these parameters maximize some prediction function across y
This is just one view. Optimization v. Bayesian and other topics out of the scope. 7
![Page 8: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/8.jpg)
Simplified Background
● The goal of a supervised learning algorithm is to discriminate
8
![Page 9: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/9.jpg)
Notation
We observe a dataset sampled from some joint distribution
Our ML model creates a hypothesis focused on good predictions
9
![Page 10: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/10.jpg)
Pip
elin
e
10
Data
Training Val (Dev) Test
Time
Test
Test
Test
Test
Development Production
![Page 11: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/11.jpg)
Ho
pe
full
y. L
et’
s as
sum
e s
o.
11
Data
Training Val (Dev) Test
Tempo
Test
Test
Test
Test
Development Production
Same Distribution
![Page 12: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/12.jpg)
Simplified Background
● The goal of a supervised learning algorithm is to discriminate ● Why are we now so worried that it does? It seems we can trust them.
12
![Page 13: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/13.jpg)
Machine Bias
● Pro Publica analysis of COMPAS (which stands for Correctional Offender Management Profiling for Alternative Sanctions)https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
13
![Page 14: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/14.jpg)
Criminal Prediction
https://callingbullshit.org/case_studies/case_study_criminal_machine_learning.html
14
![Page 15: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/15.jpg)
Discrimination
● The overall goal of a ML model is on discrimination● The best hypothesis is a good discriminator
○ Both in training time and in testing time
● In society, discrimination has a different connotation
15
![Page 16: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/16.jpg)
Regulation
16
![Page 19: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/19.jpg)
Two Guiding Principles
Disparate Treatment: Individuals from a sensitive group must be treated equally
19
![Page 20: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/20.jpg)
Two Guiding Principles
Disparate Treatment: Individuals from a sensitive group must be treated equallyDisparate Impact: The impact of decisions must affect groups equally
Extreme case: A credit scoring system that denies all loans?
20
![Page 21: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/21.jpg)
Two Guiding Principles
Disparate Treatment: Individuals from a sensitive group must be treated equallyDisparate Impact: The impact of decisions must affect groups equally
Extreme case: A credit scoring system that denies all loans?Real Case: https://en.wikipedia.org/wiki/Ricci_v._DeStefano
21
![Page 22: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/22.jpg)
COMPAS
22
![Page 23: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/23.jpg)
23
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Machine Bias
There’s software used across the country to predict future criminals. And it’s biased against blacks.by Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, ProPublica May 23, 2016
![Page 24: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/24.jpg)
COMPAS
Correctional Offender Management Profiling for Alternative Sanction
● First developed in 1998● Still used today● Predicts chance of recidivism● Developed by Northpointe (now Equivant)● Most information comes from a manual
"A Practitioner's Guide to COMPAS Core"
24
![Page 25: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/25.jpg)
COMPAS
Used in (or was used in, may be outdated)
● Florida● New York● Wisconsin● California● and others...
25
![Page 26: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/26.jpg)
Overall
It’s not the algorithm, it’s the data.
https://cacm.acm.org/magazines/2017/2/212422-its-not-the-algorithm-its-the-data/fulltext
“There is no debate that both of these types of technologies are being used on a fairly widespread basis in the U.S. According to a 2013 article published by Sonja B. Starr, a professor of law at the University of Michigan Law School, nearly every state has adopted some type of risk-based assessment tools to aid in sentencing.”
26
![Page 27: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/27.jpg)
How is the model trained?
Who knows?!
27
![Page 28: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/28.jpg)
How is the model trained?
We know the input
28
![Page 29: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/29.jpg)
How is the model trained?
Compas Questionnairehttps://www.documentcloud.org/documents/2702103-Sample-Risk-Assessment-COMPAS-CORE.html
29
![Page 30: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/30.jpg)
Is COMPAS Using Well Known ML?
● Linear Regression● Logistic Regression● Support Vector Machines● Random Forests● etc● etc
Who knows? But they cite such methods.
30
![Page 31: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/31.jpg)
Several Citations to ML Work
Note that it is a regression model. It seems that they regress to their own risk score.
31
![Page 32: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/32.jpg)
Decile Scores
My interpretation:
● It seems that COMPAS scalesinputs and outputs in deciles
● Inputs come from the questionnaireand other variables
● Outputs is usually some kind of risk● I would guess that this is fed to some
blackbox ML system
32
![Page 33: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/33.jpg)
Decile Scores
33
![Page 34: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/34.jpg)
Single Equation
Only one equation is present in the manual:
Violent Recidivism Risk Score = (age∗−w) + (age-at-first-arrest∗−w) + (history of violence∗w) + (vocation education ∗ w) + (history of noncompliance ∗ w),
where w is weight, the size of which is "determined by the strength of the item’s relationship to person offense recidivism that we observed in our study data."
34
![Page 35: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/35.jpg)
ProPublica’s Study
● Most analysis of such software is done by the developers● ProPublica decided to do it’s own analysis● COMPAS was chosen due to popularity
35
![Page 36: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/36.jpg)
Anecdotal Evidence
Several examples in the article
36
![Page 37: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/37.jpg)
Anecdotal Evidence
● Great for news articles● But the problem needs to be better understood● Use machine learning to understand machine learning
37
![Page 38: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/38.jpg)
Data
38
![Page 39: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/39.jpg)
Response
39
![Page 40: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/40.jpg)
Some Initial Data Science
● Understanding Decile Scores (output of the model)
40
![Page 41: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/41.jpg)
Some Initial Data Science
● Risk of Recidivism Score
41
![Page 42: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/42.jpg)
Some Initial Data Science
● Violent Risk Score
42
![Page 43: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/43.jpg)
Logistic Regression
● The model here is trying to predict the output of Compas● Works somewhat like a reverse engineer● We are not predicting real recidivism!
43
![Page 44: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/44.jpg)
Results of the Model
● Significant values indicatethat the features may be ableto explain the COMPAS score
● Positive values point towardsa recidivism indication by COMPAS
● Negative values are the opposite● Higher values, more impact
44
![Page 45: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/45.jpg)
Violent Recidivism
● Significant values indicatethat the features may be ableto explain the COMPAS score
● Positive values point towardsa recidivism indication by COMPAS
● Negative values are the opposite● Higher values, more impact
45
![Page 46: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/46.jpg)
Fairness
46
![Page 47: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/47.jpg)
21 fairness definitions and their politics
Arvind Narayanan - FAT Conference 2018 Tutorial
● Computer Scientist on a wild goose chase for a single definition● There is value to various definitions● Each can lead to trustworthiness
47
![Page 48: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/48.jpg)
What is Fairness?Sahil Verma and Julia Rubin (2018) -- Fairness Definitions Explained
● A lot of these metrics worry about someform of equality
● Let S be some subset of sensitive attributes.S = { col(j, X) | column j is sensitive }N = { col(i, X) | column i is not-sensitive }
48
![Page 49: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/49.jpg)
Balanced Representation
1 0 1 1
0 0 1 0
1 0 1 1
1 1 0 1
0
1
0
0
SN
1 1 1 0 1
X =
Is this fairness?
49
![Page 50: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/50.jpg)
Classifier Evaluation
If we assume that the future has the same distribution as the past.
● We can measure the error rates on our development step● Indicates that our model works
Or we can...
● Actually wait for the future● Then make claims
In either case, there are multiple ways to evaluate a classifier. Compare predictions with ground-truth labels.
50
![Page 51: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/51.jpg)
Classifier Evaluation
● Different tasks will focus on different metrics● Search engines usually optimize for precision (definition soon)
○ Retrieved cases
● When predicting some rare cancer, recall is more important○ All cases
● How should we work in recidivism?!
To understand the metrics we need a confusion matrix.
51
![Page 52: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/52.jpg)
Parity in Predictions
52
![Page 53: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/53.jpg)
Classifier Evaluation (COMPAS Example)
53
High Risk Low Risk
Recidivism True PositiveTP
False NegativeFN
Stayed Clean False PositiveFP
True NegativeTN
![Page 54: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/54.jpg)
What is the impact of each kind of error?
54
High Risk Low Risk
Recidivism True PositiveTP
False NegativeFN
Stayed Clean False PositiveFP
True NegativeTN
![Page 55: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/55.jpg)
Classifier Evaluation (COMPAS Example)
55
High Risk Low Risk
Recidivism True PositiveTP
False NegativeFN
Criminal loose!
Stayed Clean False PositiveFP
Innocent arrested!
True NegativeTN
![Page 56: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/56.jpg)
COMPAS in Practice
● Each individual is assigned a score
● Scores are broken down into deciles
56
![Page 57: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/57.jpg)
COMPAS in Practice
● Let’s assume three cut-off points● Purple individuals have recidivated● Orange ones have not● Now we need to pick a cut-off point
57
D1 D2 D2 D1 D3 D3 D1
TP FN
FP TN
1 0prediction
true1
0
![Page 58: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/58.jpg)
COMPAS in Practice
● Let’s say that everyone >= D2 is at risk● What is each score?
58
D1
D2 D2
D1
D3 D3
D1
TP FN
FP TN
1 0prediction
true1
0
![Page 59: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/59.jpg)
COMPAS in Practice
● Let’s say that everyone >= D2 is at risk● What is each score?● TP = 3, FP = 1, FN = 0, TN = 3.
59
D1
D2 D2
D1
D3 D3
D1
TP FN
FP TN
1 0prediction
true1
0
![Page 60: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/60.jpg)
COMPAS in Practice
● Let’s say that everyone >= D1 is at risk● What is each score?
60
D1 D2 D2 D1 D3 D3 D1
TP FN
FP TN
1 0prediction
true1
0
![Page 61: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/61.jpg)
COMPAS in Practice
● Let’s say that everyone >= D1 is at risk● What is each score?● TP = 3, FP = 4, FN = 0, TN = 0.
61
D1 D2 D2 D1 D3 D3 D1
TP FN
FP TN
1 0prediction
true1
0
![Page 62: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/62.jpg)
COMPAS in Practice
● Let’s say that everyone >= D3 is at risk● What is each score?● TP = 1, FP = 1, FN = 2, TN = 3.
62
D1 D2 D2 D1
D3 D3
D1
TP FN
FP TN
1 0prediction
true1
0
![Page 63: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/63.jpg)
COMPAS in Practice
● We usually explore such metrics in normalized terms● TPR = True Positive Rate or Recall
○ TP / (TP + FN)○ Row normalization
● FPR○ FP / (FP + TN)○ Row normalization
● How do we maximize each?
63
TP FN
FP TN
prediction
true
![Page 64: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/64.jpg)
COMPAS in Practice
● We usually explore such metrics in normalized terms● TPR = True Positive Rate or Recall
○ TP / (TP + FN)○ Row normalization
● FPR○ FP / (FP + TN)○ Row normalization
● How do we maximize each?○ Recall is maximized when we say everyone is going to recidivate○ FPR is maximized if say no-one is going to
64
TP FN
FP TN
prediction
true
![Page 65: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/65.jpg)
COMPAS Evaluation(From developers)
65
TP FN
FP TN
prediction
true
Predictive Validity of the COMPAS Reentry Risk ScalesAn Outcomes Study Conducted for the Michigan Department of Corrections: Updated Results on an Expanded Release Sample
https://epic.org/algorithmic-transparency/crim-justice/EPIC-16-06-23-WI-FOIA-201600805-MDOC_ReentryStudy082213.pdf
![Page 66: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/66.jpg)
COMPAS in Practice
● TPR or Recall of around 80%● 80% of every recidivism case is accurate
● FPR or around 43%● 43% of innocent individuals receive a
high score● 57% of non-recidivism cases are accurate
66
TP FN
FP TN
prediction
true
![Page 67: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/67.jpg)
COMPAS in Practice
● Suggestion of cut-off point at 4● Anybody above that is predicted as
a recidivism risk● Below are low risk● AUC of 0.72
○ Good score
67
TP FN
FP TN
prediction
true
![Page 68: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/68.jpg)
COMPAS in Practice
● In all fairness, we are only showing one result● The study considers various aspects of COMPAS● If you wish, you can present many of the studies from COMPAS
68
![Page 69: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/69.jpg)
Two views of the confusion matrix
69
![Page 70: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/70.jpg)
Other View of the Confusion Matrix
70
High Risk Low Risk
Recidivism True PositiveTP
False NegativeFN
Stayed Clean False PositiveFP
True NegativeTN
Data focused (recall)
Prediction Focused (Prec.)
![Page 71: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/71.jpg)
Column Normalization
● Positive Predictive Value (PPV) or Precision○ TP / (TP + FP)
● False Discovery Rate○ FP / (TP + FP)○ 1-PPV○ 1-Precision
● Negative Predictive Value (NPV)○ TN / (TN + FN)
● False Omission Rate○ 1 - NPV
Word of advice: Some books transpose the matrix. No decoreba!71
TP FN
FP TN
prediction
true
![Page 72: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/72.jpg)
Propublica Dataset
Now let’s look into the Propublica study.Different dataset (Michigan vs Broward County)
72
![Page 73: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/73.jpg)
Propublica Dataset
Let’s look at results comparing races
Table from Krishna Gummadi
73
TP FN
FP TN
1 0prediction
true1
0
![Page 74: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/74.jpg)
Propublica Dataset
74
TP FN
FP TN
1 0prediction
true1
0
![Page 75: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/75.jpg)
Propublica Dataset
75
TP FN
FP TN
1 0prediction
true1
0
![Page 76: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/76.jpg)
Propublica Dataset
76
TP FN
FP TN
1 0prediction
true1
0Northpointe: FDR rates are comparable!COMPAS is Fair!
![Page 77: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/77.jpg)
Propublica Dataset
● Now let’s look at the rows.● Recall that this focuses on the data!● Columns focus on predictions. Previous results, predictions balanced!
77
TP FN
FP TN
1 0prediction
true1
0
![Page 78: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/78.jpg)
Propublica Dataset
78
TP FN
FP TN
1 0prediction
true1
0
![Page 79: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/79.jpg)
Propublica Dataset
79
TP FN
FP TN
1 0prediction
true1
0
![Page 80: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/80.jpg)
Propublica Dataset
80
TP Loose
Arrest TN
1 0prediction
true1
0● Error rates are not comparable● What are the consequences?
![Page 81: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/81.jpg)
Rates on Imbalanced Datasets
● Both datasets are imbalanced to begin with● There are arguments from both sides● Presentation and Project ideas
○ Re-evaluate COMPAS■ Other classifiers■ Other metrics
○ Present counter-argument papers
81
![Page 82: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/82.jpg)
Impossibility of Fairness
82
![Page 83: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/83.jpg)
Fairness is Impossible in Practice
● Two proofs, one from:Alexandra Chouldechova
and one from
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghava
● References at the end of the slide
83
![Page 84: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/84.jpg)
In Practice
● Classifiers output some probability score● This probability is used as a threshold● This is exactly what we have done with the decile scores
84
![Page 85: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/85.jpg)
Formalizing
● The classifier now outputs a score
● From which we can create classes
● Essentially what COMPAS does85
![Page 86: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/86.jpg)
Formalizing
● Now, let each value be associated with a random variableR is for raceY is for true values (recidivism or not). Not predictions.S is for score
(note to self: exemplify)
86
![Page 87: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/87.jpg)
Formalizing
● Now, let each value be associated with a random variableR is for raceY is for true values (recidivism or not). Not predictions.S is for score
● Each has a probability distribution P(R = 1) and P(R = 0)● Also, let 𝞼 be the true fraction of individuals that have recidivated. This is called our
base rate (or prevalence): 𝞼 = P(Y = 1)
87
![Page 88: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/88.jpg)
3 Definitions of Fairness
● When y = 1. Recidivism
● When y = 0. No recidivism
88
![Page 89: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/89.jpg)
Calibration
89
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 90: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/90.jpg)
Calibrated for all Scores (Deciles)
90
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 91: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/91.jpg)
Not Calibrated for Score 2
91
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 92: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/92.jpg)
Not Calibrated for all Scores
92
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 93: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/93.jpg)
Intuition
Calibration: Scores have the same meaning.
93
![Page 94: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/94.jpg)
Predictive Parity
94
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 95: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/95.jpg)
Predictive Parity OK for sHR = 2
95
s = 1
R = white
s = 2
R = white
s = 3
R = white
s = 1
R = black
s = 2
R = black
s = 3
R = black
![Page 96: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/96.jpg)
Intuition
Calibration: Scores have the same meaning.
Predictive Parity: I expect the same precision for each group. This was Northpoint’s argument (see previous slides). Also, from the previous slides we can see that PP differs from Calibration.
96
![Page 97: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/97.jpg)
Measuring Calibration and PP
97
![Page 98: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/98.jpg)
Error Rate Balance
● Focused on the rows-normalization of theconfusion matrix
● Note that it is conditioned on true value● Recall like interpretation● This is where ProPublica showed that COMPAS is unfair● Easy to see with a plot (next slide)
98
TP FN
FP TN
1 0prediction
true1
0
![Page 99: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/99.jpg)
Error Rate Balance
99
![Page 100: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/100.jpg)
Intuition
Calibration: Scores have the same meaning.
Predictive Parity: I expect the same precision for each group. This was Northpoint’s argument (see previous slides). Also, from the previous slides we can see that PP differs from Calibration.
Error Rate Balance: We cannot bias to fewer/more errors towards a certain group.
100
![Page 101: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/101.jpg)
Error Rate Balance
Previous graph
● For different cut-off points● Imbalanced error rates● [Main Result] COMPAS is never fair from this point of view. Why?
○ Actually valid for any classifier
101
![Page 102: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/102.jpg)
Proof that FPR and FNR cannot be equal
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
102
TP FN
FP TN
1 0prediction
true1
0
![Page 103: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/103.jpg)
We can write the identity below
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
103
TP FN
FP TN
1 0prediction
true1
0
FP=
TP + FN FP TP
FP + TN FP + TN TP TP + FN
![Page 104: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/104.jpg)
We can write the identity below
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
104
TP FN
FP TN
1 0prediction
true1
0
FP=
TP + FN FP TP
FP + TN FP + TN TP TP + FN
![Page 105: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/105.jpg)
We can write the identity below
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
105
TP FN
FP TN
1 0prediction
true1
0
FP=
TP + FN FP TP
FP + TN FP + TN TP TP + FN
![Page 106: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/106.jpg)
We can write the identity below
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
Rewrite:
106
TP FN
FP TN
1 0prediction
true1
0
FP=
TP + FN FP TP
FP + TN FP + TN TP TP + FN
![Page 107: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/107.jpg)
We can write the identity below
Knowing that:
● 𝞼 = P(Y = 1) = (TP + FN) / N● 1-𝞼 = P(Y = 0) = (FP + FN) / N● FPR = FP / (FP + TN)● PPV = TP / (TP + FN)● 1 - FNR = Recall = TP / (TP + FN)
107
TP FN
FP TN
1 0prediction
true1
0
FPR =𝞼 1 - PPV
1-FNR1-𝞼 PPV
![Page 108: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/108.jpg)
Two Groups
108
FPR(w) =𝞼(w) 1 - PPV(w)
1-FNR(w)1-𝞼(w) PPV(w)
FPR(b) =𝞼(b) 1 - PPV(b)
1-FNR(b)1-𝞼(b) PPV(b)
Each with a confusion matrix, then for group w:
And for group b:
![Page 109: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/109.jpg)
Calibration and PP are equal for groups
109
![Page 110: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/110.jpg)
If PP and Calibration are Equal
110
FPR(w) =𝞼(w) 1 - PPV(w)
1-FNR(w)1-𝞼(w) PPV(w)
FPR(b) =𝞼(b) 1 - PPV(b)
1-FNR(b)1-𝞼(b) PPV(b)
PPV(w) = PPV(b)
![Page 111: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/111.jpg)
If PP and Calibration are Equal
111
FPR(w) =𝞼(w)
1-FNR(w)1-𝞼(w)
FPR(b) =𝞼(b)
1-FNR(b)1-𝞼(b)
PPV(w) = PPV(b). Divide one with the other.
The other two can only be equal when: 𝞼(w) = 𝞼(b)
This is not the case! From the data: the recidivism rate is not equal across groups.Try setting FPR(w) = FPR(b) with 𝞼(w) != 𝞼(b). Impossible to reach equal FNR.
![Page 112: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/112.jpg)
If PP and Calibration are Equal
112
FPR(w) =𝞼(w)
1-FNR(w)1-𝞼(w)
FPR(b) =𝞼(b)
1-FNR(b)1-𝞼(b)
PPV(w) = PPV(b). Divide one with the other.
The other two can only be equal when: 𝞼(w) = 𝞼(b)
This is not the case! From the data: the recidivism rate is not equal across groups.Try setting FNR(w) = FNR(b) with 𝞼(w) != 𝞼(b). Impossible to reach equal FPR.
![Page 113: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/113.jpg)
If PP and Calibration are Equal
113
FPR(w) =𝞼(w)
1-FNR(w)1-𝞼(w)
FPR(b) =𝞼(b)
1-FNR(b)1-𝞼(b)
PPV(w) = PPV(b). Divide one with the other.
Proof Sketch: 𝞼/(1-𝞼) is bijective.FPR(w) = a * bFPR(b) = c * d. When we set both FPRs to equal, b = d only when a = c.
![Page 114: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/114.jpg)
Results
● Impossibility of achieving predictive parity and equal errors for both error rates● Our classifiers will always favor one group● Where do we go from here:
○ We can tolerate some error threshold○ Decide which group should be biased○ More definitions of fairness [next class]
114
![Page 115: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/115.jpg)
What can we do?
115
![Page 116: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/116.jpg)
Hard Problem
Society (as a consequence datasets) is unfairAccountability is difficult (who do we blame?)Datasets and models are hard to understand
116
![Page 118: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/118.jpg)
Thank You!
118
![Page 119: Flavio Figueiredo Justiça na Aprendizagem de Máquinaevcomp.dcc.ufmg.br/wp-content/uploads/Evcomp-2020-Flavio.pdf · Aprendizagem de Máquina Escola de Verão 2020 Flavio Figueiredo](https://reader033.vdocuments.us/reader033/viewer/2022042209/5eacc9a03e7365579f20e9d8/html5/thumbnails/119.jpg)
References
● Predictive Validity of the COMPAS Reentry Risk Scaleshttps://epic.org/algorithmic-transparency/crim-justice/EPIC-16-06-23-WI-FOIA-201600805-MDOC_ReentryStudy082213.pdf
● Equality in Opportunity in Machine Learninghttps://arxiv.org/pdf/1609.05807.pdf
● Fair prediction with disparate impact:A study of bias in recidivism prediction instrumentshttps://www.andrew.cmu.edu/user/achoulde/files/disparate_impact.pdf
● Inherent Trade-offs in Algorithmic Fairnesshttps://www.youtube.com/watch?v=p5yY2MyTJXA
119