Fairness and transparency in machine learning

Tools and techniques

PyData Berlin – 2017Andreas Dewes (@japh44)

Introduction: Why think about this?

Fairness in Machine Learning

• Fairness is not a technological problem, but unfair behaviorcan be replicated / automated using technology.

• Machine learning systems are not per se fair or unfair, but have the potential to be either depending on how we use them.

• We have a chance to eliminate unfairness by using machinelearning and data analysis to make personal biases explicit and design systems that eliminate it!

Discrimination

Discrimination is treatment or consideration of, or making a

distinction in favor of or against, a person or thing based on the group, class, or category to which that person or thing is perceived to belong to rather than on individual merit.

Protected attributes (examples):

Ethnicity, Gender, Sexual Orientation, ...

When is a process discriminating?Disparate Impact: Adverse impact of a process C on a givengroup X

𝑃 𝐶 = 𝑌𝐸𝑆 𝑋 = 0

𝑃 𝐶 = 𝑌𝐸𝑆 𝑋 = 1< τ

see e.g. "Certifying and Removing Disparate Impact" M. Feldman et. al. (arxiv.org)

Estimating with real-world data

τ =𝑐/ 𝑎 + 𝑐

𝑑/ 𝑏 + 𝑑

c

a

d

b

Alternative Approaches: Individual-BasedFairness

𝑓 𝑥1 − 𝑓 𝑥2 ≤ 𝐿 𝑥1 − 𝑥2

similar individual => similar treatment!

Let‘s try todesigna fair & transparentalgorithm

NYC Stop & Frisk Dataset

https://gist.github.com/dannguyen/67ece10c6132282b1da2

• Design a stop & frisk algorithm that is as fair as possible

• Ensure it fulfills the other goals that we have for it

Input Variablesappearance-related behavior-relatedAttribute Name

Description

Age SUSPECT'S AGE

Weight SUSPECT'S WEIGHT

Ht_feet SUSPECT'S HEIGHT (FEET)

Eyecolor SUSPECT'S EYE COLOR

Haircolor SUSPECT'S HAIRCOLOR

Race SUSPECT'S RACE

Sex SUSPECT'S SEX

Build SUSPECT'S BUILD

CS_Cloth WEARING CLOTHES COMMONLY USED IN A CRIME

CS_Objcs CARRYING SUSPICIOUS OBJECT

CS_Bulge SUSPICIOUS BULGE

CS_Descr FITS A RELEVANT DESCRIPTION

RF_Attir INAPPROPRIATE ATTIRE FOR SEASON

Attribute Name

Description

ac_evasv EVASIVE RESPONSE TO QUESTIONING

ac_assoc ASSOCIATING WITH KNOWN CRIMINALS

cs_lkout SUSPECT ACTING AS A LOOKOUT

cs_drgtr ACTIONS INDICATIVE OF A DRUG TRANSACTION

cs_casng CASING A VICTIM OR LOCATION

cs_vcrim VIOLENT CRIME SUSPECTED

ac_cgdir CHANGE DIRECTION AT SIGHT OF OFFICER

cs_furtv FURTIVE MOVEMENTS

ac_stsnd SIGHTS OR SOUNDS OF CRIMINAL ACTIVITY

rf_othsw OTHER SUSPICION OF WEAPONS

rf_knowl KNOWLEDGE OF SUSPECTS PRIOR CRIMINAL BEHAVIOR

rf_vcact ACTIONS OF ENGAGING IN A VIOLENT CRIME

rf_verbl VERBAL THREATS BY SUSPECT

Attribute Name

Description

inout WAS STOP INSIDE OR OUTSIDE?

trhsloc WAS LOCATION HOUSING OR TRANSIT AUTHORITY?

timestop TIME OF STOP (HH:MM)

pct PRECINCT OF STOP (FROM 1 TO 123)

ac_proxm

PROXIMITY TO SCENE OF OFFENSE

cs_other OTHER

ac_rept REPORT BY VICTIM / WITNESS / OFFICER

ac_inves ONGOING INVESTIGATION

ac_incid AREA HAS HIGH CRIME INCIDENCE

ac_time TIME OF DAY FITS CRIME INCIDENCE

circumstance-related

?

Process Model

Possible goals:

Build a system that decideswhether to frisk someone or

Try to maximize discovery ofcriminals while not botheringlaw-abiding citizens.

Do not discriminate againstindividual groups of people.

Choosing A Loss Function

Give a reward α if ouralgorithm correctly

identifies a person to frisk

Give a penalty -1 if our algorithm wronglyidentifies a person to frisk

α (weight parameter)

„It‘s okay to frisk α +1 people to find one criminal“.

Measuring Fairness via Disparate Treatment

Building a First ModelClean input data

Select attributes

Convert to binary valuesusing „one hot“ method

Train a classifier on thetarget value

Measure the score and discrimination metrics

Load CSV data into dataframe,

discretize all attributes, clean input data.

Use a logistic regressionclassifier to predict the targetattribute.

Split the data in training/testsets using a 70/30 split.

Generate models for a range of

α values and compareperformance.

First Attempt: To Frisk Or Not To Frisk…

Input attributes was this person frisked?

How To Judge The Success Rate of The Algorithm

Our algorithm should at least be as good as a random algorithm at picking people to frisk.

It can „buy“ true positives by accepting falsepositives. The higher α is, the more profitable thistrade becomes.

Eventually we will havefrisked all people, which isa solution to the problem(but not a good one…)

Example: Predicting Only With Noise (NoInformation)

We give no usefulinformation to thealgorithm at all.

It will therefore pick the action (frisk / not frisk) that will globallymaximize the score when chosen for all people.

Predicting „frisk“ with all available inputattributes

Now we give it all the input attributesthat we got.

It will make a prediction that ismuch better thanrandomly choosinga person to frisk.

What does it mean for individual groups?

There is strong mistreatment ofindividual groups.

The algorithmlearned to be just asbiased as thetraining data.

Where does the bias come from?

Let‘s see!

Predict „black“ from availableattributes.

The algorithm can easilydifferentiate between„white“ and „black“

Eliminating The Bias: Different Approaches

Remove /Modify

Data Points

…

1

2

3

Constrain the

Algorithm

forbiddensolutions

allowedsolutions

…or change the target attribute!

…

1

2

3

Remove Attributes

Trying Different Attribute Sets To Predict„Black“only behavior-basedattributes

only circumstance-basedattributes

Almost nopredictionpossible! Prediction still

possible(probably due to„pct“ attribute)

Let‘s Try It Reducing the Features: Use OnlyBehavior

previous modelwith all features

Disparate Treatment Is Reduced (But So IsUsefulness)

previous modelwith all features

Let's Try Using A Different Target Attribute

Input attributeswas this person arrested /

summoned?

Training with A Different Target: Arrests + Summons(only using circumstance-based attributes)

There should be

less bias in thearrests, as it isharder (but still possible) to arrestsomeone who isinnocent.

As Expected, Bias Is Reduced

No „preferential“ treatment evident forwhite people in the data(on the contrary)

Much better!

Better (But Still Imperfect) Treatment By The Algorithm

Take-Aways

• Most training data that we use contains biases• Some of these biases are implicit and not easy to recognize (if

we don‘t look)

• To protect people from discrimination, we need to record and analyze their sensitive data (in a secure way)

• Machine learning and data analysis can uncover hidden biasesin processes (if we're transparent about the methods)

• Algorithmic systems can improve the fairness of manualprocesses by ensuring no biases are present

Outlook: What Future ML Systems CouldLook Like

ML Algorithm

Non-protectedInput Data

SanitizedInput Data

ProtectedInput Data Explainer

explanations

results

Auditor

fairness metrics

Thanks!

Slides, code, literature and data will be made available here:

https://github.com/adewes/fatml-pydata

Contact me: ich@andreas-dewes.de (@japh44)

Image Credits:https://gist.github.com/dannguyen/67ece10c6132282b1da2

https://commons.wikimedia.org/wiki/File:Deadpool_and_Predator_at_Big_Apple_Con_2009.jpg