cool vs creepy - ethics and data science - cooper 2feb

Cool vs. Creepy: Ethics in Data Science

Cathy Cooper

3 February 2017

© 2016 SAP SE or an SAP affiliate company. All rights reserved. 2Public

(Image of a hand with a credit card + JetBlue plane to Orlando?)

xxx

Planes, meals, theme

parks and … credit

cards?


Connection to ethics & data science

DATA SCIENCE

Data Analysis Applications

EXPERIENCE

Transparency Permission


Data, Analysis and Applications: Biased Data

Google tested machine learning analysis on their News data to try to find relationships between

concepts.

One of the examples was the following “gender + profession” comparison.

man + computer programmer vs. woman + _________

__________________ = Homemaker

Why? Because the system was trained with an inherently biased source.


Data, Analysis and Applications: Anonymized Data

How many time & location records does it

take to link cell phone usage data to an

individual?

Only four


Data, Analysis and Applications: Context

A financial institution was looking to predict customer churn.

Who did they identify as the prime target for their retention offers?

Spouses getting their finances in order before filing for divorce.

A test was conducted using just social media sentiment analysis to predict changes in US

unemployment rate.

It did predict an unemployment drop – but it was wrong. Why?

Relying only on the tool – and poor timing: Steve Jobs had just died.


Data, Analysis and Applications: Data/Predictions Change Behavior

and Vice Versa

GPS

Flu projection


Data, Analysis and Applications: Human Intervention Required

“… people have

too much trust in

data to be

intrinsically

objective, even

though it is in fact

only as good as

the human

processes that

collected it.” --

Cathy O’Neill“Humans will need to check on the

outcomes and see if the models and the

algorithms and the rules are performing

as intended, and then intervene if they

don’t.” – Tom Davenport


Transparency: Try it for yourself

University of Cambridge The Psychometrics Centre uses your Facebook profile and activity for:

► YouAreWhatYouLike describe openness to new ideas, extraversion and introversion, your warmth

or competitiveness, and other personality traits.

► Apply Magic Sauce, predicts your politics, relationship status, sexual orientation, gender, and

more.

Pennebaker Conglomerate’s

► AnalyzeWords leverages linguistics to discover the personality you (other others) portray on

Twitter. You can analyze anyone, not just yourself.

http://youarewhatyoulike.com/

http://applymagicsauce.com/

http://analyzewords.com/


Transparency: How It Can Work

Banks have to prove:

►their algorithms are not based on variables like race & gender

►their models don’t focus on patterns that disfavor specific demographic groups

and

they have to allow outside data scientists to assess their models for code or data that might have a

discriminatory effect.

Consider what it will takes to build transparency into AI system design: human access and

action plan


Permission/Fairness

Challenge and opportunity:

► Additional uses for previously

collected/aggregated data in

new types of analysis,

applications and businesses

► Coming legal and regulatory

requirements to audit and

correct algorithm-based

decision making


In closing…

“… algorithms have to be designed with fairness and legality in

mind, with standards that are understandable to everyone, from

the business leader to the people being scored.” – Cathy O’Neill,

“Unmasking Unconscious Bias in Algorithms”, The Digitalist,

data scientist and author Weapons of Math Destruction

“…a fundamental rethinking and a careful approach to software creation itself is needed.

Extra care has to be put into training the people who create the systems, and into

incorporating research into the machine learning algorithms so you don’t accidentally create

more bias. The system has to be re-trained to think differently.” – Yvonne Baur, “Is Machine

Learning Sexist?”, Techcrunch, and Head of Predictive Analytics/Machine Learning, SAP

SuccessFactors,


Additional Readings and Resources on Ethics & Data Science

http://www.digitalistmag.com/digital-supply-

networks/2017/01/16/cathy-oneil-

unmasking-unconscious-bias-in-algorithms-

04839140

http://www.digitalistmag.com/executive-

research/how-ai-can-end-bias


research/an-ai-shares-my-office

http://www.digitalistmag.com/digital-

economy/digital-

futures/2017/01/23/machine-learning-real-

business-intelligence-04830225


research/empathy-the-killer-app-for-

artificial-intelligence

http://www.digitaltrends.com/cool-

tech/women-in-artificial-intelligence/

https://techcrunch.com/2016/10/11/is-

machine-learning-sexist/

https://open.sap.com/courses/ds1

http://www.slate.com/articles/technology/fut

ure_tense/2014/10/youarewhatyoulike_find_

out_what_algorithms_can_tell_about_you_b

ased_on_your.html

http://searchcio.techtarget.com/opinion/Dat

a-products-introduce-ethical-dilemmas-for-

data-scientists

http://searchcio.techtarget.com/opinion/Big-

data-ethics-Why-the-CIO-needs-to-get-

involved

http://searchcloudapplications.techtarget.co

m/feature/Big-data-collection-efforts-spark-

an-information-ethics-debate

https://arstechnica.com/science/2015/11/ne

w-flu-tracker-uses-google-search-data-

better-than-google/

https://gigaom.com/2013/10/09/the-upside-

of-prism-at-least-were-talking-about-data-

privacy-or-lack-the-thereof/

http://nymag.com/thecut/2016/09/cathy-

oneils-weapons-of-math-destruction-math-

is-biased.html?mid=twitter-share-thecut

https://www.bloomberg.com/news/articles/2

016-06-23/artificial-intelligence-has-a-sea-of-

dudes-problem

http://boingboing.net/2016/01/06/weapons-

of-math-destruction-h.html

http://fortune.com/2015/12/04/ethics-

training-for-data-scientists/

https://gigaom.com/2013/03/25/why-the-

collision-of-big-data-and-privacy-will-

require-a-new-realpolitik/

http://www.nature.com/articles/srep01376


ure_tense/2016/02/how_to_bring_better_ethi

cs_to_data_science.htmlhttp://searchcio.tec

htarget.com/opinion/Big-data-bad-analytics


ure_tense/2016/02/how_to_bring_better_ethi

cs_to_data_science.html

https://open.sap.com/courses/ml1

https://motherboard.vice.com/en_us/article/

big-data-cambridge-analytica-brexit-trump

http://www.digitalistmag.com/digital-supply-networks/2017/01/16/cathy-oneil-unmasking-unconscious-bias-in-algorithms-04839140

http://www.digitalistmag.com/executive-research/how-ai-can-end-bias

http://www.digitalistmag.com/executive-research/an-ai-shares-my-office

http://www.digitalistmag.com/digital-economy/digital-futures/2017/01/23/machine-learning-real-business-intelligence-04830225

http://www.digitalistmag.com/executive-research/empathy-the-killer-app-for-artificial-intelligence

http://www.digitaltrends.com/cool-tech/women-in-artificial-intelligence/

https://techcrunch.com/2016/10/11/is-machine-learning-sexist/

https://open.sap.com/courses/ds1

http://www.slate.com/articles/technology/future_tense/2014/10/youarewhatyoulike_find_out_what_algorithms_can_tell_about_you_based_on_your.html

http://searchcio.techtarget.com/opinion/Data-products-introduce-ethical-dilemmas-for-data-scientists

http://searchcio.techtarget.com/opinion/Big-data-ethics-Why-the-CIO-needs-to-get-involved

http://searchcloudapplications.techtarget.com/feature/Big-data-collection-efforts-spark-an-information-ethics-debate

https://arstechnica.com/science/2015/11/new-flu-tracker-uses-google-search-data-better-than-google/

https://gigaom.com/2013/10/09/the-upside-of-prism-at-least-were-talking-about-data-privacy-or-lack-the-thereof/

http://nymag.com/thecut/2016/09/cathy-oneils-weapons-of-math-destruction-math-is-biased.html?mid=twitter-share-thecut

https://www.bloomberg.com/news/articles/2016-06-23/artificial-intelligence-has-a-sea-of-dudes-problem

http://boingboing.net/2016/01/06/weapons-of-math-destruction-h.html

http://fortune.com/2015/12/04/ethics-training-for-data-scientists/

https://gigaom.com/2013/03/25/why-the-collision-of-big-data-and-privacy-will-require-a-new-realpolitik/

http://www.nature.com/articles/srep01376

http://www.slate.com/articles/technology/future_tense/2016/02/how_to_bring_better_ethics_to_data_science.html

http://searchcio.techtarget.com/opinion/Big-data-bad-analytics

http://www.slate.com/articles/technology/future_tense/2016/02/how_to_bring_better_ethics_to_data_science.html

https://open.sap.com/courses/ml1

https://motherboard.vice.com/en_us/article/big-data-cambridge-analytica-brexit-trump

Thank youContact information:

Cathy Cooper

Sr. Director, Big Data Analytics

[email protected]

212 653 9555

@catcooper1

cool vs creepy - ethics and data science - cooper 2feb

Documents