cognitive biases in data science

www.flydata.com

Cognitive Biases in

Data Science

Copyright © 2014 FlyData Inc. All rights reserved. www.flydata.com

http://www.flydata.com

Introduction

Copyright © 2014 FlyData Inc. All rights reserved.

● We often think of “data” as objective information

● In reality, data can be just as subjective as the

people who record it!

● In scientific fields especially…

○ empirical methods are used to observe nature

○ data should always be collected and

interpreted impartially

www.flydata.com


Introduction

Copyright © 2014 FlyData Inc. All rights reserved.

● Cognitive biases are an obstacle when trying to

interpret information

○ Can easily skew results

○ They are innate tendencies

● Here are 4 major biases that are known to have

considerable effects on research and science:

www.flydata.com


#1 Confirmation

Bias



Confirmation Bias


● Confirmation bias is the tendency to process

information in a way that confirms one’s

preconceptions or hypotheses.

○ Actively seek out and assign more value to

data that confirms our own hypotheses...

○ And ignore/understate evidence that could

mean otherwise!


Confirmation Bias


● You may have “good” preconceptions from an

educated intuition or previous experiences…

● But it’s not like that in many cases!

○ Can directly affect the results of a study

or analysis!


#2 Observation

Bias



Observation Bias


● The tendency to look in places where it is

expected to produce good results, or where it is

very convenient to observe

○ Easy accessibility/availability doesn’t mean

it’s the most important!

● The most available and known data source

may often be a good one…

○ But no data analysis is complete without a

complete picture of your data.

● Data science is about producing actionable

insights

○ If only the wrong things are being observed

and measured, you produce false insights!


Observation Bias


● To be an efficient researcher, perhaps it’s

best to frequently ask yourself these

questions:

○ “Am I measuring the right things?”

○ “Are there better sources from which to

get data from?”


#3 Funding Bias



Funding Bias


● Unconscious tendency to skew models, data,

or interpretations of data in a way that favors

the objectives of a financial sponsor or

employer.

○ Sometimes called sponsorship bias

● Any scientist/researcher should keep this in

mind

○ Unknowingly making a business decision

with flawed data will ultimately damage

sponsor!

○ Will damage your career

○ ..and it’s just bad science!


Example


● In the 1990’s, the tobacco industry funded a

number of research studies on the effects of

tobacco and smoking cigarettes

● After investigation, industry sponsors and

research centers were found to

○ Present findings in a misleading way

○ Withhold certain findings about the

relationships between smoking and

cancer

● This is a prime example of a funding bias.


#4 Sampling Bias


www.flydata.com

Sampling Bias


● In experimentation, we take a sample, which

should be representative of a whole population

○ Achieved by statistical techniques and well-

designed randomization

○ What happens if proper randomization isn’t

achieved?

● It’s not uncommon for researchers to have a

sampling bias

○ Selection of groups or data for

experimentation is unintentionally not

representative of the population


Sampling Bias


● No matter how big/diverse the sample is..

○ Always a possibility of inconsistency in

data/sample collection

● This bias also ties in with the other 3 biases!

○ If any of those biases affects the way in

which you collect samples, then you’re

also experiencing a sampling bias!


www.flydata.com www.flydata.com

Check us out!

-> http://flydata.com

[email protected]

Toll Free: 1-855-427-9787

http://flydata.com

We are an official data integration

partner of Amazon Redshift

http://flydata.com?ref=slideshare

mailto:[email protected]

http://flydata.com?ref=slideshare

cognitive biases in data science

Internet