fewer hippos more lab coats compressed - r . …...fewer hippos more lab coats highest paid...

Catherine Leigh

Fewer HiPPOs

More Lab Coats

Highest paid person’s opinion Data driven decisions

Change is a constant

Change is a constant

2 months ago

Today

Some Examples

BlockbusterReal world examples

Ford EdselReal world examples

Ford EdselReal world examples

OJ SimpsonReal world examples

Probabilities can changeReal world examples

OJ SimpsonReal world examples

Medicine

http://www.alzheimersresearchuk.org/

Real world examples

Where do you sit?

Bad Good

Misinterpretation

Getting it right

Ignoring data

No data

How to Get it Right

Change is a constant

Change is a constant

Hypothesis testing is a tool for managing uncertainty

Change is a constant

Hypothesis generalisation:

H0: nothing is different(the null hypothesis)

H1: something is different(the alternative hypothesis)

Change is constant

Hypothesis example with coins:

H0: this coin is unbiased (the null hypothesis)

H1: this coin is biased(the alternative hypothesis)

Change is constant

Hypothesis example with coins:Change is constant

If H0 was true, what is the chance of seeing a result this extreme?

Hypothesis example with coins:Change is constant

> pbinom(9, size=20, prob=0.5) * 2[1] 0.8238029

Hypothesis example with coins:Change is constant

> pbinom(9, size=20, prob=0.5) * 2[1] 0.8238029

82%

Hypothesis example with an AB test:

H0: a new logo will not affect bounce rate*(the null hypothesis)

H1: a new logo will affect bounce rate(the alternative hypothesis)

Change is constant

Hypothesis example with an AB test:Change is constant

Hypothesis example with an AB test:Change is constant

Bounces

2+ Page Views

Total

Total

1000

4000

5000

1152

3853

5005

2152

7853

10005

Hypothesis example with an AB test:Change is constant

> library(stats)> amazingLogoTest <-+ matrix(c(1000, 4000, 1152, 3853),+ nrow = 2,+ dimnames = list(visitorAction = c("Bounce", "Stay"),+ logo = c("Boring", "Best Logo Ever")))> fisher.test(amazingLogoTest, alternative = "less")Fisher's Exact Test for CountDatadata: amazingLogoTestp-value = 0.0001316alternative hypothesis: true odds ratio is less than 195 percent confidence interval:0.0000000 0.9070399sample estimates:odds ratio0.8361718

Hypothesis example with an AB test:Change is constant

> library(stats)> amazingLogoTest <-+ matrix(c(1000, 4000, 1152, 3853),+ nrow = 2,+ dimnames = list(visitorAction = c("Bounce", "Stay"),+ logo = c("Boring", "Best Logo Ever")))> fisher.test(amazingLogoTest, alternative = "less")Fisher's Exact Test for CountDatadata: amazingLogoTestp-value = 0.0001316alternative hypothesis: true odds ratio is less than 195 percent confidence interval:0.0000000 0.9070399sample estimates:odds ratio0.8361718

0.01%

Change is a constant

Hypothesis testing is a tool for managing uncertainty

You will need...

34

You will need...

35

> install.packages(“stats”)> library(stats)