Big Data… Big Problems?

Big Data… Big Problems?

Big Data… Big Problems?

The Lessons of 2012

1. Average the Polls

2. Count to 270

3. Account for Margin of

Error

The 538 Method (Simplified)

The Lessons of 2012

The Lessons of 2012

The Lessons of 2012

The Lessons of 2012

“nate silver”

“joe biden”

The Lessons of 2012

“justin bieber”

Problem #1: Big Data… Big Bias?

Problem #1: Big Data… Big Bias?

Problem #1: Big Data… Big Bias?

The Signal-to-Noise Ratio

𝟓𝒙 𝟒𝟐

=𝟏𝟎

The Signal-to-Noise Ratio

𝟏𝟎𝒙𝟗𝟐

=𝟒𝟓

The Signal-to-Noise Ratio

𝟏𝟒𝟖 ,𝟎𝟎𝟎𝒙𝟏𝟒𝟕 ,𝟗𝟗𝟗𝟐

=𝟏𝟎 ,𝟗𝟓𝟏 ,𝟗𝟐𝟔 ,𝟎𝟎𝟎

Problem #2: Desperately Seeking Signal

http://imgs.xkcd.com/comics/sports.png

Problem #2: Desperately Seeking Signal

Problem #2: Desperately Seeking Signal

The Limits of Artificial “Intelligence”

a b c d e f g h

8

7

6

5

4

3

2

1

The Limits of Artificial “Intelligence”

+1 +1 +1 +1 +1 +1 +3 +3 +3 +5 +9 = 29

+1 +1 +1 +1 +1 +3 +3 +5 +5 +9 = 30

Kasparov

Deep Blue

The Limits of Artificial “Intelligence”

a b c d e f g h

8

7

6

5

4

3

2

1

The Limits of Artificial “Intelligence”

a b c d e f g h

8

7

6

5

4

3

2

1

The Limits of Artificial “Intelligence”

a b c d e f g h

8

7

6

5

4

3

2

1

Problem #3: Feature or Bug?

Problem #3: Feature or Bug?

Problem #3: Feature or Bug?

Suggestions

Suggestions

1. Think Probabilistically

2. Know Where You’re Coming From

3. Try, and Err

Suggestion #1: Think Probabilistically

Levee: 51’Flood Prediction: 49’

Suggestion #1: Think Probabilistically

Levee: 51’Flood Prediction: 49’

Margin of Error: ±9’

Suggestion #1: Think Probabilistically

Suggestion #2: Know Where You’re Coming From

Suggestion #3: Try, and Err

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Effort

Accuracy

Suggestion #3: Try, and Err

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Suggestion #3: Try, and Err

Effort

Accuracy

Suggestion #3: Try, and Err

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Suggestion #3: Try, and Err

Effort

Accuracy

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Suggestion #3: Try, and Err

Effort

Accuracy

Suggestion #3: Try, and Err

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Effort

Accuracy

Water level

Suggestion #3: Try, and Err

0% 20% 40% 60% 80% 100%0%

20%

40%

60%

80%

100%

Effort

Accuracy

Competitive Advantage

Suggestions

1. Think Probabilistically

2. Know Where You’re Coming From

3. Try, and Err

Suggestions

(Know Your Limitations)

2. Know Where You’re Coming From

3. Try, and Err

Suggestions

(Know Your Limitations)

(Consider Your Assumptions)

3. Try, and Err

Suggestions

(Know Your Limitations)

(Consider Your Assumptions)

(Refine Your Process)

Suggestions

“In Science, We Seek to Balance Creativity and Skepticism”

-- Michael Babyak, PhD, Department of Psychiatry and Behavioral Science, Duke University MedicalCenter