why big data will survive the hype - and change the way we work
DESCRIPTION
This deck accompanied my presentation on big data at the Digital Analytics Association NYC Symposium on 12.4.13.TRANSCRIPT
Why Big Data Will Survive the Hypeand Change the Way We Work
(and live) (and think)
The City University of New York
New York CityDecember 4, 2013
@BlairReeves
Blair ReevesProduct Lead, IBM Digital Analytics
IBM.com/digitalmarketing
I live here:Durham, North Carolina
@BlairReeves
@BlairReeves
“The Year of Big Data”
… is every year. From now on.Credit: Gartner Research
@BlairReeves
The Value of Data is Increasing
@BlairReeves
The Value of Data… is still being decided
Book Value: $13 billion
Market Value: $114 billion= 1.3 billion MAUs ~500 terabytes of data added… per day
$101 billion in data
@BlairReeves
A Short History of Data
Great Library of Alexandria (Egypt)
300 B.C.
Al-Azhar University (Egypt)
Invention of the Gutenberg printing press
Copyright law is born
Press freedom guarantees; pamphleteering
Cambridge University owns 122 books
Martin Luther translates the Latin Bible, accelerating mass literacy
Watson
970 A.D.
1400
1450s
1520s
17101770s
Herman Hollerith invents machine-readable data for U.S. Census
~2.8 billion global internet users(40% of world’s population)
ARPANET – first TCP/IP Protocol
1969
1890
2013
@BlairReeves
The Way We Use Data Will Change
Trade Exactitude for Size
Why Sample?
Correlation Over Causality
@BlairReeves
1 – Trade Exactitude for Size
Precision < Size
More data > Better algorithms
@BlairReeves
250 word pairs
1954 1990
3 million word pairs2006
>100 billion word pairs(and counting)
1 – Trade Exactitude for Size
@BlairReeves
2 – Why Sample?
• Sampling relies on randomness
• Difficult to drill down into subcategories
• Requires careful pre-planning
@BlairReeves
2 – Why Sample?
• Sumo wrestlers• Google Flu• Non-linear
relationships (social media)
@BlairReeves
3 – Correlation Over Causality
When does knowing “why” matter?
Data rather than hypotheses
Correlations are value
@BlairReeves
3 – Correlation Over Causality
A/B Testing Attribution
@BlairReeves
“Everything is obvious once you know the answer.”
- Duncan Watts
