ray poynter big data and advanced analytics

Ray Poynter, The Future Place – JMRX Lectures 2015

Big Data and Advanced Analy0cs

Ray Poynter The Future Place

JMRX – Tokyo – May 12, 2015


Agenda

1.  What is big data?

2.  The strengths of big data 3.  When big data misses the mark

4.  The challenges of big data 5.  Linking big data, analyEcs and market

research

6.  ImplicaEons for Japan


How big is big data? BIG! •  If it fits in Excel, it is not really big data •  If it fits in SPSS, it is not really big data •  If you have a profile of 10 million telco customers, it is not really big data

•  If you have second-‐by-‐second locaEon and usage data for 10 million telco customers, that is big data

•  If you need to use Hadoop, it is big data


What is ?

InstrucEon

Results


IBM’s four Vs


What does Google Know?

And shares with you


Google Ads http://www.google.com/settings/ads/


Loca0on History https://maps.google.com/locationhistory

7 days of travel


Google Search History https://www.google.com/history/


Google Other

4.  Google monthly security and privacy report – lisEng all the services you use hVps://www.google.com/seYngs/dashboard

5.  Apps and extensions that have access to your Google data hVps://security.google.com/seYngs/security/permissions

6.  Export all your Google data hVps://www.google.com/takeout

7.  History of all your YouTube searches hVps://www.youtube.com/feed/history/search_history


The Signal and the Noise – Nate Silver


Weather forecas0ng

PMSL – Pressure at Mean Sea Level


Weather forecas0ng benchmarks

1. Same as today

2. Same as average of last few years -‐ climate

What benchmark are you going to use for your predictive analytics?


Google Flu Trends

http://www.google.org/flutrends/ December 2014


Google Flu Trends

http://www.google.org/flutrends/ May 2015


Google Flu Trends -‐ Japan


Target


BPP and USA Prices Index


Tesco •  Loyalty card data

•  Real-‐Eme monitoring of refrigerators across 120 stores in UK and Ireland to save 20million Euros a year –  70 million data points

•  Real-‐Eme monitoring of lighEng and heaEng, across 120 stores – Management dashboard shows by 7am which stores are not at the right temperature

•  Buying SocialmaEcs – programmaEc and re-‐targeEng ads


INTERESTING BIG DATA PROJECTS


Ebola and mobile phones

Flowminder using data from telcos to map populaEon mobility – but level of granularity is quite large – journeys over 20KM


The smartphone Accelerometer

Temperature

Gravity

Gyroscope

Light

Air pressure

Proximity

Humidity

GPS

Call acEvity

App acEvity

Internet usage

WiFi

Bluetooth

Cameras

Near Field Comms

GSM/CDMA


Aberdeen tracks ‘hundreds of thousands’


Kingsgate Shopping Mall Huddersfield, Yorkshire (UK)


Mount Sinai Hospital

Mount Sinai are using Big Data approaches to uElise data to make personalised health and treatment predicEons. Project led by Jeff Hammerbacher, a 30-‐year-‐old known for being Facebook’s first data scienEst.


Big Data Failures


The Signal and the Noise – Nate Silver


Earthquakes

Given the abruptly self-organizing nature of earthquakes, it is extremely unlikely that precursors can attain such levels of accuracy. I therefore conclude that prediction of major earthquakes is, in any practical sense, impossible. Russ Evans, 1998


Big Data thinks I am a pregnant dude!

http://www.carolroth.com/blog/google-dude-target-pregnant/


Google Flu – the failures


Flu Near You – a crowdsourced approach


The Samaritans App


ScrapeGate


BIG DATA CHALLENGES

Source: Gizmodo

Correla0on Annual Chocolate Consump0on & Nobel Prizes per 10 Million of Popula0on

New England Journal of Medicine.


Correla0on and Causa0on

1.  CorrelaEon predicts the past – Which is someEmes enough – Especially when the past repeats itself

2.  CausaEon is needed to predict new futures – But causaEon is hard to establish in the real world

3.  Experiments are key – Market research can help


Perverse incen0ves

•  Colonial Hanoi – bounEes for rats tails •  Duplessis Orphans – Canada 1945-‐60, Orphans=70 cents,

mentally ill $2.25, 20K children confined •  Paying for acEon (e.g. doctors and firemen) decreases

prevenEon work •  Facebook likes correlate with success – unless they are set as

a target, creaEng a black market in them •  TwiVer menEons correlate with success – unless they are

benchmarked, creaEng value in them •  Big Data benchmarks change behaviour to improve scores,

rather than underlying performance


Where are the data scien0sts going to come from?

Google Trends


MARKET RESEARCH AND ANALYTICS


A\ribu0on Modelling


Knowing where to dig and digging Lucien Bowater, Director Strategy and Insight at BSkyB – UK media company MRS Conference, UK, 2013

Issue MR Digs Big Data AcEon CEO

PaVern Big Data Digs MR AcEon CEO


T hVps://www.ted.com/talks/ben_wellington_how_we_found_the_worst_place_to_park_in_new_york_city_using_big_data


Where to dig and digging

1.  When is rush hour? MR asks the quesEon, Big Data digs

2.  Why is this fire hydrant generaEng so much money in parking Eckets? Big Data asks the quesEons, but it was Qual that answered it.


Thank You!

Questions?


IMPLICATIONS FOR JAPAN?

1.   What new skills do we need? 2.   What new people do we need? 3.   What new tools do we need? 4.   How should MR work with Big Data

and Analy0cs?

ray poynter big data and advanced analytics

Marketing

future place jmrx lectures

future place jmrxtokyomay12

googleflutrends http

exportallyourgoogledata

googleads http

loca0onhistory https

googlesearchhistory

isbigdata ifyouneedtousehadoop