![Page 1: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/1.jpg)
UN Global Pulse: Harnessing Big Data for a Revolution in
Sustainable Development and Humanitarian Action
Robert Kirkpatrick Director
@rkirkpatrick
www.unglobalpulse.org @unglobalpulse
![Page 2: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/2.jpg)
Global Pulse Vision: Big Data harnessed responsibly as a public good Mission: Accelerate discovery and adoption of big data innovation for sustainable development and humanitarian action
Pulse Lab New York
Est. 2010
Pulse Lab Jakarta
Est. 2012 Pulse Lab Kampala
Est. 2013
![Page 3: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/3.jpg)
PULSE LABS: COLLABORATIVE BIG DATA INNOVATION
![Page 4: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/4.jpg)
Why do we need a data revolution?
![Page 5: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/5.jpg)
It is no longer enough to have processes designed for less disruptive times.
![Page 6: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/6.jpg)
![Page 7: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/7.jpg)
![Page 8: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/8.jpg)
![Page 9: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/9.jpg)
Real-time analytics allows continuous observation of human behavior.
![Page 10: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/10.jpg)
real-time…
Real-time isn’t just faster. It’s different.
![Page 11: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/11.jpg)
continuous…
survey data
big data
![Page 12: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/12.jpg)
…observation…
![Page 13: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/13.jpg)
Hypothesis:
DIGITAL SERVICES ARE SENSOR NETWORKS FOR MEASURING HUMAN WELLBEING
![Page 14: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/14.jpg)
“What People Say” Online news Social media
Retail advertising Radio & TV
“What People Do”
Online search Mobile phone usage Transaction records
Postal traffic
Data sources which reveal behavior change:
![Page 15: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/15.jpg)
PRIVACY OFFICERS
http://www.unglobalpulse.org/privacy-and-data-protection
![Page 16: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/16.jpg)
Global Post2015 Dashboard
![Page 17: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/17.jpg)
Climate monitoring dashboard
![Page 18: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/18.jpg)
Program: Food & Agriculture Project: Nowcasting food prices via Indonesian Twitter Partners: WFP, BAPPENAS, University of Seoul
http://www.unglobalpulse.org/nowcasting-food-prices
![Page 19: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/19.jpg)
Program: Public Health Project: Perceptions of Vaccination in Indonesia Partners: Ministry of Health, WHO, UNICEF Product: Real-time map and trends in Twitter user sentiment toward vaccination and disease risk
![Page 20: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/20.jpg)
SITUATIONAL AWARENESS – IMMUNIZATION DISCUSSION IN INDONESIA
0
1000
2000
3000
Rank 2012-06-20 2012-10-08 2013-04-28 2013-12-23
1 Autism (213) Death(1030) Fever (1498) Death (224)
2 Death (5) Fever (14) Swelling (1494) Fever (3)
3 Sick (4) Sick (4) Pain (1491) Crying (1)
4 Fever (2) Crying (3) Autism (1011) Autism (1)
5 Crying (1) Fever (3) Fever (4) -
June 2012 Oct 2012 Apr 2013 Dec 2013
“There are some autism cases after
MMR vaccine”
“A baby suddenly died after vaccine ”
“Is it dangerous to have fever, swelling, pain, after vaccine?” “China is
investigating death cases of babies”
Monitoring a few keywords in digital media allows to understand up-to-date situation in real-time; an addition to existing MoH media monitoring for immediate action
http://www.unglobalpulse.org/immunisation-parent-perceptions
![Page 21: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/21.jpg)
EARLY WARNING AND RAPID RESPONSE
Early warning Rapid response with actionable plan
Disseminate correct information through Twitter via influential users @dr_piprim @dirgarambe @blogdoktor
……
Detect people concerned about death after vaccine from Twitter
0
200
400
600
800
1000
1200
Number of tweets of ‘death’
http://www.unglobalpulse.org/immunisation-parent-perceptions
![Page 22: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/22.jpg)
Program: Public Health Project: HIV Risk Awareness at the World Cup Partners: UNAIDS
![Page 23: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/23.jpg)
Call Detail Records
CDR Format:
Mobility Social Interaction Economic Activity
![Page 24: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/24.jpg)
Real-time Estimation of Household Food Expenditures Using Anonymized Mobile Phone Airtime Expenditures
![Page 25: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/25.jpg)
Network of night displacements during floods
Program: Humanitarian Action Project: Visualizing displacement due to floods via mobile phone data Partners: WFP, Polytechic University of Madrid, Telefonica Research
http://www.unglobalpulse.org/tabasco-floods-CDRs
Control night (random)
![Page 26: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/26.jpg)
http://www.unglobalpulse.org/tabasco-floods-CDRs
![Page 27: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/27.jpg)
Program: Privacy Project: Data privacy risk and the limits of decision making Partners: MIT
![Page 28: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/28.jpg)
Speech-to-Text Radio Mining
Record Audio
Identify Language (Luganda,
Acholi)
Convert Speech to Text
Transcript
Filter for Topics (health, food prices, jobs, crops, etc.
Analyze volume, mood,
associated topics
Visualize on dashboard
Toolkit Speech-to-text engine + near real-time analysis of live public FM radio conversation in Uganda Features location, volume, mood, associated topics, anomaly detection
PARTNERS: Leiden University, Makerere University, Sheffield University
![Page 29: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/29.jpg)
Postal Flows
![Page 30: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/30.jpg)
1. Enhanced Early Warning: Earlier detection of anomalies and events allows rapid response to crises.
1. Real-Time Awareness: Real-time trend
analysis of population activities and dynamics can inform the design and targeting of programmes and policies.
1. Real-time Evaluation: Real-time feedback
from citizens, and measurement of behavior change, allows for adaptive course corrections in programmes and policies.
Big Data for Development: 3 Opportunities
![Page 31: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/31.jpg)
Cost of data collection
Digital systems can be
significantly less resource
intensive than traditional statistics
Risk of data collection
Allows remote analysis, allowing
data to be tracked in risky or unstable locations
Speed of response
Response can significantly
improve on lag in
traditional statistics
Adaptive execution
Continuous real-time feedback
allows strategy to evolve with
changing realities on the ground
New insights
New sources provide data historically
unavailable, yielding new
insights
Advantages of using big data
![Page 32: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/32.jpg)
2014-2015 Track 1 Innovation Programs
POST-2015 BASELINING
GENDER
DATA PRIVACY
FOO
D &
A
GR
ICU
LTU
RE
ECO
NO
MIC
W
ELLB
EIN
G
HU
MA
NIT
AR
IAN
A
CTI
ON
UR
BA
N
DYN
AM
ICS
PUB
LIC
H
EALT
H
CONCEPT INCUBATOR C
LIM
ATE
CH
AN
GE
REAL-TIME EVALUATION
DATA ANALYTICS TOOLKITS
Text Mobile
Speech Transactions
![Page 33: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/33.jpg)
INNOVATING TOGETHER: Typical cycle of a joint project
![Page 34: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/34.jpg)
Conceptualization Phase I: Feasibility
Phase II: Research
Phase III: Prototype
Phase IV: Pilot
Determine a topic for exploration based on a real need, question or data gap as articulated by a "domain expert.” Confirm availability of relevant data, technical expertise & partners.
Test whether there are any relevant “signals” in the data. Assess the landscape to determine justification for moving to research phase.
Initiate research to test the hypothesis (“proof-of-concept”). Often involves comparing a digital data source against official or “groundtruth” data for comparison. Generate Findings, develop and evaluate methodology
Build or repurpose a real-time analytical software tool, based on method determined through proof-of-concept.
Implement the approach or tool in an operational setting
INNOVATING TOGETHER: Typical cycle of a joint big data project
![Page 35: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/35.jpg)
PRIVACY OFFICERS
You Can’t Do This Alone
Big Data Access • Twitter • Orange France Teleco • Telenor • Telefonica • Real Impact • Universal Postal Union
Data Mining & Analysis Technologies
• Amazon Web Services (supercomputing) • DataSift (data filtering) • SAS (analytics & data visualization) • Crimson Hexagon (data analysis)
Data Science Expertise
• Université catholique de Louvain • Institut des Systèmes Complexes de Paris Ile-de-France • Universidad Politécnica de Madrid • Stockholm University • Karolinska Institutet • University of Sheffield • Microsoft Research • Google
“Data philanthropy” – a new form of shared value corporate social responsibility in which private sector companies participate in making big data and real-time analytics available for social good.
![Page 36: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/36.jpg)
www.unglobalpulse.org
@unglobalpulse
![Page 37: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/37.jpg)
DEVELOPING FRAMEWORKS
FOR DATA PRIVACY &
SUSTAINABLE ACCESS
STRENGTHENING THE INNOVATION COMMUNITY OF
PRACTICE
SUPPORTING PUBLIC SECTOR INSTITUTIONAL
ADOPTION
SUPPLY DEMAND
Catalyzing the formation of a new information ecosystem
![Page 38: UN Global Pulse: Harnessing Big Data for a Revolution in ... · 11/24/2014 · relevant data, technical expertise & partners. Test whether there are any relevant “signals” in](https://reader034.vdocuments.us/reader034/viewer/2022042312/5eda2301b3745412b570d485/html5/thumbnails/38.jpg)