June 2017
Big Data at BBVA Research
using BigQuery Tomasa Rodrigo
Google Cloud Next
Click here to modify the style of the master title
Big Data at BBVA Research using BigQuery
Summary
What is GDELT and how BigQuery helps us to exploit it 01
Geopolitical analysis 02
Economic and digital analysis 03
Big Data at BBVA Research using BigQuery
01 What is GDELT and how
BigQuery helps us to exploit it
Big Data at BBVA Research using BigQuery
What is GDELT?
4
… georeferenced across
the entire planet…
… including over 300 events
around the world and more than
30000 themes…
…and collecting emotions using
some of the most sophisticated
algorithms
Open database of human
society from every corner of the
globe dating back to 1979 …
Global Database on Events Location and Tone
Big Data at BBVA Research using BigQuery
Why do we use BigQuery?
5
Really fast dealing with Big
Databases
Easy to use using SQL
Open access
Complex data analysis
Flexibility and scalability
Combination of historical
data with real time data
Big Data at BBVA Research using BigQuery
Our working proccess
6
GDELT (Global Database
on Events,
Location
and Tone)
Clean,
Aggregate
& transform
the data
Fuse,
visualize
& analyze
the data
BigQuery Data
Storage
(SQL)
Big Data at BBVA Research using BigQuery
Why is it important?
7
Among other things we
can focus in news
intensity, geographic
density of events and
emotions across the
world
Novel data-driven
computational
approaches as needed
to enable a new era in
which these data can be
used to study “real life”
at population scales
At its core, we analyze
geopolitical, political,
social and economic
questions using
quantitative data-
driven methods rather
than qualitative
introspection
Big Data at BBVA Research using BigQuery
What is GDELT and how BigQuery helps us to exploit it
Our products
Political, Geopolitical Social Indexes (Political Indexs)
Color Maps NAFTA Topics (Nafta Project)
Politics & Financial Networks (Political Netwoks)
Mix Hard data & Sentiment & VAR models (CBSI and Turkey Sentiment Indexes)
Geographical Analysis Housing Prices (sentiment on Housing Prices)
Measuring Sentiments (sentiment Analysis on Economy and Society)
Financial Stability & Macroprudential (ECB & FED FS index by FED Board)
8
Big Data at BBVA Research using BigQuery
Tracking Geopolitics on real time is useful to identify the main hot spots and potential spillovers
9 Source: www.gdelt.org & BBVA Research
Conflict Intensity Map May 2017 (Number of conflicts/ Total events)
Big Data at BBVA Research using BigQuery
From an historical perspective…
10 Source: www.gdelt.org & BBVA Research
BBVA Research World Protest and Conflict Intensity Index 1979-2017
World Protest Intensity Map 1979- 2017
79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17
USA
UK
Norway
Sweden
Austria
Germany
France
Netherlands
Italy
Spain
Belgium
Ireland
Portugal
Greece
Poland
Czech Republic
Hungary
Bulgaria
Romania
Croatia
Turkey
Russia
Ukraine
Georgia
Kazakhstan
Moldova
Azerbaijan
Armenia
Morocco
Algeria
Tunisia
Libya
Egypt
Israel
Jordan
Syria
Iraq
Iran
UAE
Bahrain
Qatar
Oman
Saudi Arabia
Mexico
Brazil
Chile
Colombia
Peru
Argentina
Venezuela
China
Hong Kong
Korea
Thailand
Indonesia
Malaysia
Philippines
India
Pakistan
Afghanistan
EM
Eu
rop
e &
CIS
Develo
ped
Mark
ets
N.
Afr
ica &
Mid
dle
East
LA
TA
MA
sia
Protests Conflict
Big Data at BBVA Research using BigQuery
… to the main hot spots
11 Source: www.gdelt.org & BBVA Research
BBVA Research Refugees Flows Map in 2015-17 Number of media citations about refugees’ inflows and outflows
BBVA Research Asia Conflict Intensity Index 2008-17
Big Data at BBVA Research using BigQuery
Social unrest events across the world: Cairo, Istanbul and Hong Kong cases Protest events
Source: www.gdelt.org & BBVA Research
Geopolitical analysis …at the exact geolocation
Big Data at BBVA Research using BigQuery
Spanish perception around the world according to the media
Negative tone
Positive tone
Neutral tone
Source: www.gdelt.org & BBVA Research
Economic and digital analysis Sentiment analysis: the case of Spain
Big Data at BBVA Research using BigQuery
Contagion effects of China’s slowdown
14 Source: www.gdelt.org & BBVA Research
Chinese slowdown: media perception and country network
Oman
Qatar
Iran
Kazakhstan
Russia
U.A.E.
Iraq
NicaraguaSaudi ArabiaMexico
Chile
Dominican R.Brazil
Bolivia
Ecuador
Venezuela
Peru
Panama
Argentina
Spain
Austria
Ukraine
Israel
Greece
Poland
Belgium
Czech Republic
ItalyNetherlands
Finland
Ireland
Iceland
Portugal
Hungary
Yemen
Sri Lanka
Macau
Indonesia
Philippines
Taiwan
Cambodia
Pakistan
Turkey
Brunei
N. Zealand
Burkina Faso
Singapore
Thailand
Malaysia
Zimbabwe
UgandaNigeria
Zambia
CongoMozambique
Kenya
Sweden
Angola
E. Guinea
EthiopiaSouth Africa
France
US
UK
Japan
Australia
Canada
S. Korea India
Switzerland
Germany
Hong Kong
China
Big Data at BBVA Research using BigQuery
Measuring Chinese uncertainty
15 Source: www.gdelt.org & BBVA Research
Chinese Vulnerability Sentiment Index (CVSI): components and evolution
-3
-2
-1
0
1
2
3
Ma
r-15
Apr-
15
Ma
y-15
Jun-1
5
Jul-15
Aug-1
5
Sep-1
5
Oct
-15
Nov-1
5
Dec-1
5
Jan-1
6
Feb-1
6
Ma
r-16
Apr-
16
Ma
y-16
Jun-1
6
Jul-16
Aug-1
6
Sep-1
6
Oct
-16
Nov-1
6
Dec-1
6
Jan-1
7
Feb-1
7
Ma
r-17
Apr-
17
7 days mov.avg 30 days mov.avg
3% dev aluation
Economic work conf erence
Neutral Area +- 0.5 std
1st stock market crash
"Black Monday "
2nd stock market crash,
trade halted f or 3 day s
PMI f alls to 4-Y low
NPC meeting
RMB enters IMF’s SDR basket
De
clin
ing
Se
ntim
en
t (H
igh
er
Vu
lne
rab
ilit
y)
Imp
rov
ing
Se
nti
me
nt
(Lo
we
r V
uln
era
bil
ity
)
Big Data at BBVA Research using BigQuery
Turkey GDP & Economic Sentiment (%YoY and media economic sentiment in Turkish and English)
Systematic bias!
Economic and digital analysis
Narratives matter: language bias
Source: www.gdelt.org & BBVA Research
Big Data at BBVA Research using BigQuery
Media Sentiment Digital Index
17 Source: www.gdelt.org & BBVA Research
Media sentiment digital index and components
-2
-1,5
-1
-0,5
0
0,5
1
1,5
2
2,5
Ma
r-1
5
Ap
r-15
Ap
r-15
Ma
y-1
5
Jun-1
5
Jun
-15
Jul-1
5
Au
g-1
5
Au
g-1
5
Se
p-1
5
Oct
-15
No
v-1
5
No
v-1
5
De
c-1
5
Jan
-16
Jan
-16
Fe
b-1
6
Ma
r-1
6
Above average
ON average
Below average -2,4
-1,4
-0,4
0,6
1,6
2,6
ap
r-15
ma
y-1
5
jun
-15
jul-15
au
g-1
5
sep-1
5
oct
-15
no
v-1
5
de
c-1
5
jan
-16
feb
-16
ma
r-1
6
Digital Economy
-2,4
-1,4
-0,4
0,6
1,6
2,6
ap
r-15
ma
y-1
5
jun-1
5
jul-1
5
au
g-1
5
sep-1
5
oct
-15
no
v-1
5
dec-
15
jan
-16
feb-1
6m
ar-
16
Fintech
-2,4
-1,4
-0,4
0,6
1,6
2,6
ap
r-15
ma
y-1
5
jun
-15
jul-
15
au
g-1
5
sep-1
5
oct
-15
no
v-1
5
de
c-1
5
jan
-16
feb
-16
ma
r-1
6
Digital Regulation
Emerging Asia US Europe Confidence Interval for Emerging Asia
Big Data at BBVA Research using BigQuery
You can find us at:
18 Source: www.gdelt.org & BBVA Research
Media sentiment digital index and components
BBVA Research webpage
June 2017
Big Data at BBVA Research
using BigQuery Tomasa Rodrigo
Google Cloud Next