Statcom 2013 New YorkStatcom 2013 New York 1111
High Level GroupHigh Level Groupfor the Modernization of for the Modernization of
Statistical Products and ServicesStatistical Products and Services
Big Data: Big Opportunity!Big Data: Big Opportunity!
Gosse van der Veen,Statistics Netherlands
Statcom 2013 New Statcom 2013 New YorkYork 22
BackgroundWe are a group of chief statisticians that are noticing the rapid changes occurring in the world and we want to modernize official statistics
•Created by the CES bureau in 2010
•10 heads of national and international statistical organizations
•Strategic vision endorsed by CES in 2011/2012
Members of the HLGMembers of the HLG
Gosse van der Veen (Netherlands) - Chairman Gosse van der Veen (Netherlands) - Chairman Brian Pink (Australia) Brian Pink (Australia) Eduardo Sojo Garza-Aldape (Mexico) Eduardo Sojo Garza-Aldape (Mexico) Enrico Giovannini (Italy) Enrico Giovannini (Italy) Woo, Ki-Jong (Republic of Korea) Woo, Ki-Jong (Republic of Korea) Irena Križman (Slovenia) Irena Križman (Slovenia) Katherine Wallman (United States) Katherine Wallman (United States) Walter Radermacher (Eurostat) Walter Radermacher (Eurostat) Martine Durand (OECD) Martine Durand (OECD) Lidia Bratanova (UNECE) Lidia Bratanova (UNECE)
Statcom 2013 New Statcom 2013 New YorkYork 33
Statcom 2013 New Statcom 2013 New YorkYork 44
Problem StatementProblem Statement
2008!!
http://www.youtube.com/watch?v=vkU2liaHZd4
Statcom 2013 New Statcom 2013 New YorkYork 55
Paradigm shiftParadigm shift
In 1990 In 1990 data were scarcedata were scarce,,interpretation was readily availableinterpretation was readily available
In 2013 data are everywhere, In 2013 data are everywhere, interpretation is scarceinterpretation is scarce
From collecting to filtering of dataFrom collecting to filtering of data
Statcom 2013 New Statcom 2013 New YorkYork 66
It means that:It means that:
the the sexy jobsexy job in the next 10 years in the next 10 years will be statistician will be statistician
Or being in a seminar like thisOr being in a seminar like this
Hal Varian
Anonymous
Statcom 2013 New Statcom 2013 New YorkYork 77
Observation:Observation:
The internet and big data are changing the The internet and big data are changing the way the world operates in a profound wayway the world operates in a profound way
SmallSmall Data: Consumer Data: Consumer confidenceconfidence
Statcom 2013 New Statcom 2013 New YorkYork 88
Sentiment towards the economic climate
Netherlands
Piet Daas, Statistics Netherlands
Example 1
Big Data: Twitter textBig Data: Twitter text
Statcom 2013 New Statcom 2013 New YorkYork 99
and in Social Media
Corr = 0.88
Sentiment towards the economic climate
Netherlands
Piet Daas, Statistics Netherlands
300 M/yr
Example 1
The Potential of Big DataThe Potential of Big Data
Seeing what you asked for versus asking yourself what you see;
In the first your model defines the info you want; in the second the info defines the model you need.
Statcom 2013 New Statcom 2013 New YorkYork 1010
Big Data: Things !Big Data: Things !
Easy to doEasy to do
Insightful, more so than Insightful, more so than numbersnumbers
Outperforming surveysOutperforming surveys
Different type of Different type of qualityquality
Verification neededVerification needed
Telephone traffic
Piet Daas, Statistics Netherlands 1111
Example 2
Big Data: Things !Big Data: Things !
Telephone trafficVehicle traffic
Piet Daas, Statistics Netherlands 1212
Example 2
Statcom 2013 New Statcom 2013 New YorkYork 1313
High Level Group High Level Group Vision and Strategy:Vision and Strategy:
We have to re-invent We have to re-invent our products and our products and processes and adapt to processes and adapt to a changed worlda changed world
On Products: Innovate!
Address the global dimension– In the data and in the products
We need to learn to find data instead of surveying– Procurement and harvesting
The exponential increase of data is The exponential increase of data is the key: we MUST use those datathe key: we MUST use those data
Statcom 2013 New Statcom 2013 New YorkYork 1515
On Products: Innovate!On Products: Innovate!
Take position in the (new) information Take position in the (new) information value chainvalue chain
Rethink our products related to needs Rethink our products related to needs of a changing societyof a changing society– Who ARE our customers nowadays?Who ARE our customers nowadays?– And tomorrow?And tomorrow?
Create pockets of innovationCreate pockets of innovation– nurture talent and create right conditionsnurture talent and create right conditions
Statcom 2013 New Statcom 2013 New YorkYork 1616
On Process: ModerniseOn Process: Modernise
Common Generic lndustrialised Statistics
GSBPM GSIM
Methods Technology
Business Concepts Information Concepts
Statistical HowTo Production HowTo
conc
eptu
alpr
actic
al
Common Generic lndustrialised Statistics
GSBPM GSIM
Methods Technology
Business Concepts Information Concepts
Statistical HowTo Production HowTo
conc
eptu
alpr
actic
al
reducediversity
Statcom 2013 New Statcom 2013 New YorkYork 1717
On Process: ModerniseOn Process: Modernise
Standards save moneyStandards save money New methods for large New methods for large
volumes of datavolumes of data Minimize labor, innovateMinimize labor, innovate Collaborate, ease the Collaborate, ease the
burdenburden Achieve process quality Achieve process quality
through standardisationthrough standardisation
Common Generic lndustrialised Statistics
GSBPM GSIM
Methods Technology
Business Concepts Information Concepts
Statistical HowTo Production HowTo
con
cep
tua
lp
ract
ica
l
Common Generic lndustrialised Statistics
GSBPM GSIM
Methods Technology
Business Concepts Information Concepts
Statistical HowTo Production HowTo
con
cep
tua
lp
ract
ica
l
Strategy: GovernanceStrategy: Governance
CES initiated, but is reaching outCES initiated, but is reaching out HLG oversees execution of the HLG oversees execution of the
strategystrategy Directly in the CES-subordinate groupsDirectly in the CES-subordinate groups Formal governance arrangements underwayFormal governance arrangements underway New group structure pendingNew group structure pending
Yearly list of key priorities assigned Yearly list of key priorities assigned to appropriate expert groupsto appropriate expert groups
2012 2012 Санкт-Санкт-ПетербургПетербург 1818
New Governance StructureNew Governance Structure
Statcom 2013 New Statcom 2013 New YorkYork 1919
High Level GroupHigh Level Group
HLG ProjectsHLG ProjectsOrganizational framework
and evaluation
Organizational framework
and evaluation
Production and
methods
Production and
methods
Products and sources
Products and sources StandardsStandards
HLGSecretariat
HLGSecretariat
Conference of European StatisticiansConference of European Statisticians
CES BureauCES Bureau
Modernization Committees Executive Board
HLG ResultsHLG Results
GSIM V1.0 GSIM V1.0 (Generic Statistical Information Model) (Generic Statistical Information Model)
now operational now operational
as complement to the as complement to the GSBPM V4.0 GSBPM V4.0 (Generic Statistical Business Process (Generic Statistical Business Process Model)Model)
Statcom 2013 New Statcom 2013 New YorkYork 2020
Acquisition Program
Methodology
Statistical Program
Information Request
Dissemination Program
Statistical Project
Data Resource
Population Units
Concept
Variable
Classification
Value Domain
ProviderProvision
Agreement
Statistical ProductsData Set
Data Structure
Unit Data Structure
Cube Structure
Record Structures
Process Step Execution
Process Step Design
Process Method
Separation of Statistical, Acquisition, and Dissemination programs, with central role for Methodology
Balanced support for all data acquisition channels
Balanced support for multiple dissemination channels
Shared Data Resource, maintained corporately, for use by all statistical programs
Basic infrastructure for critical base elements
Mapping of processes to support managed operations
All structures and relationships described in metadata to support automated processes
Statistical projects access shared data
Provision for formalisation of arrangements for data acquisition and dissemination
Process management for all areas of activity
Methodology applies to processes in all areas
Process Step Definition
Process Control
Rule
ProductionBusiness
ConceptsStructures
GSIM sprint
G S I
M
V1.0
HLG ResultsHLG Results
Guidelines on Guidelines on multilingual softwaremultilingual software
Statcom 2013 New Statcom 2013 New YorkYork 2222
What is it to you?What is it to you?
Statistics will change thoroughlyStatistics will change thoroughly Leveling the playing fieldLeveling the playing field Creating opportunitiesCreating opportunities
Emerging economies can leapfrogEmerging economies can leapfrog Unburdened by legacyUnburdened by legacy Creating better infrastructuresCreating better infrastructures
We, the HLG are committed to: We, the HLG are committed to: Collaboration and sharing between NSOs Collaboration and sharing between NSOs Uniting the official statistical industryUniting the official statistical industry
Statcom 2013 New Statcom 2013 New YorkYork 2323
Next HLG stepsNext HLG steps
Project for CSPA Project for CSPA (Common Statistical Process (Common Statistical Process Architecture)Architecture)
Bringing GSBPM and GSIM to lifeBringing GSBPM and GSIM to life Shareable modular production softwareShareable modular production software
Big DataBig Data Position paper soon to be publishedPosition paper soon to be published Pilot projects; results in AutumnPilot projects; results in Autumn
Statcom 2013 New Statcom 2013 New YorkYork 2424
Next HLG stepsNext HLG steps
Possible Global conference on Possible Global conference on statistics end of this year??statistics end of this year??
Finding the bright spotsFinding the bright spots Sharing insights and solutionsSharing insights and solutions
Statcom 2013 New Statcom 2013 New YorkYork 2525
ConclusionsConclusions
More challenges are uniformMore challenges are uniform More common solutions are possibleMore common solutions are possible Time to work togetherTime to work together HLG wants to:HLG wants to:
Facilitate cooperationFacilitate cooperation Help you find solutions for the balance Help you find solutions for the balance
between official statistics and the new between official statistics and the new challenges and possibilitieschallenges and possibilities
Statcom 2013 New Statcom 2013 New YorkYork 2626
Statcom 2013 New Statcom 2013 New YorkYork 2727
Thank You
Google: UNECE HLG or: