new techniques and technologies for statistics · jaap bloem, menno van doorn, sander duivestein,...
TRANSCRIPT
![Page 1: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/1.jpg)
Eurostat
Walter J. Radermacher Chief Statistician of the EU
NTTS 2013 – Brussels, 5-7 March 2013 Opening address, 5 March 2013
New Techniques and Technologies for Statistics
Eurostat
![Page 2: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/2.jpg)
Need for research in official statistics?
W. J. Radermacher 2
• Increasing emphasis on evidence-based policymaking • New sources of evidence (big data) • Theories, frameworks and tools for evidence-based
policymaking is still “work in progress” • Need at policymaker level: e.g. ”beyond GDP” indicators • Need upstream: e.g. methods for integrating big data into
official statistics in a sound way
Study on the need for research in Official Statistics published in 2012 by Eurostat: http://www.cros-portal.eu/content/analysis-future-research-needs-official-statistics-0
Eurostat
![Page 3: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/3.jpg)
Overview • Challenges and opportunities:
• Zero growth • New technologies • Big Data and Open Data • Indicators
• Instruments • EMOS • ESSnets (in the ESS.VIP programme) • The CROS portal • Horizon 2020
• NTTS 2013
3 W. J. Radermacher
Eurostat
![Page 4: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/4.jpg)
Challenges and opportunities
requiring research in official statistics
4 W. J. Radermacher
Eurostat
![Page 5: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/5.jpg)
Zero-growth scenario: to do more with less • Alternative sources to surveys
• registers • administrative data • data in the public domain
• Use modelling • Small Area Estimation • Imputation
• Increase efficiency • Standardisation • Common tools across domains – and across Europe • The ESS.VIP programme
• Respondent burden is also a limited resource!
5 W. J. Radermacher
Eurostat
![Page 6: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/6.jpg)
The ESS.VIP programme Aims
• realising economies of scale and productivity gains through sharing information, services and costs;
• at developing a common ESS infrastructure and appropriate legal framework and new administrative mechanisms that will allow for sharing of information, services and costs among ESS partners.
Components 1. Building up common infrastructure for sharing information and
services; 2. Piloting the sharing of information, services and costs in selected
statistical domains; 3. Developing frameworks and administrative mechanisms for
sharing information, services and costs.
6 W. J. Radermacher
Eurostat
![Page 7: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/7.jpg)
ESS.VIP programme – the “to-be-state”: Where do we want to be in 10 years?
7 W. J. Radermacher Eurostat
![Page 8: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/8.jpg)
New technologies
Data capture: • Travel surveys by means of GPS • Internet panels • Online census • Smartphones • Handheld devices
Dissemination: • Smartphones – again! • Visualisation (HTML5, Geographical Information Systems)
8 W. J. Radermacher
Eurostat
![Page 9: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/9.jpg)
The data deluge challenge:
Three broad areas for experimentation:
• Combining Big data with official statistics. • Replacing official statistics by Big data. • Filling new data gaps, i.e. developing new “Big data – based”
measurements to address emerging phenomena (not known in advance […])
Big data and its Potential Use by the Statistical Community. HLG working paper 2013/6
9 W. J. Radermacher
Eurostat
…we need to learn how surveys can be designed to incorporate transaction data continuously produced by the Internet and other systems in useful ways […] the biggest payoff will lie in new combinations of designed data and organic data, not in one type alone. Robert M. Groves (2011), Three eras of Survey Research, Public Opinion Quarterly, Vol. 75, No. 5, 2011, pp. 861–871.
(“big data” or “organic data”)
![Page 10: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/10.jpg)
Do Big data render surveys obsolete?
• A huge untapped potential – but we need methods for integrating big data into official statistics in a sound way
• Otherwise, representativity and precision may end up in the back seat – and the “evidence” presented may mislead policymakers
10
...a field of Sentiment Analysis has sprung up in the past decade to address the question What do people feel about a certain topic? [...] The Global Pulse project of the United Nations makes use of sentiment analysis among the population to predict unemployment... Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with Big Data, Vint research report 2 of 4
W. J. Radermacher
Eurostat
“Why LFS when there are social media?”
![Page 11: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/11.jpg)
Are administrative registers an intermediate level?
• Far smaller than ”big data”, but: • More comprehensive coverage than ”big data” • More structured than ”big data”
• Larger than survey data, but:
• Not always designed with statistics in mind
• Data warehousing technology needed for effectively integrate administrative data and survey data
11 W. J. Radermacher
Eurostat
![Page 12: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/12.jpg)
Free dissemination of EU statistics
• On 1 October 2004 Eurostat made all its (non-confidential) data available free of charge on its website
• “…downloading and reproduction of Eurostat data/documents for personal use or for further non-commercial or commercial dissemination are authorised provided appropriate acknowledgement is given to Eurostat as the source...” http://epp.eurostat.ec.europa.eu/portal/page/portal/about_eurostat/policies/copyright_licence_policy
• Eurostat data linked to e.g. by • the Open Knowledge Foundation • LATC (7th Research Framework Programme)
12 W. J. Radermacher
Eurostat
![Page 13: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/13.jpg)
Open data
• Official statistics free of charge a good example
• Some of the ”big data” are also open data (and may be used in official statistics), other ”big data” are proprietary
• Statistical Disclosure Control expertise is needed to guarantee that “big data” and “open data” are combined in such a way that the privacy of individual citizens and enterprises is protected.
13 W. J. Radermacher
Eurostat
![Page 14: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/14.jpg)
Indicators: GDP – and beyond
• Improving the quality of the GDP estimate
• Indicators going beyond the traditional GDP
• Sound statistical methods needed for new indicators – and for assessing their precision
14
Statistical techniques and computing technologies have also developed beyond recognition since the inception of national accounts and GDP. […] There are no insuperable technical obstacles to developing the quality and scope of our indicators even further so that policy decisions can progressively be based on a more integrated, balanced and timely view of social, economic and environmental facts.
GDP and beyond – Measuring progress in a changing world; communication COM(2009) 433 from the Commission
W. J. Radermacher
Eurostat
![Page 15: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/15.jpg)
European instruments
facilitating research and education
in official statistics
15 W. J. Radermacher
Eurostat
![Page 16: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/16.jpg)
EMOS – investing in people
• The European Master in Official Statistics (EMOS) is:
• project aimed at developing a programme for Training and Education in Official Statistics
• certified training in methodologies, statistical surveys, statistical production, analysis and statistical law
• offered by a network of National Statistical Institutes and universities
• The first EMOS courses may be offered at the end of 2014;
16 W. J. Radermacher
Eurostat
![Page 17: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/17.jpg)
The ESSnet projects • Funded by the European Commission (Eurostat)
• In an ESSnet project, National Statistical Institutes collaborating on European solutions to common problems
• Dissemination of ESSnet project results to non-participating ESS members a core principle
• The new ESSnet programme has recently been reoriented to: • Emphasis on “top-down”/vision implementation ESSnet projects,
embedded in the ESS.VIP programme
• Ascertaining the sustainability of ESSnet project results through Centres of Competences
17 W. J. Radermacher
Eurostat
![Page 18: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/18.jpg)
The CROS portal
• Funded by the European Commission (Eurostat)
• An important collaboration instrument
• A dissemination channel for ESSnet projects
• CROS also serves the Official Statistics research and methodology community at large (>1500 registered users) • A large CROS community for Seasonal Adjustment
• If researchers express an interest, and are prepared to take the lead Eurostat may set up new CROS communities to facilitate international collaboration on a topic
18 W. J. Radermacher
Eurostat
![Page 19: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/19.jpg)
EU support to research
Main channel for support in European statistics: Research Framework Programmes
• FP5 (DACSEIS, EURAREA, EUREDIT, …)
• FP6 (ESS, SHARE, …)
• FP7 (BLUE-ETS, DwB, e-Frame, InGRID, …)
• Various FP7 research projects are represented at NTTS 2013
and at the satellite events
19 W. J. Radermacher
Eurostat
![Page 20: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/20.jpg)
Horizon 2020 • Next research framework programme (2014-2020)
• Work programmes covering 2 years each (2014-2015)
• Statistics is not a policy area of its own – but supports a variety of policy areas
• Opportunities for research related to official statistics hard to discern
• State of play:
• Parliament and Council negotiations on EU budget 2014-20 (including overall budget for Horizon 2020)
• 1 January 2014: Horizon 2020 starts; launch of first calls http://ec.europa.eu/research/horizon2020/
20 W. J. Radermacher
Eurostat
![Page 21: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/21.jpg)
Session 12A on Horizon 2020
• Objective • New networks formed (may become project consortia) • Ideas discussed (may become project proposals) • Researchers in official statistics prepared once the calls for
proposals are issued
• Format • Speed-dating event - networking IRL (in real life) • 14 early-stage project ideas will be presented (4 minutes/each) • Discussion between presenter and participants at the
simultaneous cocktail
21 W. J. Radermacher
Eurostat
![Page 22: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/22.jpg)
NTTS - Conferences on New Techniques and Technologies for Statistics
The meeting place for research in official statistics
22 W. J. Radermacher
Eurostat
![Page 23: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/23.jpg)
NTTS 2013 • 380 registered participants • Gathering key actors in research on official statistics • Large number of submissions
• Many tough choices for the Scientific Committee • An intensive programme; 37 sessions with around 150
different oral and poster presentations
• A huge collective intellectual effort (well over 1000 pages in the proceedings!)
• Possible special issue of JOS • All full papers presented here are eligible for submission • The majority of the guest editorial team is from the
NTTS 2013 Scientific Committee
23 W. J. Radermacher
Eurostat
![Page 24: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/24.jpg)
Keynote speakers at NTTS 2013 • Opening session:
• Walter Radermacher (Chief Statistician of the European Union) • Robert-Jan Smits (Research on Official Statistics in Horizon 2020) • Robert M. Groves (Official Statistics and Big Data) • Sander Duivestein (Reboot 2020)
• Keynotes throughout the conference:
• Enrico Giovannini (Statistics 2.0: a new perspective for official statistics) • Nigel Shadbolt (Open Data: New Opportunities and New Challenges) • JNK Rao (Small Area Estimation: Methods, Applications and New Developments) • Eric Graf (Variance estimation by linearization for indicators of poverty and social exclusion)
• Closing session:
• Anne Glover (Chief Scientific Adviser to the President of the European Commission) • Denise Lievesley (Head of the School of Social Science & Public Policy at King’s college) • Daniel Defays (Director a.i. for corporate statistical and IT services, Eurostat)
24 W. J. Radermacher
Eurostat
![Page 25: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/25.jpg)
Satellite events on 4 and 8 March • European FP7 and COST research projects • Data without Boundaries
• To support equal and easy access to official microdata for the European Research Area
• BLUE-Enterprise and Trade Statistics • Identifying concrete options to provide high quality and robust
statistical business information for better policy making as well as for socio-economic research.
• Survey Harmonisation with New Technologies Improvement (COST/Shanti) • Guidelines for harmonising national travel surveys across Europe.
• 120 registered participants • Special sessions during the main conference as well
25 W. J. Radermacher
Eurostat
![Page 26: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/26.jpg)
Make use of the conference format!
• Much information could be gathered online
• Webstreaming may eventually render physical meetings obsolete
• Personal meetings still have a value
• NTTS 2013 offers you the opportunity to network – and create or expand your research consortia
26 W. J. Radermacher
Eurostat
![Page 27: New Techniques and Technologies for Statistics · Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen and Erik van Ommeren (2012), Big Social - Predicting behavior with](https://reader034.vdocuments.us/reader034/viewer/2022052408/5f935edaef342d10011d4969/html5/thumbnails/27.jpg)
27
Thank you for your attention!
[email protected] http://ec.europa.eu/eurostat
Eurostat
W. J. Radermacher