escwa sdmx workshop
Post on 08-Jan-2016
67 Views
Preview:
DESCRIPTION
TRANSCRIPT
© Metadata Technology
What is SDMX?
• The problem space (2001):– Statistical collection, processing, and
exchange is time-consuming and resource-intensive
– Various international and national organisations have individual approaches for their constituencies
– Uncertainties about how to proceed with new technologies (XML, web services …)
© Metadata Technology
What is SDMX?
The Statistical Data and Metadata Exchange (SDMX) initiative addresses these challenges and opportunities that have just been mentioned:– By focusing on business practices in the field
of statistical information– By identifying more efficient processes for
exchange and sharing of data and metadata using modern technology
© Metadata Technology
Who is SDMX?
• SDMX is an initiative made up of seven international organizations:– Bank for International Settlements– European Central Bank– Eurostat – International Monetary Fund– Organisation for Economic Cooperation and
Development– United Nations– World Bank
• The initiative was launched in 2001
© Metadata Technology
Official Recommendations
• SDMX has been officially recommended:– February 2007: SDMX endorsed by the
European Union’s Statistical Programme Committee (now ESS Committee)
– March 2008: UN Statistical Commission declares SDMX to be the preferred standard for data and metadata
SDMX: Benefits
• For Data Consumers– Search for, locate, and query data and related metadata– Query directly the database and metadata repositories of
the data and metadata producers– View data and related metadata in table, charts, graphs,
maps using SDMX visual components
• For Data Collectors– Share the data collection burden– Eliminate many and varied formats for data and reference metadata– Automate the database and metadata repository loading process to
assist validation
SDMX: Benefits
• For Data Reporters– Common reporting format (including coding schemes) for
all reporting needs– Ability to open databases and metadata repositories to
direct query for data collectors
• For Data Publishers– Automated web dissemination system for data and
reference metadata– As soon as data is published it is available for query, and
linked to the reference metadata
© Metadata Technology
Exchange Patterns
• Bilateral: Institutions exchange data according to bilateral agreements regarding format, timing, protocols, etc.
• Gateway: Institutions share the data they collect with their peers, in agreed formats among counterparty communities
• Data-sharing: standard exchange of data using standard formats and protocols
© Metadata Technology
Data/Metadata-Sharing Exchange
Registration of Data Source
query/retrieval
RSS or SDMX Notification
Data/Metadata Reporters
Data/Metadata Consumers
© Metadata Technology
Adopters/Interest• The following are known adopters (or planning to adopt):
– US Federal Reserve Board and Bank of New York– European Central Bank– Joint External Debt Hub (WB, IMF, OECD, BIS)– UN/TRADECOM at UN Statistical Division– NAAWE (National Accounts from OECD/Eurostat)– European Statistical System (Eurostat and National Statistical Institutes)– Mexican Federal System– IMF (BOP, SNA, SDDS/GDDS)– Food and Agriculture Organization– Millennium Development Goals (UN System, others)– International Labor Organization– Bank for International Settlements– OECD– World Bank– Marchioness Islands (Spanish/Portuguese Statistical Region)– UNESCO (Education)– Australian Bureau of Statistics– International Energy Agency– World Health Organization– There are many others
© Metadata Technology
SDMX Status and Future
• Revised version 2.1 standards have been released (end April)– Increasing availability of SDMX tools
• In May 2011 the IMF and World Bank hosted a 3-day SDMX Global Conference– 250 people attended– Many implementation scenarios– Focus on web services– There was a vendor demonstration area on day 3
• Commitment of the Sponsors– 2011 – Web “Sandbox” for users to play with SDMX– 2012 - Global SDMX Registry
• Retrieval of common structural metadata by local registries• Sharing of structural metadata• Portal to find data and SDMX web services
– 2015 - 20 International Data Structure Definitions– World Bank and IMF asked to help with training and tools for
developing nations
© Metadata Technology
SDMX and Domain Standards
• SDMX are keen that organisations prominent in a specific domain take the lead to define the domain standards– Structural definitions
• Code lists• Concepts• Data structure definitions• Metadata structure definitions
– Quality metadata and quality frameworks• Based on the SDMX content oriented guidelines
© Metadata Technology
Dataor Metadata
Structure Definition
SDMX Version 2.0/2.1: High Level Overview
Category Scheme
CategoryData or Metadata
Flow
Data Provider
Provision Agreement
Data Set or Metadata
Set
Content
Constraint
Structure and Item Scheme
Maps
Registered Data Source or
Metadata Source
Attachment
Constraint
Categorisation
Process
© Metadata Technology
Dataor Metadata
Structure Definition
Data or Metadata
Flow
Data Provider
Registered Data Source or
Metadata Source
Register
Data or Metadata Source
references
Category Scheme
Category Categorisation
Provision Agreement
Data Provider Scheme
High Level Overview: Flow
© Metadata Technology
Existing Websites Using SDMX
• BIS– http://stats.bis.org
• ECB– http://www.ecb.int/stats/exchange/eurofxref/html/index.en.html
– http://stats.ecb.europa.eu/stats/sdmx/visualisation/icp/dashboard/rc1/
Code available at (http://flex-cb.googlecode.com/ )
• OECD– http://stats.oecd.org/WBOS/index.aspx– http://stats.oecd.org/OECDregionalstatistics/
© Metadata Technology
SDMX: Course Introduction
• At the end of the Sessions on SDMX you will have– Understood the scope of SDMX– Gained a high level understanding of the SDMX
Information Model– Know the key SDMX structures that can be used to
create, report, store, maintain, visualise, MDG data and structural metadata
© Metadata Technology
SDMX: Course Introduction
• In order to do this you will learn about– SDMX Structural Metadata
• DSD and related Concepts and Code lists• MSD and related Concepts and Code lists• Category Scheme and Classification• Data flow and Metadata flow• Provision Agreement• Registration• Subscription and Notification• Registry• Database• Query
– SDMX Data and Metadata• Data set formats• Metadata set
– SDMX Web Services
© Metadata Technology
Tools Demonstrated
SDMX Structure and Data Converter
SDMX Relational Database
SDMX Data Viewer
top related