sdmx overview nsf accounting interoperability workshop may 12-13 washington dc arofan gregory...
TRANSCRIPT
SDMX Overview
NSF Accounting Interoperability WorkshopMay 12-13
Washington DCArofan Gregory ([email protected])
Rene Piche ([email protected])
What is SDMX?
• The problem space:– Statistical collection, processing, and
exchange is time-consuming and resource-intensive
– Focus on aggregate data (esp. time series)– Various international and national
organisations have individual approaches for their constituencies
– Uncertainties about how to proceed with new technologies (XML, web services, SOA …)
What is SDMX?
The Statistical Data and Metadata Exchange (SDMX) initiative is taking steps to address these challenges and opportunities that have just been mentioned:– By focusing on business practices in the field
of statistical information– By identifying more efficient processes for
exchange and sharing of data and metadata using modern technology and open standards
International OrganisationsRegional Organisations
accountsstatistics
Banks, CorporatesIndividual Households
trans-actions
accounts
National Statistical Organisations
accountsstatistics
180
+ C
ount
ries
180
+ C
ount
ries
Inte
rnet
, S
earc
h, N
avig
atio
nIn
tern
et,
Sea
rch,
Nav
igat
ionwww.z.org
www.hub.org
www.x.org
www.y.org
Historical Note
• SDMX uses an approach based on the 10-year-long success of an earlier standard – GESMES/TS
• GESMES/TS was an initiative that is used today in many countries for collecting, exchanging, and updating statistical databases– GESMES/TS is now SDMX-EDI
• Focus is on time-series, and is mostly used by central banks
Who is SDMX?• SDMX is an initiative made up of seven
international organizations:– Bank for International Settlements– European Central Bank– Eurostat – International Monetary Fund– Organisation for Economic Cooperation and
Development– United Nations– World Bank
• The initiative was launched in 2002
SDMX Products• Technical standards for the formatting and
exchange of aggregate statistics:– SDMX Technical Specifications version 1.0
(now ISO/TS 17369 SDMX – TC 154 WG2)– SDMX Technical Specifications version 2.0
(soon to be submitted to ISO – TC 154 WG2)
• Content-Oriented Guidelines (in draft)– Common Metadata Vocabulary– Cross-Domain Statistical Concepts– Statistical Subject-Matter Domains
Detailed SDMX Goals• Reduce national reporting burden to international institutions• Fostering consistency, accuracy, and timeliness of data and
metadata dissseminated by national and international institutions, relying on what is decentrally released via national websites
• Enhancing national statistical processing efficiency, especially through internationally-recognised standard formats for exchanges between statistical silos within institutions and with other national statistical agencies
• Providing standards for web-based dissemination formats that are computer readable and facilitate updating of databases
• Enhancing comparison of data and metadata analysis through standard formats and content-oriented guidelines
Recent Events
• Jan 2007 – Launch meeting at the World Bank for SDMX 2.0 Technical Specifications
• February 2007 – Endorsement of SDMX by EU’s Statistical Programme Committee
• March 2008 – SDMX becomes the preferred standard for data and metadata of the UN Statistical Commission
Adoption
• By most estimates, the SDMX user base doubled in 2007
• Expectation is that growth will accelerate in 2008
• Areas of adoption include financial and monetary statistics, but also include many other domains: education, labor, population, health, etc.
Adopters/Interest• The following are known adopters (or planning to adopt):
– US Federal Reserve Board and Bank of New York– European Central Bank– Joint External Debt Hub (WB, IMF, OECD, BIS)– UN/TRADECOM at UN Statistical Division– NAAWE (National Accounts from OECD/Eurostat)– SODI (Eurostat and European Governments)– Mexican Federal System– Vietnamese Ministry of Planning and Investment– Qatar Information Exchange– IMF (BOP, SNA, SDDS/GDDS)– Food and Agriculture Organization– Millenium Development Goals (UN System, others)– International Labor Organization– Bank for International Settlements (FSI, many others)– OECD– World Bank– Marchioness Islands (Spanish/Portugese Statistical Region)– UNESCO (Education)– Australian Bureau of Statistics– Statistics Canada– There are probably others we are not aware of
Scope
• SDMX covers three major areas:– (Meta)model and XML formats for aggregate
data– (Meta)model and XML formats for “reference”
and “provisioning” metadata– Registry-based, distributed architecture
• Technical specifications and “statistical” aspects are cleanly separated
The SDMX Information Model
• SDMX is a model-driven standard– All technology implementations are an
implementation of a common meta-model– All concepts within SDMX for data and
metadata are configurable– SDMX works with any type of classification or
“ontology”– Instances of the metamodel are expressed in
a dedicated XML – SDMX-ML
SDMX-ML
• SDMX-ML is the set of XML interfaces/messages which provide:– Processable metadata (for data structures, for
metadata structures, for data and metadata provisioning)
– Data formats– Metadata formats– Registry interactions (metadata queries,
registration, classifications, concepts, etc.)– Data and metadata queries
Alignment with Other Standards
• SDMX was designed to fit into the standards environment:– Web-services and SOA standards (WSDL,
SOAP, WSS, etc.)– ISO/IEC 11179 & ebXML CCTS– ebXML Registry/Repository specifications– Data Documentation Initiative (DDI)
• SDMX and XBRL are working to produce a standard cross-walk
Support for Other Standards
• The SDMX distributed architecture is designed to work with related standards– Generic ability to model other (standard)
processes– Generic ability to register other (standard)
metadata and data formats– Ability to integrate with an ISO/IEC 11179
semantic registry
• Integration with sources and outputs is a design feature of SDMX
High-Level Vision – Standards MappingsFederated Registries (Based on SDMX, ebXML, web services)
Aggregated Data/Metadata
(SDMX)
XBRLBusiness Reports
DDI Microdata
Sets
ISO 19115Geographies
Dublin CoreCitations
Used in
registered
References to source data
Standard classifications
Organizedusing
ISO 11179
Semantic definitions
METS Packaging
Standards Alignment
• To support the submission of the SDMX Technical Specifications to ISO, papers will be provided covering several mappings/alignments:– XBRL– ISO/IEC 11179– ebXML CCTS– ebXML Registry/Repository– Data Documentation Initiative (DDI)
Questions?