SDMX Overview NSF Accounting Interoperability Workshop May Washington DC Arofan Gregory Rene Piche

What is SDMX? The problem space: –Statistical collection, processing, and exchange is time-consuming and resource- intensive –Focus on aggregate data (esp. time series) –Various international and national organisations have individual approaches for their constituencies –Uncertainties about how to proceed with new technologies (XML, web services, SOA …)

What is SDMX? The Statistical Data and Metadata Exchange (SDMX) initiative is taking steps to address these challenges and opportunities that have just been mentioned: – By focusing on business practices in the field of statistical information – By identifying more efficient processes for exchange and sharing of data and metadata using modern technology and open standards

International Organisations Regional Organisations accounts statistics Banks, Corporates Individual Households trans- actions accounts National Statistical Organisations accounts statistics Countries Internet, Search, Navigation

Historical Note SDMX uses an approach based on the 10-year- long success of an earlier standard – GESMES/TS GESMES/TS was an initiative that is used today in many countries for collecting, exchanging, and updating statistical databases –GESMES/TS is now SDMX-EDI Focus is on time-series, and is mostly used by central banks

Who is SDMX? SDMX is an initiative made up of seven international organizations: –Bank for International Settlements –European Central Bank –Eurostat –International Monetary Fund –Organisation for Economic Cooperation and Development –United Nations –World Bank The initiative was launched in 2002

SDMX Products Technical standards for the formatting and exchange of aggregate statistics: –SDMX Technical Specifications version 1.0 (now ISO/TS SDMX – TC 154 WG2) –SDMX Technical Specifications version 2.0 (soon to be submitted to ISO – TC 154 WG2) Content-Oriented Guidelines (in draft) –Common Metadata Vocabulary –Cross-Domain Statistical Concepts –Statistical Subject-Matter Domains

Detailed SDMX Goals Reduce national reporting burden to international institutions Fostering consistency, accuracy, and timeliness of data and metadata dissseminated by national and international institutions, relying on what is decentrally released via national websites Enhancing national statistical processing efficiency, especially through internationally-recognised standard formats for exchanges between statistical silos within institutions and with other national statistical agencies Providing standards for web-based dissemination formats that are computer readable and facilitate updating of databases Enhancing comparison of data and metadata analysis through standard formats and content-oriented guidelines

Recent Events Jan 2007 – Launch meeting at the World Bank for SDMX 2.0 Technical Specifications February 2007 – Endorsement of SDMX by EU’s Statistical Programme Committee March 2008 – SDMX becomes the preferred standard for data and metadata of the UN Statistical Commission

Adoption By most estimates, the SDMX user base doubled in 2007 Expectation is that growth will accelerate in 2008 Areas of adoption include financial and monetary statistics, but also include many other domains: education, labor, population, health, etc.

Adopters/Interest The following are known adopters (or planning to adopt): –US Federal Reserve Board and Bank of New York –European Central Bank –Joint External Debt Hub (WB, IMF, OECD, BIS) –UN/TRADECOM at UN Statistical Division –NAAWE (National Accounts from OECD/Eurostat) –SODI (Eurostat and European Governments) –Mexican Federal System –Vietnamese Ministry of Planning and Investment –Qatar Information Exchange –IMF (BOP, SNA, SDDS/GDDS) –Food and Agriculture Organization –Millenium Development Goals (UN System, others) –International Labor Organization –Bank for International Settlements (FSI, many others) –OECD –World Bank –Marchioness Islands (Spanish/Portugese Statistical Region) –UNESCO (Education) –Australian Bureau of Statistics –Statistics Canada –There are probably others we are not aware of

Scope SDMX covers three major areas: –(Meta)model and XML formats for aggregate data –(Meta)model and XML formats for “reference” and “provisioning” metadata –Registry-based, distributed architecture Technical specifications and “statistical” aspects are cleanly separated

The SDMX Information Model SDMX is a model-driven standard –All technology implementations are an implementation of a common meta-model –All concepts within SDMX for data and metadata are configurable –SDMX works with any type of classification or “ontology” –Instances of the metamodel are expressed in a dedicated XML – SDMX-ML

SDMX-ML SDMX-ML is the set of XML interfaces/messages which provide: –Processable metadata (for data structures, for metadata structures, for data and metadata provisioning) –Data formats –Metadata formats –Registry interactions (metadata queries, registration, classifications, concepts, etc.) –Data and metadata queries

Alignment with Other Standards SDMX was designed to fit into the standards environment: –Web-services and SOA standards (WSDL, SOAP, WSS, etc.) –ISO/IEC & ebXML CCTS –ebXML Registry/Repository specifications –Data Documentation Initiative (DDI) SDMX and XBRL are working to produce a standard cross-walk

Support for Other Standards The SDMX distributed architecture is designed to work with related standards –Generic ability to model other (standard) processes –Generic ability to register other (standard) metadata and data formats –Ability to integrate with an ISO/IEC semantic registry Integration with sources and outputs is a design feature of SDMX

High-Level Vision – Standards Mappings Federated Registries (Based on SDMX, ebXML, web services) Aggregated Data/Metadata (SDMX) XBRL Business Reports DDI Microdata Sets ISO Geographies Dublin Core Citations Used in registered References to source data Standard classifications Organized using ISO Semantic definitions METS Packaging

Standards Alignment To support the submission of the SDMX Technical Specifications to ISO, papers will be provided covering several mappings/alignments: –XBRL –ISO/IEC –ebXML CCTS –ebXML Registry/Repository –Data Documentation Initiative (DDI)

Questions?