SDMX and Metadata SDMX Basics Course 12 April 2013 Daniel Suranyi Eurostat B5 Management of statistical data and metadata

The main types of metadata Structural metadata are: acting as identifiers and descriptors of the data, such as: dimensions of statistical cubes variables titles of tables navigation tree Structural metadata must always be associated with the data to allow their identification, retrieval and browsing.

Example for structural metadata

The main types of metadata Reference metadata are: acting only as descriptors of the data, they don’t help to actually identify the data. They can be of different kinds: conceptual metadata methodological metadata quality metadata (process and output) Reference metadata can be exchanged in- dependently from the data they are related to, but are however often linked to them.

Example for reference metadata

Metadata and the ESS vision The ESS vision is based on the Commission Communication 404/2009 “on the production methods of EU statistics: a vision for the next decade”. Some main ideas of this vision are: From statistical ‘stove pipes’ to more integrated statistical production processes; Better integration of the ESS in terms of IT infrastructure, IT tools, data quality, metadata, methodology etc. (both in terms of horizontal and vertical integration); Broader use of administrative data sources in the statistical data production processes; Statistical legislation should also be cross-cutting in covering larger statistical domains (first cross-cutting legislation drafted).

Standardisation of structural metadata Code lists describe dimensions in data tables, giving a meaning to the data. Are based on official statistical classifications such as NACE, NUTS, ISCO… the SDMX Content Oriented Guidelines Domain specific codifcations A standard code list is a code list already harmonised Standard code lists should be used all along the statistical business process data design, collection, aggregation, dissemination, archiving…

8 Example of a harmonised code list (NACE Rev. 1.1) Old version (before harmonisation) New version (after harmonisation) DomainsOld codesOld label_enNew codesNew label_en hrst, htecMA_TOTALManufacturing sector DManufacturing fatsMANManufacturing industries theme3RDManufacturing industry theme4B0200Manufacturing industry theme8SE0_4Manufacturing industry theme9TOT_MANUFManufacturing industry ds, hrst, htecMA_LOW_TECLow technology manufacturing sector D_LTC Low-technology manufacturing fats / innLOT Low Technology (incl. following NACE codes: 15-22; 36, 37) innI_LOW_TEC Low tech industries: NACE Rev.1 codes 15 to 22, 36 and 37 hrst, htecSE_TOTAL Services: NACE Rev. 1.1 sections G to Q = 50 to 99 G-QServices fatsSERServices sector

Impact on the statistical business processes  Better comparability: same codes for the same concepts  Increase efficiency: less transcoding; less code lists; clean lists  Improve accuracy: facilitate data management and exchange and reduce the number of errors  Re-usability and integration of the data: data warehouse are only possible if codes corresponding to the same concept are the same  SDMX implementation: it is essential for the implementation of a SDMX data/metadata exchange process.  The ESS standard code lists will also be made available in the Euro SDMX Registry (currently Ramon)

Standard Code Lists in RAMON

Standardisation of Reference Metadata ESMS Euro SDMX Metadata Structure ESQRS ESS Standard for Quality Reports Structure EPMS Eurostat Process Metadata Structure 12

Standardisation of reference metadata The Euro SDMX Metadata Structure (ESMS)

Standardisation of reference metadata The ESS Standard for Quality Reports Structure (ESQRS)

Dissemination of reference metadata

Dissemination of national reference metadata

The ESS Metadata Handler Common user Interface Output produced for the Eurostat Web Other output for Eurostat or external users Metadata from the Eurostat domain manager Eurostat as main administrator NRME EMIS RAMON CODED ESS–Metadata Handler Euro SDMX Registry Domain specific non harmonized metadata (inserted from the Eurostat production databases) Input from national metadata Common user Interface Output produced for the Eurostat Web Other output for Eurostat or external users Metadata from the Eurostat domain manager Eurostat as main administrator NRME EMIS RAMON CODED ESS–Metadata Handler Euro SDMX Registry Input from national metadata

The National Reference Metadata Editor (NRME)

Impact on the statistical business processes ESS reference metadata standards are integrated into the National Reference Metadata Editor for production, exchange and dissemination of reference metadata in the ESS, allowing: + More AUTOMATIC PRODUCTION of the reference metadata in the ESS. + Information collected ONLY ONCE and reused (ESQRS -> ESMS; NSI->Eurostat; Eurostat-> IMF/OECD). + More harmonised and better availability of metadata on quality. + Full SDMX compliance (metadata creation, exchange and dissemination). + Cost and resources savings in the ESS.

Questions? 20 SDMX Support Team SDMX Website Eurostat SDMX Info Space