Structural and reference metadata in the European Statistical System August Götzfried, august.goetzfried@ec.europa.eu Marco Pellegrino, marco.pellegrino@ec.europa.eu METIS work session on statistical metadata Luxembourg, 9 to 11 April 2008

Working Paper 10: Metadata in the E.S.S. Overview Functions of statistical metadata across the statistical business cycle SDMX content-oriented guidelines (COG): a step forward towards international harmonisation The implementation of the SDMX COG within the European Statistical System Metadata systems for structural and reference metadata Conclusions SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

Functions of statistical metadata across the statistical business cycle Metadata = information about data that make figures understandable to users and inform users where and how data is located and can be retrieved A metadata system provides the following functions: Search and navigation interpretation and post-processing Several metadata standards or guidelines were developed in recent years which were not always fully compatible with each other. SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

More international harmonization on metadata initiated by The SDMX Content-oriented Guidelines: a step forward towards international harmonization More international harmonization on metadata initiated by European Statistics Code of Practice, principle 15 on accessibility and clarity: “Metadata are to be documented according to a standardized metadata system” SDMX Content-oriented Guidelines (COG) released in 02/2008, in particular with regard to Cross-domain Concepts, Cross-domain code lists and Metadata Common Vocabulary. SDMX COG allow – inter alia - the creation of “Metadata Structure Definitions” for reference metadata. SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

The implementation of the SDMX COG within the ESS has started with: The implementation of the SDMX COG within the European Statistical System The implementation of the SDMX COG within the ESS has started with: the creation and the successive implementation of the Euro SDMX metadata structure (ESMS): 21 SDMX cross-domain concepts are part of the ESMS and successively implemented within the European Statistical System (attention: a better measurement of the domain specific data quality will be possible) the checking and mapping of a limited number of SDMX Cross domain-code lists (released as pivot code lists) against the code lists and descriptions used at Eurostat; further work steps are needed for completing these lists or for the adding of more code lists to the “SDMX acquis” SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

The Euro-SDMX Metadata Structure (ESMS) Generic SDMX-compliant Concept Scheme: ESMS SDMX provides a lists of 62 statistical concepts which are used across statistical domains (ANNEX 1: Cross-Domain Concepts) The new reference metadata structure of Eurostat (Euro-SDMX Metadata Structure, ESMS) uses 21 of those concepts, with a limited breakdown of sub-concepts Templates to be derived from the structure Transmission tests of reference metadata for SODI Logical mapping between metadata formats Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

The implementation of the SDMX COG within the European Statistical System Eurostat started to harmonize the codes to be used within the European Statistical System (first harmonized code list published in 03/2008). Full integration of the Metadata Common Vocabulary (MCV) into the Eurostat database for concepts and definitions; this promotes the MCV as standard vocabulary for structural and reference metadata within the European Statistical System. SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

Eurostat Metadata System Structural metadata: Data structure definitions, database dictionaries and around 500 code lists which will be successively harmonized Reference metadata in SDDS format which will be replaced by the ESMS format in the near future (also recommended for national reference metadata within the European Statistical System). Classifications, concepts and definitions, and domain-specific quality reports: in general, these additional metadata need to be better integrated with existing structural and reference metadata and with data. SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

National Metadata Systems Main focus on the national metadata systems existing within the European Statistical System First analysis of those national systems done by Eurostat with some first findings (such as more standardization needed, more freshness needed, metadata should be linked better to data) A more in depth assessment of the national metadata systems will be done in using a number of standard assessment criteria such as accessibility, types of metadata available, links between data and metadata, etc. Regular monitoring reports should get available in the years to come. SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

Conclusions Some big steps forward made in the international standardization and harmonization of all types of metadata in the recent years However More work needs to be done towards high-quality metadata systems at Eurostat and within the European Statistical System SDMX Tools can help defining data structures and metadata structures, but they do not replace the statistical conceptual work. Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008

Thanks for your attention August.Goetzfried Marco.Pellegrino @ ec.europa.eu Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Luxembourg, 9 to 11 April 2008