Eurostat 1.SDMX: Background and purpose 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015

Eurostat BACKGROUND: Exchanging data and metadata: -is taking place against a backdrop of developments; -is faced with a number of issues; -BUT offers various opportunities. 2

Eurostat 3 - Increasing demand for data (ease of use of Internet); -Faster exchange of electronic data; -More frequent and important exchanges; -Growing types of information exchange; - between businesses - between businesses and customers - between individuals BACKGROUND DEVELOPMENTS

Eurostat 4 -Data being collected in multiple ways (surveys, files, web queries, metadata etc.); -Data being transmitted in various formats (paper, excel, flat files etc.) -Data being transmitted in various media ( , CD-ROMs, file uploads etc.); -Data being stored in various places (USB, hard drive, servers, cloud etc.); BACKGROUND ISSUES:

Eurostat 5 -Multiple organisations collecting similar or same data; -Increasing burden on organisations (collection, maintenance and managing); -The intensive and manual nature of data collection; -Errors and inconsistencies; BACKGROUND ISSUES: -Similar concepts in wording can have a different content;

Eurostat 6 ' A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty.' Winston Churchill

Eurostat 7 -Afforded by new technological developments; -For the process to be more efficient; -To improve trust and reliability; BACKGROUND OPPORTUNITIES -To improve web dissemination;

Eurostat 8 -Simplification (streamlining data flows, central management); -Standardisation (software tools, data sharing); ESPECIALLY OPPORTUNITIES FOR: -Harmonisation (data structures, concepts and code lists);

Eurostat 9 The 'Statistical Data and Metadata eXchange' is an international initiative aimed at developing and employing more efficient processes for the exchange and sharing of statistical data and metadata among international organisations and member countries. So what is SDMX? It consists of technical and statistical standards, guidelines, an IT service infrastructure and IT tools.

Eurostat 10 -SDMX is a global response: 7 international organisations as sponsors, in collaboration with countries throughout the world; What is the business case for SDMX? -SDMX is an ISO IS standard (17369): - a document, established by consensus; - approved by a recognised body; - providing rules and guidelines; - for common and repeated use; - for optimum degree of order; - viewed as safe, reliable, good quality.

Eurostat 11 -SDMX improves timeliness: - faster access to data; - move towards automation. -SDMX improves accessibility: - bilateral, gateway and data-sharing; - push and pull modes; -SDMX improves interpretability: - standardises structural metadata (the identifiers and descriptors of data); - standardises reference metadata (the content and quality of data);

Eurostat 12 -SDMX improves coherence: - uses cross domain concepts; - uses shared code lists; - uses content oriented guidelines; - reuse across domains and agencies - aims for single figure dissemination. -SDMX can reduce data errors: - some automated validation; - agreed structures for transmission; - time saved on conversion, mapping; - less manual intervention.

Eurostat 13 -SDMX can reduce the reporting burden: - pre-validated content; - automated publication; - possible 'pull' by collecting agencies. -SDMX can reduce IT development and maintenance costs: - open source approach; - no licensing costs; - shared toolbox; - improved interoperability between systems and applications.

Eurostat 14 SDMX is well suited to supporting a data sharing process: reporting every number only once.

Eurostat 15 SDMX is about changing from a multiple, diverse and complex exchange system, to a common, harmonised and standardised exchange system.

Eurostat 16 -SDMX is not investment free: it means training; it means changes. What are the downsides? -SDMX is not a magic wand: it is suited to aggregated data but not microdata. -SDMX is dynamic: software versions are updated to increase functionalities and overcome bugs.

Eurostat 17 KEY messages: -SDMX responds to a business need; -SDMX improves quality in data and metadata exchanges; -SDMX is an international standard based on shared experiences; -SDMX offers cost-efficiencies.