1. SDMX as basis for water data reporting
Working Group on Education and Training Statistics
25 April 2013
Daniel Suranyi
Eurostat B5
Management of statistical data and metadata

2. Questions What is SDMX? Why use it? How do we get there?
Where are we today?
What can data providers do?

3. Statistical Data and Metadata EXchange
World Bank
UNSD
SDMX
ISO TS 17369 3 3 3 3

4. The SDMX Components SDMX Information Model Content Oriented Guidelines
IT Infrastructure for exchange and sharing
Content Oriented Guidelines
SDMX Information Model 4 4

5. The SDMX Components Describe statistics in a standard way
Objects and their relationships
Data Structure Definition (DSD), Concepts, Code List
Central management and standard access
SDMX Registry, SDMX Web Services
Cross Domain Concepts
Cross Domain Code Lists
Statistical Domains
Metadata Common Vocabulary
Push
Provider generates and sends file to receiver
Pull
Provider opens web service to data
Receiver downloads regularly
Hub
Special case of pull: receiver downloads on end user request 5 5

6. ...not just a transmission format !
The SDMX Components
IT Infrastructure for exchange and sharing
Content Oriented Guidelines
SDMX Information Model
...not just a transmission
format ! 7 7

7. What is a DSD? DIMENSIONS ATTRIBUTES MEASURES FREQUENCY COUNTRY
TIME
COUNTRY
FREQUENCY
TOURISM TOPIC
OBS_VALUE
OBS_STATUS P E
DIMENSIONS
ATTRIBUTES
MEASURES

8. What is a DSD? Describe statistics in a standard way DIMENSIONS
TIME
COUNTRY
FREQUENCY
TOURISM TOPIC
OBS_VALUE
OBS_STATUS P E
Describe statistics in a standard way
DIMENSIONS
ATTRIBUTES
MEASURES

9. Questions What is SDMX? Why use it? How do we get there?
Where are we today?
What can data providers do?

10. Eurostat SDMX Implementation Process

11. Questions What is SDMX? Why use it? How do we get there?
Where are we today?
What can data providers do?

12. Eurostat project status (data exchange)
JVS/LCI (Job Vacancy Statistics/ Labour Cost Index)
Water
International Trade*
Education?
STS (Short Term Statistics)
TEC (Trade by Enterprise Characteristics)
Maritime
R&D* (Research & Development)
National Accounts*
Balance of Payments*
EGR IS (EuroGroups Register Identification Service)
Aviation
Fisheries*
Orchards
Pesticides
TIC (Trade by Invoicing Currency)
Waste
* Other international organisations involved

13. Questions What is SDMX? Why use it? How do we get there?
Where are we today?
What can data providers do?

14. SDMX Implementation in Member States
Tools offered
Action
Web Forms (only selected domains)
No costs for organisation
Manual work for senders (type, copy/paste)
SDMX Converter
(sender’s PC)
Installation on sender’s PC
Manual work for senders (convert)
(batch mode)
Installation on server
SDMX Reference Infrastructure
Mapping of database to DSDs
Own SDMX Implementation
Local development in organisations

15. SDMX Implementation in Member States
Example SDMX Converter
Generate local data file
Take DSD and data file
Produce SDMX-ML
All steps each time
Example SDMX-RI (installed in ~20 NSIs for Census Project)
Take DSD
Link to database
Produce SDMX-ML
Step 1 & 2 only once
Only step 3 each time

16. Data Validation Process
Before/During Transmission (“First Level”)
- Implicit through SDMX
- Format Check (SDMX-ML)
- Code Check (SDMX DSD)
After Transmission ( “Second Level”)
- Currently not in SDMX
- Detailed value check
- Mirror check
- …

17. Support for data providers
SDMX Reference Infrastructure
Installed in ~20 countries (NSIs) for Census Hub
Can be extended to other domains
1st contact point - Local Transmission Coordinators
In National Statistical Institutes
Experience with EDAMIS and SDMX
2nd contact point - Eurostat support teams

18. Questions? SDMX Support Team SDMX Website Eurostat SDMX Info Space
SDMX Website
Eurostat SDMX Info Space