1. SDMX IT Tools SDMX Registry
Jean-Francois LEBLANC
Christian SEBASTIAN
29-31 May 2017
Eurostat, Unit B3 – IT solutions for statistical production

2. Table of Contents Objectives In a nutshell SDMX Global Registry
Content of the SDMX registries
Availability
Basic Principles
Relations
In which registry?
SDMX Global Registry
How to use it?

3. Table of Contents SDMX Eurostat Registry Hands-on exercise
Availability
How to use it?
Architecture
Entity-Relationship model
Interface
Hands-on exercise

4. 1. Objectives What are the different SDMX registries
What can I find in them
What are their principles
Who should use them
How to use them
What is their architecture

5. 2. In a nutshell General features of SDMX registries
Central repositories
Containing all artefacts needed for exchanging statistical information using SDMX
Publicly accessible to anyone without constraints
Advanced features of SDMX registries
Subscription to specified elements
Display of linked structures
Versioning of artefacts implying strong dependencies

6. 2.1 Content of the SDMX registries
DSDs and their related artefacts
Concept schemes
Code lists
Data flows
etc.
Metadata Structure Definitions
Various other structures
Link to related data

7. 2.2 Availability Accessible to
Production side: any authorized person or organization involved in SDMX data exchanges
Dissemination side: any end-user of statistical data
Publicly available information (exceptions may apply)
No need to log in to view or to download information
Web services

8. 2.3 Basic Principles Each registry stores its own material
Global DSDs in SDMX Global Registry
ESS DSDs in Euro-SDMX Registry
UN DSDs in UN SDMX Browser, etc.
Each registry links to the other registries
Information stored once (easier to manage)
Users are informed of all potential information sources

9. 2.4 Relations Global Registry Euro SDMX Registry UN SDMX Browser
ECB SDMX Registry
Etc.
National SDMX Registry 1
National SDMX Registry 2
National SDMX Registry 3

10. 2.5 In which registry?

11. 3. SDMX Global Registry
Launched in 2011 by the SDMX Sponsoring institutions
To make SDMX artefacts public and centrally available, which are used and maintained by national and international organisations
Designed to be developed into a global portal for SDMX structural data and metadata, providing:
how statistical information is structured
where the related data can be accessed

12. 3.1 Availability
Any national or international agency that wants to use SDMX standards for data management or for exchanging data with other organisations using SDMX standard
Accessible to any end-user of statistical data

13. 3.2 How to use it?
All information published in the SGR is publicly available and freely viewable and downloadable.
No need to log in to view or to download information - SDMX Global Registry Content.
Users can view the full content of the Global Registry , see how the various objects are interlinked and use the SDMX Global Registry Technical viewer.
Help is available in the Global Registry Wiki

14. Home page

15. Navigate the content of the registry
Registry Content
Navigate the content of the registry
Search for an item that contains the text entered
Technical view of SDMX objects

16. 4. SDMX Eurostat Registry
Eurostat's implementation of the SDMX Registry specifications as published by the SDMX initiative sdmx.org.
Based on SDMX 2.1 (as published on April 2011) Also capable of importing and exporting SDMX 2.0 artefacts.
Allows browsing, searching, editing and subscribing to artefacts
Advanced access control mechanism for distributed maintenance of artefacts controlling also their visibility.

17. 4.1 Availability Any Domain manager in Eurostat
Any end-user of statistical data can access the PUBLIC data structures posted on the SDMX Eurostat Registry (SER)
Access to artefacts can be controlled per domain to limit the visibility of those which are private or under construction, thus not to be disclosed

18. 4.2 How to use it?
Some information published in the SER is publicly available and freely viewable and downloadable
Some information published in the SER is private
No need to log in to view or to download information - SDMX Eurostat Registry Content
Users can view the public content of the Registry, see how the various objects are interlinked
Contextual help is available online

19. Access to the content of the Registry: text search
Home page
Access to the content of the Registry: text search
Access to the content of the Registry by type
Most recent items

20. Actions

21. 2. Detailed view
3. Download
1. Quick view

22. 2. Detailed view
3. Download
1. Quick view

23. 2. Detailed view
3. Download
1. Quick view

24. 2. Detailed view
3. Download
1. Quick view

25. 4.3 Architecture Web Service Database
All requests to the database are handled by the WS
Request-Response Interface pattern
Used by the GUI using SOAP
Used by other applications, e.g.
SODI-WS
SDMX Converter
Database
Schema according to SDMX – IM
The Graphical User Interface (GUI) is actually a web interface that allows users (with a valid ECAS login) access the contents of Eurostat’s SDMX Registry. The GUI accesses the Repository via the Registry Web Service using the SDMX Registry Interface messages. Thus, it is capable of accessing any SDMX compliant Registry Web Service as long as it implements the SDMX Registry Interface messages.
The Web Service (WS) is the core of Eurostat’s SDMX Registry. All requests to the database are handled by the WS (even those from the GUI). This means that the database is only exposed via the WS to any potential client (human or machine). It follows the Request-Response Interface pattern described by the SDMX standard, since it implements a large part of the SDMX Registry Interface specification. Other potential users of the WS are the SDMX Converter, the DSW.
The Repository (DB) is the actual data-storage of Eurostat’s SDMX Registry. To be more precise, it is a metadata storage, since it stores structural ad provisioning metadata. The database schema has been built according to the SDMX-IM.

26. 4.4 Entity-Relationship model
SDMX Information Model -> ER model
ER does not support all relationships of class diagram
Inheritance
Many to many relationship
No one-to-one mapping of classes and tables
Query performance (e.g. number of joins)
Query simplicity for application development
Maintainability and table size
Enforcing data constraints

27. 4.5 Interface Request/Response pattern J2EE application
Model-View-Controller (MVC approach)
Struts framework
The SDMX Registry Graphical User Interface (GUI) is a J2EE application based on a Model-View-Controller (MVC) approach.
It has been built using Struts framework ( which is a free open-source framework for creating Java web applications.
Apache Struts incorporates an MVC architecture, thus separating software applications in manageable parts. Specifically in an MVC architecture, the Model represents the business or database code, the View represents the page design code, and the Controller represents the navigational code. The Struts framework is designed to help developers create web applications that utilize a MVC architecture.
The key components of the Struts framework are:
A "request" handler provided by the application developer that is mapped to a standard URI.
A "response" handler that transfers control to another resource which completes the response.
A tag library that helps developers create interactive form-based applications with server pages.
Moreover, the GUI application incorporates:
JSP custom tag libraries and associated support in the controller Servlet
Utility classes that support XML parsing, automatic population of Java Beans properties based on the Java reflection APIs, and internationalisation of prompts and messages
SubmitStructureRequest - SubmitStructureResponse
Insert, update, delete DSDs, Code Lists, Concept Schemes, Dataflows, etc.
QueryStructureRequest - QueryStructureResponse
Query DSDs, Code Lists, Concept Schemes
SubmitProvisioningRequest - SubmitProvisioningResponse
Insert, update, delete provision agreements
QueryProvisioningRequest - QueryProvisioningResponse
SubmitSubscriptionRequest - SubmitSubscriptionResponse
SubmitRegistrationRequest - SubmitRegistrationResponse
QueryRegistrationRequest - QueryRegistrationResponse
NotifyRegistryEvent

28. 5. SDMX Registry
Hands-on exercise
How to download DSDs

29. SDMX Registry