1. 3. An overview of the SDMX implementation process
Edward Cook
Eurostat
Unit B5: “Central data and metadata services”
SDMX Basics course, 1-2 March 2017

2. Who is involved in SDMX exchanges?
What are our roles?
Initiator?
Facilitator?
Implementer?

3. What is a process?
It is a series of actions or steps taken to achieve a particular end.
Why is understanding a process important?
Without explaining the process, stakeholders risk becoming alienated and resistant to change.
Why is understanding the SDMX implementation process important?
SDMX is quite complicated. But, understanding the process makes it more accessible, more collective and more likely to succeed.

4. THEORY
Any implementation process is about a methodical approach to (among other things):
Planning;
Organising;
Managing; and
Optimising resources.
(work and people)
(workflows, meeting, paperwork)
(even in public administrations!)
And all within the constraints of:
Scope;
Time;
Budget; and
Quality.
(final & intermediate deadlines)
(with competing demands)
(fitting within a C.O.P.)

5. PRACTICE
The SDMX implementation process can be broken down into key phases. This is the ESTAT process:
Phase 1: Preparation
Phase 2: Compliance
Phase 3: Implementation
Phase 4: Production

6. 'By failing to prepare, you are preparing to fail.'
Benjamin Franklin

7. THEORY
The 'Preparation Phase' is arguably the most critical phase.
It determines the objectives, scope, expected benefits and outputs of the project.
It requires a thorough discussion and assessment of needs, of stakeholders (and their roles) and of the business process.
It bears in mind policies, mandates, technological changes and different conceptual languages.

8. PRACTICE
Bearing in mind the SDMX mandate for the ESS; (Directors' Meeting April 2009): As a first step before making SDMX compulsory for all domains in Eurostat, the use of SDMX would be made compulsory for all new or considerably changed data sets and reference metadata sets.
Being conscious of the efficiency gains that could be harnessed from technological advances; and
Being mindful of the different technical languages spoken by IT units and production units…

9. …good communication is essential.
'Communication and trust are the two main ingredients for a successful relationship.'
Unknown

10. PRACTICE
There is no agreed list of preparatory questions for kick-off discussions, but these are the kind of topics to be covered:
A needs assessment:
Why do you want SDMX? What should it solve? What constraints? What timetable? What risks?
An overview of the current production process:
What production systems are used? What file formats for transmission? What code lists are used?

11. PRACTICE
An overview of the data / metadata as well as the basic source of information for the DSD:
What is the frequency of the data flows? What will be used to build the DSDs? Existing questionnaires, legislation, methodologies …?
An overview of stakeholders, both external and internal:
Who is involved? What agreements are in place? Who are the data providers?

12. THEORY
At the end of the preparatory phase all stakeholders should be clear about:
The goals of the project;
The timeline for implementation;
A draft of the project plan; and
Their roles and responsibilities.

13. THEORY
This is the phase of the project during which:
The groundwork is carried out;
The system is designed; and
The workflow is sequenced.
It is the phase during which a high investment of time is likely to be required, in part because loops back into the preparation tasks are likely.

14. PRACTICE
The following steps are taken in the compliance phase:
Analysis of the current data exchange;
Decision on the appropriate structure for the exchanged data;
Application of SDMX statistical guidelines; cross-domain concepts, re-use of existing code lists ( );

15. PRACTICE
The following steps are taken in the compliance phase:
Design, then agreement, then creation of DSDs and MSDs;
Development of draft maintenance agreements;

16. 'Coming together is a beginning. Keeping together is progress
'Coming together is a beginning. Keeping together is progress. Working together is success.'
Henry Ford

17. THEORY
This is the phase of the project during which:
Plans are implemented;
Communication with stakeholders is emphasised;
Problems revealed / solutions sought / corrective action taken;

18. PRACTICE
The following steps are taken in the implementation phase:
SDMX artefacts are uploaded to the registry –
particularly DSDs;
Appropriate IT infrastructure for SDMX compliant transmission established;
Pilot projects conducted: - DSD structure reviewed, and
- IT infrastructure tested; …

19. …;
Feedback from testing incorporated;
Finalise DSDs and maintenance agreement;
Roll-out schedule agreed;
Support provided through formal training events, helpdesk services, capacity building etc.

20. The following steps are taken in the production phase:
SDMX compliant data exchange used in production;
Maintenance of the SDMX artefacts at regular intervals;

22. WHAT IS GOVERNANCE ?
A common and shared effort by all SDMX stakeholders to define clear organisational and operational rules and arrangements to which everyone should abide.
More simply,
Guidance: recommendations, best practices;
Organisation: rules, administration, control, law, authority etc.
If your experiment needs statistics, you ought to have done a better experiment (E. RUTHERFORD)

23. Why is governance important ?
Establish a quality threshold
Save time by avoiding uncertainty on what to do
Have an agreed, central, unambiguous reference on what is allowed and what are the best practices
Fit the needs of the SDMX community
Support standardisation
Definition of statistics : The science of producing unreliable facts from reliable figures (E. ESAR)

24. GOVERNANCE BODIES
SDMX governance bodies
Project
governance bodies

25. MAINTENANCE AGREEMENTS FOR GLOBAL DSDs
General arrangements (Example from National Accounts DSD)
Three types of changes are foreseen:
- structural changes,
- annual maintenance changes and
- fast-track maintenance changes
Agreed timetable established for change requests