Eurostat 1 3.An overview of the SDMX implementation process Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015

Eurostat 2 -Initiator? -Facilitator? -Implementer? Who is involved in SDMX exchanges? What are our roles?

Eurostat 3 It is a series of actions or steps taken to achieve a particular end. What is a process? Why is understanding a process important? Without explaining the process, stakeholders risk becoming alienated and resistant to change. Why is understanding the SDMX implementation process important? SDMX is quite complicated. But, understanding the process makes it more accessible, more collective and more likely to succeed.

Eurostat 4 Any implementation process is about a methodical approach to (among other things): And all within the constraints of: -Planning; -Organising; -Managing; and -Optimising resources. (work and people) (workflows, meeting, paperwork) (even in public administrations!) -Scope; -Time; -Budget; and -Quality. (final & intermediate deadlines) (with competing demands) (fitting within a C.O.P.) THEORY

Eurostat 5 -Having an SDMX mandate; (Directors' Meeting April 2009): As a first step before making SDMX compulsory for all domains in Eurostat, the use of SDMX would be made compulsory for all new or considerably changed data sets and reference metadata sets. Two background factors that help make implementation easier: -Speaking the same language; Being mindful of the different technical languages spoken by IT units and production units… … good communication is essential.

Eurostat 6 As Mrs Cook likes to remind me: 'Communication and trust are the two main ingredients for a successful relationship.'

Eurostat 7 The SDMX implementation process in ESTAT is broken down into key phases: -Phase 1: Preparation -Phase 4: Production -Phase 2: Compliance -Phase 3: Implementation PRACTICE

Eurostat 8 'By failing to prepare, you are preparing to fail.' Benjamin Franklin

Eurostat 9 PLAN AND ORGANISE SDMX training for team Project roadmap and work breakdown Collaboration platform and issue log SDMX project templates Define pilot phase details and participants Describe needs; Get approval Identify stakeholders Check existing content Project proposal Project governance Maintenance agreement; Ownership group structure; ToR SPECIFY NEEDS A r g u a b l y m o s t c r i t i c a l p h a s e. Communication & consultation planning

Eurostat 10 As well as the guideline checklist, here are some examples of the type of preparatory questions for kick-off discussions: -A needs assessment: -An overview of the current production process: -Why do you want SDMX? What should it solve? What constraints? What timetable? What risks? -What production systems are used? What file formats for transmission? What code lists are used?

Eurostat 11 -An overview of the data / metadata as well as the basic source of information for the DSD: -An overview of stakeholders, both external and internal: -What is the frequency of the data flows? What will be used to build the DSDs? Existing questionnaires, legislation, methodologies …? -Who is involved? What agreements are in place? Who are the data providers?

Eurostat 12 At the end of the preparatory phase all stakeholders should be clear about: -The goals of the project; -The timeline for implementation; -A draft of the project plan; and -Their roles and responsibilities.

Eurostat 13 This is the phase of the project during which: -The groundwork is carried out; -The system is designed; and -The workflow is sequenced.

Eurostat 14 A r g u a b l y m o s t t i m e c o n s u m i n g p h a s e. DESIGN Describe data flows and architectures Decide which artefacts to reuse Design “core” artefacts Design supporting artefacts Execute pilot phase 1: Content review SDMX Content-oriented Guidelines: Cross-domain Code Lists Cross-domain Concepts SDMX Glossary Registries Reuse of existing Code Lists Design principles on modelling a statistical domain Look at provision agreements, constraints, category schemes Analysis of current data exchange

Eurostat 15 The following steps are taken in the compliance phase: -Analysis of the current data exchange; -Decision on the appropriate structure for the exchanged data; -Application of SDMX statistical guidelines; cross-domain concepts, re-use of existing code lists ( ) ;

Eurostat 16 The following steps are taken in the compliance phase: -Design, then agreement, then creation of DSDs and MSDs; -Development of draft maintenance agreements; PRACTICE

Eurostat 17 'Coming together is a beginning. Keeping together is progress. Working together is success.' Henry Ford

Eurostat 18 This is the phase of the project during which: -Plans are implemented; -Communication with stakeholders is emphasised; -Problems revealed / solutions sought / corrective action taken; THEORY

Eurostat 19 The following steps are taken in the implementation phase: -SDMX artefacts are uploaded to the registry – particularly DSDs; -Appropriate IT infrastructure for SDMX compliant transmission established; -…-… PRACTICE -Pilot projects conducted: - DSD structure reviewed, and - IT infrastructure tested;

Eurostat 20 -…; -Feedback from testing incorporated; -Support provided through formal training events, helpdesk services, capacity building etc. -Finalise DSDs and maintenance agreement; -Roll-out schedule agreed;

Eurostat 21 The following steps are taken in the production phase: -SDMX compliant data exchange used in production; -Maintenance of the SDMX artefacts at regular intervals;

Eurostat 22

Eurostat 23

Eurostat 24 Thank you for your attention. I am happy to take questions and receive feedback. For further information please contact: