3. An overview of the SDMX implementation process Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, March 2016

Who is involved in SDMX exchanges? What are our roles? Initiator? Facilitator? Implementer?

What is a process? It is a series of actions or steps taken to achieve a particular end. Why is understanding a process important? Without explaining the process, stakeholders risk becoming alienated and resistant to change. Why is understanding the SDMX implementation process important? SDMX is quite complicated. But, understanding the process makes it more accessible, more collective and more likely to succeed.

THEORY Any implementation process is about a methodical approach to (among other things): Planning; Organising; Managing; and Optimising resources. (work and people) (workflows, meeting, paperwork) (even in public administrations!) And all within the constraints of: Scope; Time; Budget; and Quality. (final & intermediate deadlines) (with competing demands) (fitting within a C.O.P.)

PRACTICE The SDMX implementation process can be broken down into key phases. This is the ESTAT process: Phase 1: Preparation Phase 2: Compliance Phase 3: Implementation Phase 4: Production

'By failing to prepare, you are preparing to fail.' Benjamin Franklin

THEORY The 'Preparation Phase' is arguably the most critical phase. It determines the objectives, scope, expected benefits and outputs of the project. It requires a thorough discussion and assessment of needs, of stakeholders (and their roles) and of the business process. It bears in mind policies, mandates, technological changes and different conceptual languages.

PRACTICE Bearing in mind the SDMX mandate for the ESS; (Directors' Meeting April 2009): As a first step before making SDMX compulsory for all domains in Eurostat, the use of SDMX would be made compulsory for all new or considerably changed data sets and reference metadata sets. Being conscious of the efficiency gains that could be harnessed from technological advances; and Being mindful of the different technical languages spoken by IT units and production units…

…good communication is essential. 'Communication and trust are the two main ingredients for a successful relationship.' Unknown

PRACTICE There is no agreed list of preparatory questions for kick-off discussions, but these are the kind of topics to be covered: A needs assessment: Why do you want SDMX? What should it solve? What constraints? What timetable? What risks? An overview of the current production process: What production systems are used? What file formats for transmission? What code lists are used?

PRACTICE An overview of the data / metadata as well as the basic source of information for the DSD: What is the frequency of the data flows? What will be used to build the DSDs? Existing questionnaires, legislation, methodologies …? An overview of stakeholders, both external and internal: Who is involved? What agreements are in place? Who are the data providers?

THEORY At the end of the preparatory phase all stakeholders should be clear about: The goals of the project; The timeline for implementation; A draft of the project plan; and Their roles and responsibilities.

THEORY This is the phase of the project during which: The groundwork is carried out; The system is designed; and The workflow is sequenced. It is the phase during which a high investment of time is likely to be required, in part because loops back into the preparation tasks are likely.

PRACTICE The following steps are taken in the compliance phase: Analysis of the current data exchange; Decision on the appropriate structure for the exchanged data; Application of SDMX statistical guidelines; cross-domain concepts, re-use of existing code lists (http://sdmx.org/?page_id=11 );

PRACTICE The following steps are taken in the compliance phase: Design, then agreement, then creation of DSDs and MSDs; Development of draft maintenance agreements;

'Coming together is a beginning. Keeping together is progress 'Coming together is a beginning. Keeping together is progress. Working together is success.' Henry Ford

THEORY This is the phase of the project during which: Plans are implemented; Communication with stakeholders is emphasised; Problems revealed / solutions sought / corrective action taken;

PRACTICE The following steps are taken in the implementation phase: SDMX artefacts are uploaded to the registry – particularly DSDs; Appropriate IT infrastructure for SDMX compliant transmission established; Pilot projects conducted: - DSD structure reviewed, and - IT infrastructure tested; …

…; Feedback from testing incorporated; Finalise DSDs and maintenance agreement; Roll-out schedule agreed; Support provided through formal training events, helpdesk services, capacity building etc.

The following steps are taken in the production phase: SDMX compliant data exchange used in production; Maintenance of the SDMX artefacts at regular intervals;

Thank you for your attention. I am happy to take questions and receive feedback. For further information please contact: ESTAT-SUPPORT-SDMX@ec.europa.eu