Using the Checklist for SDMX Data Providers I’m presenting this but the Global DSDs have required a great collaboration at all levels in many different organisations – involving senior management, statisticians, technicians, project managers, methodologists, etc.

The Checklist https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Introduction The aim of the Checklist for SDMX Data Providers is to help statistical agencies implement the provision of SDMX data based on existing structures A prerequisite before starting SDMX implementation is to have SDMX structural metadata artefacts defined and stored in a shared location (such as the Global SDMX Registry) The design project may not have been done by you The checklist is complementary to the Checklist for SDMX Design Projects, which describes how such artefacts are created Based on Generic Statistical Business Process Model (GSBPM) This is a work-in-progress

Examples of Use Preparing a pilot phase for a Global DSD Dissemination of data using SDMX for national needs or for international agreements (e.g. SDDS+ adherence in cooperation with IMF) Use as a point of reference for project managers and other project participants to speed up implementations based on best practises

How to use it Every activity can be clicked to view its details Every activity has a required output to guide the user how to advance the project Not all activities are compulsory: it depends on the statistical agencies starting point, such as on the level of technology available, and the statistical and SDMX knowledge The order of activities do not require a sequential/waterfall implementation, and there will naturally be some feedback loops; for example, after the Test phase it could be necessary to return to the Design or Planning phases. The checklist should be usable with most project management methodologies such as Waterfall, KANBAN, Scrum, etc.

Specify Needs https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Participation to data exchange implementation groups The data provider representatives should participate in (or be informed by) groups (e.g. working parties) for the implementation of the SDMX exchange. These meetings will usually involve data providers from different SDMX implementation maturity levels and several disciplines e.g. subject-matter areas, IT, methodology, senior management. Output of this phase: Input to the SDMX implementation documents (such as the project plan and maintenance agreement) Information for the data provider in order to implement the SDMX exchange, such as allowing the drafting of internal implementation project plans (the next phase)

Analyse project internally, make "go/no go" decision The main goal of this phase is to analyse those requirements and decide whether to participate in the project or not. Involve all stakeholders to compile the following information: the impact on the current statistical business process, can use the GSBPM  support from management to secure a mandate for the project the IT actions to be performed for the success of the project; the resources and budget required for the project; identify capacity building that is required and the cost. Previous project evaluations/lessons learned  may help Outputs of this phase: Impact analysis of project implementation; Cost/benefit report; Resource and capacity building planning; A go/no go decision for the project, or agreement to defer a decision to another time

Plan & Organise https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Appoint project manager and team; define project plan Appoint a project manager Define internal project plan Stakeholders to approve it Include a roadmap with milestones List consultation groups and their roles, e.g. Inform, Approve Create an Issue Log to track issues An example template is available in the checklist Outputs of this phase: the project manager appointment; the project plan; the project team member appointments

Organise training courses for project members and stakeholders Training for the needs of the project Could be SDMX, subject matter, IT-related, tool awareness, etc. Training for statisticians could focus on understanding the DSDs, Code Lists, and mappings to internal databases Training for IT people could be on tool implementation, SDMX technical notes Training should be a part of the roadmap External courses by organisations should be considered, but There is already a lot of online material already available on the SDMX.org Learning tab Outputs of this phase: a training plan for the project team and stakeholders.

Design https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Design data reporting architecture The goal is to design the data reporting architecture Two provisioning methods: Pull and/or Push Push: Create files and send Pull: Don’t send. Instead, make the data available through a web service Pull is recommended to increase timeliness, flexibility, avoid creation of data packages, ease of updating Need to take security, confidentiality, data volume into account SDMX Registries should be used to store the artefacts Outputs of this phase:  the data reporting architecture design

Design IT infrastructure and create test plan The goal is to design the IT system architecture Many free SDMX tools already exist for this such as the SDMX Reference Infrastructure, Connectors, Convertors and others Freemium tools also exist such as SDMX Fusion products More details on the SDMX.org Tools tab If new tools are required, use SDMX Source to build them Contact the SDMX Technical Working Group for guidance with IT infrastructure SDMX Registries should be used to store the artefacts Outputs of this phase:  an IT infrastructure design for the project for implementation

Data production analysis and design The goal is to design a new data production process Compare the current domain implementation to the one required by this project For example, it may require a new data collection or aggregation to satisfy the reporting requirements There may be mappings required from existing structures to the DSDs Outputs of this phase: A new data production design

Build https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Implement IT Infrastructure The goal is to implement the design from the previous Design IT Infrastructure phase Mobilise the IT resources outlined in the project plan If new tools have been created, it is important to consider offering them to the SDMX community. Contact the SDMX Technical Working Group for this Outputs of this phase: The IT infrastructure, ready for testing

Map internal structure metadata to DSDs If it has been decided to map internal data to the reporting framework, the goal is to create mappings between the DSDs and the local databases The mappings are for: Translate incoming SDMX queries to query the correct internal data Output the internal data using the coding of the DSDs Existing Tools are available to create and maintain mappings Outputs of this phase: The mappings themselves The transformation system

Testing The goal is to test the tools in the BUILD phase so that they satisfy the outputs from the DESIGN phase Internal testing followed by a pilot phase Pilot phase often a part of a data exchange project such as National Accounts Use of the Issue log is crucial to track problems and resolutions Testing continues until issue log is empty or issues are prioritised for a later project Outputs of this phase: The updated Issue log The updated test plan

Disseminate https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Release production data The goal is to release the production data to the terms agreed in the project plan Important to monitor the correct functioning of the system Mobilise the resources required to maintain the systems and data flows Outputs of this phase: the functioning data exchange; the project end move to a maintenance/support phase

End of project A milestone rather than a phase Make any announcements as specified in the project plan Useful to inform SDMX community of the implementation to encourage reuse: contact@sdmx.org

Evaluate https://statswiki.unece.org/display/SDMXPM/Checklist+for+SDMX+Data+Providers

Evaluate project The goal is to make a “lessons learned” document for future projects to learn from successes and challenges Important to monitor the correct functioning of the system Mobilise the resources required to maintain the systems and data flows Outputs of this phase: a project evaluation/"lessons learned" document to be used as starting point for a new project

Thank you Google for “Checklist for SDMX Data Providers” David Barraclough david.barraclough@oecd.org SDMX Official Website: https://sdmx.org/ Technical Working Group: twg@sdmx.org Statistical Working Group: swg@sdmx.org