Implementing SDMX exchange of national development indicators SDMX Expert Group Meeting, Paris, 13 – 14 September 2012

UNSD-DFID Project 1.Introduction 2.Progress on country SDMX 3.Design 4.Key issues 5.Questions & discussion

Introduction: Objectives Improve the coherence and clarity of devt indicators - Improve coordination in the NSS - Collate development data in 1 place/database - Explain differences between intl. & nat. data Improve accessibility and visibility - Make access to national data easier - Draw attention to wider set of indicators - Reduce data request burden Enhance knowledge - Strengthen IT support - Use of latest IT software & practices - Training & study tours

Introduction: Background Implementation in 11 countries across Africa and Asia A pilot ran during , full phase started in late 2010 All countries initiated now, 5 countries started implementing and 5 about to start. Project runs until Jul 2014

Progress on country SDMX Cambodia : SDMX registry installed & mappings of 46 time series complete/ available (with further 175 time series identified for mapping) Burundi : SDMX registry installed Rwanda : Expanded SDMX registry Lao & Uganda : Expanded SDMX registry & mapping tool May 2011 Feb 2012 Jun 2012 Planned to Dec 2012

Design: Overview Most project countries are unfamiliar with SDMX –Many do not have experience with underlying technologies such as XML, XSLT. Conflicting pressures on project solution: –Must be simple to use, to facilitate its adoption –Must be powerful, to fulfill requirements –Must give the user a chance to familiarize themselves with SDMX.

Design: SDMX solution Some of the ways in which project solution simplifies the implementation of SDMX: –Automated mapping between DevInfo database structures and CountryData DSD –Automated data export from DevInfo database into CountryData SDMX –Automated publication of data and registration of datasets at a DevInfo registry.

Design: System Line MinistriesNational Statistical OfficeUnited Nations Line Ministry Database National Indicator Registry National Repository DB DevInfo Upload XLS Scripts Register files Post notification Publish SDMX-ML Download Mapping tool

Design: SDMX add-ons Project works with DevInfo to expand functionality offered by DevInfo registry and offer such new features as: –User-defined dataflows, metadataflows, and provision agreements –Constraints –Metadata discovery and other Web service functionality.

Design: MDG DSD Supports exchange of MDG Indicator data between international agencies (UN, UNICEF, UNESCO, …) Developed by SDMX Task Team of Interagency and Expert Group on Development Indicators (IAEG) Implemented in SDMX 2.0 CountryData DSD is based on the MDG DSD.

Design: CountryData DSD Codelists amended to support non-MDG development indicators published by countries To avoid confusion and clearly separate national and international data, a new DSD with its own ID was defined for the project, which reuses MDG artefacts as much as possible. Codelists are currently maintained by UNSD –New indicators proposed by project countries are introduced with a lag –Centralised codelist maintenance unsustainable in the long run as more countries join the project.

Design: CountryData MSD Based on the MDG global database, agreed by the IAEG (i.e. definition, method of computation etc.). Attachment at the indicator level Key to understanding differences between international estimates and country estimates of the same indicator Also benefits in consistency, transparency, reporting burden, …

Key issues: Codelist maintenance Codelist maintenance must eventually be taken over by the countries –Compatibility becomes an issue. Design under consideration envisages using constrained dataflows to maintain compatibility.

Key issues: Matching Trying to match a country indicator with an available indicator code At what point is a new code required i.e. For the indicator Using improved drinking water / sanitation does measurement on a household basis rather than population make it different indicators? Or using Gross instead of Net to calculate Gender Parity Indices make them a separate indicators?

Key issues: Mapping Process of mapping the indicator time series to other dimensions (i.e. Sex, Age group, Location etc.) Complicated by the fact that MDG and other development indicators vary greatly in their dimensionality Further guidance developed to ensure codelist items are applied properly

Key issues: Metadata exchange Metadata is usually not comprehensive enough to understand comparability. The importance of good metadata (and even what it is?) is little understood within some National Statistical Systems. Few examples of metadata exchange using SMDX Format issues: HTML vs. Plain Text, Presentation vs. security.

Key questions Project Sustainability Sharing national development indicators in SDMX Commitment to maintaining good metadata Governance Managing structural metadata among multiple national and international organizations Expansion –System used for greater sharing between different parties.

Thank you for your attention Sources of further reference: Cambodia SDMX Registry Rwanda SDMX Registry