The Data Quality Assessment Framework and IMF’s Dissemination Standards Bulletin Board Tools and Practices for Collecting and Disseminating Metadata CCSA CONFERENCE ON DATA QUALITY FOR INTERNATIONAL ORGANIZATIONS APRIL 2006 Prepared by Kim Zieschang, IMF Statistics Department (STA). The views expressed in this presentation are those of the author and should not be attributed to the International Monetary Fund, its Executive Board, or its management..

The IMF Data Standards Program Special Data Dissemination Standard (SDDS) General Data Dissemination System (GDDS) Data Quality Program (DQP)

Background – SDDS The Fund initiated the SDDS in 1996 in the wake of the Mexican financial crisis of the mid-1990s Response to a need to inform financial markets better, not only about country macroeconomic data themselves, but also information about the data (metadata) Focus on dissemination practices: Coverage, Periodicity, Timeliness Reserves template added in 1998 (Second Review) following the Asian financial crises of External debt added in 2000 (Fourth Review)

Background – GDDS The Fund initiated the GDDS in 1997 as a developmental framework to improve member countries’ statistical capacities  Statistical capacity is measured, among other things, by the ability to meet SDDS dissemination (coverage, periodicity, timeliness) requirements  More extensive socio-demographic metadata than SDDS

Background – The DQP and the Data Quality Assessment Framework (DQAF) The DQP was introduced in the Fourth Review of the Data Standards Initiatives (2001) and introduced the DQAF as the framework for the Data Module of the Reports on Observance of Standards and Codes (Data ROSCs)Data ROSCs The Fifth Review of the Data Standards Initiatives (2003, Supplement 2) introduced an update (July 2003, the current version) to the first version of the DQAFJuly 2003, the current version The Sixth Review (2005, Supplement 1) sets the DQAF of the DQP as the underlying metadata model of the SDDS and GDDS, as well as the Data ROSCs, noting its connection with the emerging Statistical Data and Metadata eXchange (SDMX) standard.SDMX

Metadata model of the Data Standards: DQAF 0. Prerequisites of quality  0.1 Legal and institutional environment  0.2 Resources  0.3 Relevance  0.4 Other quality management 1. Assurances of integrity  1.1 Professionalism  1.2 Transparency The terms and conditions under which statistics are collected, processed, and disseminated are available to the public Internal governmental access to statistics prior to their release is publicly identified Products of statistical agencies/units are clearly identified as such Advance notice is given of major changes in methodology, source data, and statistical techniques.  1.3 Ethical standards 2. Methodological soundness  2.1 Concepts and definitions  2.2 Scope  2.3 Classification/sectorization  2.4 Basis for recording 3. Accuracy and reliability  3.1 Source data  3.2 Assessment of source data  3.3 Statistical techniques  3.4 Assessment and validation of intermediate data and statistical outputs  3.5 Revision studies 4. Serviceability  4.1 Periodicity and timeliness Periodicity follows dissemination standards Timeliness follows dissemination standards  4.2 Consistency  4.3 Revision policy and practice 5. Accessibility  5.1 Data accessibility Statistics are presented in a way that facilitates proper interpretation and meaningful comparisons Dissemination media and format are adequate Statistics are released on a preannounced schedule Statistics are made available to all users at the same time Statistics not routinely disseminated are made available upon request  5.2 Metadata accessibility  5.3 Assistance to users

Metadata model of the Data Standards: DQAF The Generic DQAF contains 3-digit detail not shown in the previous exhibit for 2-digit elements other than 1.2, 4.1, and 5.1 This detail is suppressed only to show how the DQAF maps to the SDDS/GDDS presentation structure The internal organization of SDDS/GDDS metadata will eventually contain the full 3-digit DQAF granularity for the four macroeconomic data categories Conversion of existing metadata is proceeding for a first wave of about 10 of the 61 SDDS countries

Dissemination Standards Bulletin Board (DSBB) dsbb.imf.org Displays a dissemination practices view of metadata  This structure dates from the beginning of the SDDS (1996) and GDDS (1997)  Four dimensions and a summary methodology (SDDS)/comprehensive framework (GDDS)

Metadata model of the Data Standards: DSBB dissemination practices presentation Data  Coverage  Periodicity  Timeliness Access  Advance release calendar  Simultaneous release Integrity  Terms and conditions  Advance access  Ministerial commentary  Revisions and advance notice of methodological changes Quality  Documentation Summary methodology  Analytical Framework, Concepts, Definitions, and Classifications  Scope of the data  Accounting conventions  Nature of the basic data sources  Compilation practices  Other aspects  Detail/reconcilation/frameworks

Metadata model of the Data Standards: DQAF mapping to the presentation model Data  Coverage (2.1, 2.2, 2.3, 2.4, 3.1, 3.3, 3.4, 5.1.1)  Periodicity (4.1.1)  Timeliness (4.1.2) Access  Advance release calendar (5.1.4)  Simultaneous release (5.1.3) Integrity  Terms and conditions (0.1, 1.1, 1.2.1, 1.3)  Advance access (1.2.2)  Ministerial commentary (1.2.3)  Revisions and advance notice of methodological changes (1.2.4, 3.5, 4.3) Quality  Documentation (5.2) Summary methodology  Analytical Framework, Concepts, Definitions, and Classifications (2.1)  Scope of the data (2.2)  Accounting conventions (2.4)  Nature of the basic data sources (3.1,3.2)  Compilation practices (3.3, 3.4, 3.5)  Other aspects (4.2, 4.3)  Detail/reconcilation/frameworks (4.2, 5.1.5)

Metadata model of the Data Standards: DQAF mapping to the presentation model The DQAF provides greater “granularity” to the SDDS and GDDS metadata presentation framework 2-digit DQAF topics map cleanly into the Data Standards presentation model, except for three 2-digit categories whose 3-digit items that map to multiple DSBB presentation items:  DQAF 1.2 Transparency goes to the SDDS/GDDS Integrity items: Integrity: Terms and conditions (..., 1.2.1,...) Integrity: Advance access (1.2.2) Integrity: Ministerial commentary (1.2.3) Integrity: Revisions and advance notice of methodological changes (1.2.4,...)  DQAF 4.1 Periodicity and timeliness goes to the SDDS/GDDS Data items: Data: Periodicity (4.1.1) Data: Timeliness (4.1.2)  DQAF 5.1 Data accessibility goes to the following SDDS/GDDS Data, Access, and Quality items Data: Coverage (..., 5.1.1) Access: Advance release calendar (5.1.4) Access: Simultaneous release (5.1.3) Quality: Detail/reconcilation/frameworks (..., 5.1.5)

Metadata model of the Data Standards: DQAF mapping to the presentation model The DQAF allows the DSBB to easily use ROSC Detailed Assessment metadata as a basis for updating SDDS and GDDS metadata and, conversely, to use the SDDS and GDDS as a basis for ROSC mission preparation.Detailed Assessment metadata The DQAF prospectively will allow a similar two-way transmission of information between technical assistance documentation and SDDS and GDDS metadata.

Metadata model of the Data Standards: Economic subject area Real sector  National accounts  Production  Labor market Employment Unemployment Wages  Price statistics CPI PPI Fiscal sector  General government operations  Central government operations  Central government debt Financial sector  Analytical accounts of depository corporations  Analytical accounts of the central bank  Interest rates  Share price index External sector  Balance of payments  Reserves template  Merchandise trade  International investment position  External debt  Exchange rates Socio-demographic sector  Population  Health  Education  Poverty

Sixth Review (November 2005) initiatives – SDDS and GDDS metadata Implement 3-digit DQAF “granularization” of SDDS and GDDS metadata Improve coverage of oil and gas activities and products in the metadata for existing data categories

DQAF and the European Statistics Code of Practice IMF DQAF  51 headings Eurostat Statistics Code of Practice  77 headings Detailed cross-domain concepts from merging DQAF and Code of Practice  106 headings

DQAF, European Code of Practice, and the SDMX SDMX version 2.0 guidelines for broad cross-domain concepts now in discussion  26 headings Merged IMF DQAF and Eurostat Code of Practice  106 headings  Very general—could serve as a basis for possible SDMX version 2.0 guidelines for detailed cross-domain concepts that allow broad interoperability across virtually all data quality frameworks now available.

Proposed Broad SDMX Cross- Domain Concepts 01Accessibility of documentation 02Accounting conventions/basis 03Accuracy 04Classification systems 05Comparability/Coherence 06Confidentiality 07Contact 08Data presentation 09Date of update 10Dissemination formats 11Frequency and Periodicity 12Institutional framework 13Professionalism and ethical standards 14Quality management (including resource management) 15Release calendar 16Relevance 17Revision policy and practice 18Scope / coverage 19Simultaneous release 20Source data 21Statistical concept 22Statistical processing 23Supplementary data 24Timeliness and punctuality 25Transparency 26Validation

Illustrative Detail of SDMX “Accuracy” Notional code SDMX Broad cross- Domain concept DQAF Indicator European Statistics Code of Practice Indicator Possible SDMX detailed cross-domain concept Notional code 03Accuracy3.2.1 Source data—including censuses, sample surveys, end administrative records—are routinely assessed, e.g., For coverage, sample error, response error, and nonsampling error; the results of the assessments are monitored and made available to guide statistical processes Source data, intermediate results and statistical outputs are assessed and validated [ Use of accuracy assessments—source data] Use of accuracy assessments— source data Accuracy3.2.1 Source data—including censuses, sample surveys, and administrative records—are routinely assessed, e.g., for coverage, sample error, response error, and nonsampling error; the results of the assessments are monitored and made available to guide statistical processes Sampling errors and non – sampling errors are measured and systematically documented according to the framework of the ESS quality components Measurement and documentation of sampling and non–sampling errors Accuracy3.4.2 Statistical discrepancies in intermediate data are assessed and investigated Source data, intermediate results and statistical outputs are assessed and validated [ Use of accuracy assessments—intermediate data] Use of accuracy assessments— intermediate data—validation Accuracy3.4.3 Statistical discrepancies and other potential indicators or problems in statistical outputs are investigated Source data, intermediate results and statistical outputs are assessed and validated [ Use of accuracy assessments—statistical outputs] Use of accuracy assessments— statistical outputs Accuracy3.5.1 Studies and analyses of revisions are carried out routinely and used internally to inform statistical processes (see also 4.3.3) Revisions follow standard, well-established and transparent procedures Documentation of revisions03.05

Concluding remarks Interoperability of metadata frameworks requires maintaining sufficient “granularity” in the topical structure of the framework Merge of IMF DQAF and Eurostat Statistics Code of Practice may take us a long way to determining what metadata topical “granules” we need Although SDMX technical standards permit successful electronic data transmission of datasets and sub-cubes of datasets, interoperability requires international agreement at the granular level Both the SDMX technical standards and the content oriented guidelines (granular cross-domain concepts) are needed to realize the potential of SDMX to  Greatly reduce the cost of metadata capture by the IMF and other international organizations and international bodies  Greatly reduce the respondent burden on the national reporters of metadata