1. 1 Work session convened by the Friends of the Chair Group on Integrated Economic Statistics Bern, 6-8 June 2007 Session 3(c) DISSEMINATION STANDARDS (DATA AND METADATA), DATA EXCHANGE AND REVISION POLICY

2. 2 DISSEMINATION STANDARDS Main aim of presentation To present a simple framework of dissemination standards for data and metadata Draws extensively from OECD Data and Metadata Reporting and Presentation Handbook – published in late 2006 Available at: http://www.oecd.org/dataoecd/46/17/37671574.pdf

3. 3 DISSEMINATION STANDARDS Presentation also covers some elements of this Framework: Data and metadata exchange standards Importance of terminology Guidelines for the reporting and dissemination of metadata Data revision

4. 4 DISSEMINATION STANDARDS “Dissemination standards” in context of this presentation cover issues related to: The presentation of statistics and related metadata in various dissemination media used by national agencies and international organisations Mechanisms for the efficient exchange of statistics and metadata between agencies (Data exchange) – including to international organisations

5. 5 DISSEMINATION STANDARDS Existing international guidelines touch on dissemination issues but references are partial or incomplete The elements of statistics and metadata dissemination are interrelated Need for an integrated approach on dissemination fits into the theme of this Work Session – Integrated Economic Statistics OECD has been an advocate of the development of a more modular approach in the development of statistical guidelines at the international level

6. 6 DISSEMINATION STANDARDS Modular approach on dissemination standards could be applied to other statistical standards currently being developed, such as: Distributive trade statistics (IRDTS) Industrial production National accounts (SNA update) This means using either the same text on dissemination or links to dissemination standards

7. 7 DISSEMINATION STANDARDS There are three broad problems when comparing data over time and between agencies. Conceptual: Differences in variable definitions, units and classifications. Operational: Differences in data collection and processing practices. Different data dissemination practices – data may look different even if they aren’t – complicates comparisons

8. 8 DISSEMINATION STANDARDS The need for articulation of a comprehensive set of standards in one source is therefore driven by two broad imperatives. To: Improve data quality

9. 9 DISSEMINATION STANDARDS IMPROVE DATA QUALITY Data interpretability Is an important dimension of data quality imbedded in all existing frameworks (IMF, Eurostat, OECD, Canada, Sweden, etc)

10. 10 DISSEMINATION STANDARDS IMPROVE DATA QUALITY Interpretability facilitated through the provision of appropriate metadata. Statistical transparency embodied in UN Fundamental Principles of Official Statistics Responsibility of all statistical agencies to accompany their statistics with appropriate metadata; and Provide efficient facilities for the dissemination of their metadata

11. 11 DISSEMINATION STANDARDS The need for articulation of a comprehensive set of standards in one source is therefore driven by two broad imperatives. To: Improve data quality Minimise reporting burden in provision of data and metadata to international organisations

12. 12 DISSEMINATION STANDARDS MINIMISE REPORTING BURDEN A lot of activity to reduce reporting burden of member countries international organisations, through: Coordinated collection of data. Use of common questionnaires. Data sharing between international organisations More can be done to improve the efficiency of data exchange between organisations

13. 13 DISSEMINATION STANDARDS MINIMISE REPORTING BURDEN Data sharing has received a lot of attention in recent years Assisted through evolution of new technologies – in particular, the Internet. Requires further development of both technical and data content exchange standards – SDMX has a key role Agreement on key data presentation practices is also vital

14. 14 DISSEMINATION STANDARDS FRAMEWORK - Key processes - Efficient data exchange processes Reporting and presentation of data Common quality standards / frameworks Common platforms for validation, metadata, dissemination?

15. 15 DISSEMINATION STANDARDS FRAMEWORK - Key processes - Efficient data exchange processes Reporting and presentation of data Metadata reporting & exchange guidelines Data revision standards SDMXDissemination standards

16. 16 ELEMENTS OF DRAFT DISSEMINATION STANDARDS FRAMEWORK Simple framework presented in paper Intended only to outline main elements Could be further elaborated in future

17. 17 ELEMENTS OF DRAFT DISSEMINATION STANDARDS FRAMEWORK Different types (e.g. absolute figures; indices; growth rates) and forms (e.g. original; seasonally adjusted; trend-cycle) of data:. Different types (e.g. absolute figures; indices; growth rates) and forms (e.g. original; seasonally adjusted; trend-cycle) of data:. Presentation and dissemination of metadata Key data presentation practices

18. 18 ELEMENTS OF DRAFT DISSEMINATION STANDARDS FRAMEWORK Key data presentation practices for revised data for breaks in time series For information about sampling and non-sampling errors of base years in the presentation of indices For data and metadata citation for the reporting of administrative data for revised data for breaks in time series For information about sampling and non-sampling errors of base years in the presentation of indices For data and metadata citation for the reporting of administrative data Initial list in OECD Presentation Handbook – Obviously, there are other issues

19. 19 DATA EXCHANGE SDMX Consortium of BIS, ECB, Eurostat, IMF, OECD, UNSD, World Bank To develop standards for the more efficient exchange of statistical and metadata – from countries to I/Os – between I/Os.

20. 20 DATA EXCHANGE SDMX Developing two distinct but complementary sets of standards The technical standards - provide a data model for exchanging a dataset using a “Data Structure Definition” - Version 2.0 released in 2006 Content standards

21. 21 DATA EXCHANGE SDMX Content standards – requires Standardization of concepts - Common Metadata Items Standardization of terminology - Metadata Common Vocabulary Standardization of Data/Metadata Structure Definitions Responsibility of the Domain Groups; identified using the Statistical Domain Lists

22. 22 GUIDELINES FOR REPORTING AND DISSEMINATION OF METADATA Provision of adequate metadata still a problem despite efforts at national and international levels. Absence makes international comparisons difficult Numerous initiatives and forums have developed metadata standards over the years – ISO, DDI, Dublin Core + work of METIS, etc Still a need for further standards on where and how metadata should be presented – OECD Presentation Handbook

23. 23 GUIDELINES FOR REPORTING AND DISSEMINATION OF METADATA OECD Presentation Handbook Further emphasizes need for metadata Access to metadata (free of charge, linkage to data, structuring, searchable, etc) Adoption of a set of common metadata items – SDMX content standards Adoption of a common set of terminology for metadata preparation – SDMX content standards

24. 24 DATA REVISION Defined as any change in value of a statistic released to the public by an official national statistical agency

25. 25 DATA REVISION Data revision is a fact of statistical life Many users still have difficulty with the fact that data needs to be revised for any one or combination of reasons Therefore, all organisations should develop a sound revisions policy and practices that are transparent to users

26. 26 DATA REVISION OECD Handbook touches on data revision in detail Work derived from previous IMF material Both IMF and OECD believe that implementation of corporate revisions strategy is still an exception at both national and international levels Such a strategy touches on eight elements - to be outlined shortly

27. 27 DATA REVISION - Causes - Incorporation of more complete source data Incorporation of source data that more closely match concepts Replacement of first release estimates from judgmental or statistical techniques when data become available or as a result of benchmarking Incorporation of updated seasonal factors Updating of base period Changes in statistical methods Changes in concepts, definitions or classifications Correction of errors Source data Routine recalcu- lations Improve- ments in metho- dology

28. 28 DATA REVISION Eight elements of corporate revisions strategy embedded in recommendations outlined in OECD Presentation Handbook

29. 29 DATA REVISION Recommended practices Obtain user views before developing a revisions policy Provide a short summary statement on when to expect revisions and why – Make this readily available to users Attempt to maintain a stable revisions cycle from year to year Balance need to introduce new concepts / methods against user need for “stable” data Backcast data back several years to give a consistent time series Make documentation on revisions readily available to users Undertake analyses to give users an indication of the size of revisions based on past history Correct mistakes in a transparent and timely manner

30. 30 DATA REVISION Documentation of revisions Should cover: Whether data are preliminary or provisional Provision of advance notice of major changes in concepts, classifications or methods, etc., Explanation of the sources of revisions when the revised series are released Breaks in series when backcasting cannot be undertaken