1. ESS conceptual standards for quality reporting
Giorgia Simeoni
Istat,
ESTP Training Course
“Advanced course on quality reporting”
Luxembourg, June 2018

2. Outline ESS quality reporting: commitment
The process of harmonisation of ESS quality reporting
ESS standards for metadata and quality reporting

3. ESS Quality reporting: commitment
Starting point:“Quality” section on EU Regulations
Domain specific requirements
Mandatory, but only for some domains
Comparability of Quality Reports among Countries and not among domains
Producer-oriented reports (internal, monitoring purposes)

4. ESS Quality reporting: commitment
European statistics Code of Practice
Indicator 4.3: Output quality is regularly monitored, assessed with regard to possible trade-offs, and reported according to the quality criteria for European Statistics
Harmonisation of contents
Application to all European Statistics

5. ESS Quality reporting: commitment
Quality Assurance Framework of the European Statistical System
Indicator Methods at product/survey level
User oriented quality reports. User oriented quality reports are made available to the public.
Producer oriented quality reports. Producer oriented quality reports are published regularly […].
Product quality monitoring. Users and producers quality reporting are used for regular quality monitoring over time.
Promotion of dissemination to improve transparency
Distinction user/producer oriented
Useful for quality monitoring purposes

6. ESS Quality reporting: commitment
European Statistical Law
Regulation (EU) 2015/759 of 29 April 2015 amending Regulation (EC) No 223/2009 on European statistics.
Art. 12 (3): “Member States shall provide the Commission (Eurostat) with reports on the quality of data transmitted…”

7. The process of harmonisation of ESS quality reporting
National implementation of SQR starts (ESS-MH)
How to make a quality report
Standard quality reports
2003
Standard quality indicators
2005
ESMS
ESS Handbook for quality reports
2009
ESS Quality and Performance Indicators
ESQRS
2010
WG “Assessment of Quality in Statistics”
Task force on quality indicators
Eurostat starts implementing ESMS
Expert Group on Quality Indicators

8. Grants to support national implementation
The process of harmonisation of ESS quality reporting
SIMS
ESS Handbook for quality reports 2014
2014
SIMS, ESMS, ESQRS v.2.0
Implementation plan (2018)
Interoperability
No html in ESMS
2015
ESTP Advanced course on quality reporting
ESS Metadata and Quality Reporting Handbook
Alternative solution for domain specific needs
Task force on quality reporting
Grants to support national implementation
Task force on metadata implementation

9. The Single Integrated Metadata Structure (SIMS)
SIMS is a dynamic and unique inventory of ESS standard reference metadata statistical concepts with definitions and reporting guidelines.

10. SIMS 2.0

11. ESS Quality reporting: commitment
European Statistical Law
Regulation (EU) 2015/759 of 29 April 2015 amending Regulation (EC) No 223/2009 on European statistics.
Art. 12 (3): “Member States shall provide the Commission (Eurostat) with reports on the quality of data transmitted…”
The Single Integrated Metadata Structure (SIMS) version 2.0 and the ESS Handbook for Quality report 2014 are approved as ESS standard by the ESSC
SIMS can be considered the standard for quality reporting according to Article 12 of Regulation 223/2009 on European statistics

12. What are ESMS and ESQRS?
ESMS and ESQRS are 2 structures (templates) that can be derived from SIMS
Euro SDMX Metadata Structure (ESMS) recommended for user-oriented quality reporting
ESS Standard for Quality Reports Structure (ESQRS) recommended for producer-oriented quality reporting

13. ESMS 2.0

14. ESQRS 2.0

15. SIMS 2.0
ESQRS 2.0
ESMS 2.0

16. ESS quality reporting: tools and standards

17. Methodological manuals
Further methodological indications are needed to fill-in quality reports…
ESS Handbook for Quality Reports

18. ESS Handbook for quality reports
The new version of the Handbook will be probably available in mid Eurostat will provide more information on this topic tomorrow

19. ESS handbook for quality reports 2014
Aim: to provide guidelines for the
preparation of comprehensive quality reports for a full range of statistical processes and their
outputs
Types of statistical processes:
Sample survey
Census
Statistical process using administrative source(s)
Statistical process involving multiple data sources
Price or other economic index process
Statistical compilation

20. The structure of the new handbook will be aligned with SIMS 2.0
ESS handbook for quality reports 2014
The chapters follow the (old) ESQRS structure because it is the standard for producer-oriented quality reports
Main sections:
Introduction
Relevance
Accuracy and Reliability
Timeliness and Punctuality
Coherence and Comparability
Accessibility and Clarity
Cost and burden
Confidentiality
Statistical processing
The structure of the new handbook will be aligned with SIMS 2.0

21. Annex I: ESS Quality and Performance Indicators
Set of standard quality indicators developed by an expert group
Indicators for each ESS quality dimension, if possible “product” oriented

22. ESS Quality and Performance Indicators
Timeliness and punctuality
Time lag - first results
Time lag - final results
Punctuality - delivery and publication
Comparability and coherence
Asymmetry for mirror flows statistics - coefficient
Length of comparable time series
Accessibility and clarity
Data tables - consultations
Metadata - consultations
Metadata completeness - rate
Relevance
Data completeness - rate
Accuracy and reliability
Sampling error - indicators
Over-coverage - rate
Common units - proportion
Unit non-response - rate
Item non-response - rate
Imputation - rate
Data revision - average size

23. Annex II: Technical manual of the SIMS
1. Introduction
2. Use and general characteristics of the SIMS
3. Distinction between the short user- (U) and the detailed producer- (P) oriented quality reports
4. Integration of the Quality and Performance Indicators in the SIMS
5. Implementation of the "once for all purposes" quality reporting strategy
Annex 1 Creation of the Single Integrated Metadata Structure from the ESMS and ESQRS
Annex 2 ESS guidelines for SIMS
Annex 3 New ESMS guidelines as derived from SIMS
Annex 4 New ESQRS guidelines as derived from SIMS
Annex 5 How to create and submit a report in NRME
SIMS v 2.0 – ESMS 2.0 and ESQRS 2.0 – 2015
ESS Metadata Handler

24. References
Eurostat (2015) Single Integrated Metadata Structure v 2.0 – ESMS 2.0 and ESQRS 2.0 – 2015
Eurostat (2015) Single Integrated Metadata Structure and its Technical Manual – 2014
Eurostat (2014) ESS Handbook for Quality reports (2014 edition)
Eurostat (2014) ESS Guidelines for the implementation of the ESS quality and performance indicators (QPI)
ESS Standard for Quality Reports Structure (release 1, October 2010)
Euro SDMX Metadata Structure (release 3, March 2009)
Eurostat (2009). Eurostat Methodologies and Working papers “ESS Standards for Quality Reports”.