The European Statistics Code of Practice - a Basis for Eurostat’s Quality Assurance Framework Marie Bohatá Deputy Director General, Eurostat ... Strategic Management Seminar, Split 2007

European Statistics Code of Practice Institutional Environment (Professional independence, Mandate for data collection, Adequacy of resources, Quality commitment, Statistical confidentiality, Impartiality and objectivity) Statistical Processes (Sound methodology, Appropriate statistical procedures, Non-excessive burden on respondents, Cost effectiveness) Statistical Outputs (Relevance, Accuracy and reliability, Timeliness and punctuality, Coherence and comparability, Accessibility and clarity)

European Statistics Code of Practice Self-assessments in 2005 (1)

European Statistics Code of Practice Self-assessments in 2005 (2)

European Statistics Code of Practice Self-assessments in 2005 (3)

Features of Eurostat Quality Initiatives within the Implementation of the CoP Eurostat approach to quality based on general Total Quality Management Objective: a permanent improvement of an organisation Recently the focus shifted to the CoP which provides an encompassing conceptual common ground Objective: enhancement of quality of statistics produced jointly by the ESS; institutional and legal aspects constitute an integral part of the framework

The Context of TQM and the CoP User needs Statistical products Production processes Institutional environment Management systems and leadership Support 1 Professional independence 2 Mandate for data collection 3 Adequacy of resources 4 Quality commitment 5 Statistical confidentiality 6 Impartiality and objectivity 7 Sound methodology 8 Appropriate statistical procedures 9 Non-excessive burden on respondents 10 Cost effectiveness 11 Relevance 12 Accuracy and reliability 13 Timeliness and Punctuality 14 Coherence and comparability 15 Accessibility And clarity TQM Code of Practice N.B. Figure derived from draft Handbook on Data Quality Assessment Methods and Tools (DatQAM), version 31.01.2007.

Eurostat Quality Assurance Framework - Objectives To establish a system of coordinated methods and tools guaranteeing adherence to minimum requirements concerning processes and outputs including some kind of assessment To build upon existing initiatives and complement them with new approaches and tools

Methods and Tools for Quality Assessment Production processes Statistical products User perception Process variables Quality indicators reports User satisfaction survey Self assessments Quality reviews Labelling Institutional/ legal environment User requirements Standards III. Conformity II. Evaluation I. Documentation Measurement Improvement actions N.B. Figure derived from draft Handbook on Data Quality Assessment Methods and Tools (DatQAM), version 31.01.2007.

Methods and Tools for Quality Assessment Production processes Statistical products User perception Process variables Quality indicators reports User satisfaction survey Self assessments Quality reviews Labelling Institutional/ legal environment User requirements Standards III. Conformity II. Evaluation I. Documentation Measurement Improvement actions DatQAM (DESTATIS) DESAP Checklist (DESTATIS, Lithuania) Auditing activities in NSI’s (INE-PT) Methods for evaluating response burden (SSB) Handbook on process-variables (ONS) ESS Standard Quality Indicators Customer/ user satisfaction surveys (SCB) ESS Quality Reports Handbook Questionnaire development (ISTAT) Editing and Imp in Business surveys (ISTAT) Handbook on seasonal adjust-ment (HCSO) Guidelines on accuracy and delays (INSEE)

Quality Assessment Packages Fundamental package Intermediate Advanced Process descriptions, product documentation, quality guidelines Quality reports Self assessments Quality reviews User satisfaction surveys Quality indicators Key process variables Labelling N.B. Figure derived from draft Handbook on Data Quality Assessment Methods and Tools (DatQAM), version 31.01.2007.

Implementation Activities in Eurostat Adaptation of DESAP/CCSA checklist Definition of a methodology for quality reviews Harmonisation and extension of quality reporting Preparation of an approach for labelling Office-wide synthesis management tools

DESAP/CCSA Adapted Checklist Objectives of the Checklist To provide an overall view on the quality of outputs and processes produced (part of systematic quality assurance activities) To identify the potential quality risks and improvement actions To allow monitoring and comparisons of the quality level over time and with caution across the domains

DESAP/CCSA Adapted Checklist Resulting assessment diagram – an example

Methodology for Quality reviews Different type of reviews to be developed: Internal reviews Self-assessments Reviews with participation of Eurostat staff External reviews Including NSIs, Commission’s DGs staff, etc.. Rolling reviews (external contractors, producer/users surveys, cost assessments) Mapping between domains/reviews Program of reviews linked to 2008-2012 Methodology and program of reviews ready by the end of September

Harmonisation/Extension of Quality Reporting Revision of guidelines for quality reporting Harmonisation of Regulations in the area of quality: Revised umbrella legislation 322/97 Standard articles in new regulations Specification in (Commission) implementation regulations (evaluation criteria)

Synthetic office wide management tools Performance scoreboard Eurostat product and services Structured according to quality dimensions Quality barometer (QB) Monitor data quality in the ESS Accross domains/over time Based on quality reports and quality indicators First release in 2008

Objectives of Labelling To inform and guide users To promote the CoP among users To position official European statistics in the information market To enhance quality of European statistics and quality reporting

User satisfaction surveys not (fully) compliant with CoP P.7-15 compliance with CoP P.7-15 statistics not (fully) compliant with CoP P.7-15 compliance with CoP P.7-15 Advanced package Labelling Procedure improvement actions Key process variables Intermediate package User satisfaction surveys Quality reviews Quality indicators Self assessments Quality reports Fundamental package Process descriptions, product documentation, quality quidelines European statistics not eligible for labelling official European statistics (compliance with CoP P.1-6)

Features of labelling Eurostat relies on quality reporting by NSIs and promotes systematic quality reviews of statistical domains at national level The assessment basis is derived from the ESS self-assessment checklist for survey managers and incorporates elements from the CoP questionnaire Quality reviews of Eurostat domains form a crucial element. While all domains will be subject to some quality review, domains selected for labelling will undergo a more thorough exercise involving also external stakeholders The labelling authority will be designated to grant the label and a communication and dissemination strategy will need to be carefully designed