1. Quality Reporting in CBS
ESTP "Advanced course on Quality Reporting"
Eurostat, Luxembourg, 21 – 22 March 2017
Quality Reporting in CBS
Slavica Rogar
Croatian Bureau of Statistics
Industry, Energy and Information Society Department
Luxembourg, 2017

2. Content Database of quality information (DBQI) Quality Reports
Conclusion and future developments
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

3. Database of quality information (DBQI)
Key tool for quality assessment, quality documentation and quality reporting for CBS surveys
Established on the exhaustive list of quality information based on two ESS structure
Part of the national IPA project
A central place for storing:
Direct information on quality estimation
Reference metadata
Documentation of survey processes
DBQI is planned to become a key tool for quality assessment, quality documentation and quality reporting for CBS surveys
It is established on the exhaustive list of quality information based on two widely accepted ESS structure-ESMS and ESQRS
As part of the national IPA project, much progress has been achieved within the component on the production of survey reports relating quality reports across the statistical survey work in the CBS.
QD is planned to be a central place for storing:
Direct information on quality estimation
Reference metadata
Documentation of survey processes
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

4. DBQI Homepage/ Data Entering
Homepage Data entering Research Num. Indicators Administrations Information
Data entering
List of surveys divided by sectors
Here we can see the homepage for data entering into the DBQI (QD). It has a list of all surveys grouped by sectors that can enter data for its quality database and quality reports.
You need to choose the name of the survey, and the reference period (month and year) for which you want to enter the data.
Survey
Reference period
Enter data
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

5. DBQI for STS- Industrial Production (1/2)
Monthly Survey on Industrial Production and Persons Employed (IND-1/KPS/M form) – 12/2016
Descriptive and numerical information/values
A list of all indicators with descriptions / instructions (excel)
Number Concept name Description Value/information Instructions
Here we can see the database look for Monthly Survey on Industrial Production and Persons Employed for December 2016
All the information in the database can roughly divide into two parts:
Descriptive (textual) information which we can see in this picture/slide and
Numerical information (quality indicators)- shown later
For help in filling in the textual information, we have an excel table with all indicators and theirs detailed descriptions and instructions to download.
In the end, we can generate the quality report in Croatian and/or English.
Quality report
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

6. DBQI Metadata Structure1
Contact
1.1
Contact name
1.2
Contact organization/unit
1.3
Contact mail address
1.4
Contact phone number
1.5
Contact address
1.6
Purpose, goal and subject of the survey (introduction) 2
Metadata update
2.1
Metadata last certified
2.2
Metadata last posted
2.3
Metadata last update 3
Statistical presentation
3.1
Data description
3.2
Classification system
3.3
Sector coverage
3.4
Statistical concepts and definitions
3.5
Statistical unit
3.6
Statistical population
3.7
Reference area
3.8
Time coverage
3.9
Base period 4
Unit of measure 5
Reference period 6
Institutional mandate
6.1
Legal acts and other agreements
6.2
Data sharing 7
Confidentiality
7.1
Confidentiality- policy
7.2
Confidentiality- data processing
7.3
Data protection treatment
7.4
Accessibility and clarity 8
Release policy
8.1
Release calendar
8.2
Release calendar access
8.3
User access 9
Frequency of dissemination 10
Dissemination format
10.1
News release
10.2
Publications
10.3
On-line database
10.4
Micro-data access
10.5
Other 11
Accessibility of documentation
11.1
Documentation on methodology
11.2
Quality documentation 12
Quality management
12.1
Quality assurance
12.2
Quality assessment 13
Relevance
13.1
Data users
13.2
User needs
13.3
User satisfaction
13.4
Completeness 14
Accuracy and reliability
14.1
Overall accuracy
14.2
Sampling error
14.3
Non-sampling error
14.4
Measurement error
14.5
Non-response error
14.6
Processing error
14.7
Coverage error
14.8
Model assumption error 15
Timeliness and punctuality
15.1
Timeliness
15.2
Punctuality 16
Comparability
16.1
Comparability- geographical
16.2
Asymmetry for mirror flows statistics
16.3
Seasonal adjustment
16.4
Comparability- over time
16.5
Reasons for time break series 17
Coherence
17.1
Coherence- cross domain
17.2
Coherence- internal 18
Cost and burden
18.1
Cost
18.2
Burden 19
Data revision
19.1
Data revision- policy
19.2
Data revision- practice 20
Statistical processing
20.1
Source data
20.2
Frequency of data collection
20.3
Data collection
20.4
Data validation
20.5
Data compilation
20.6
Adjustment 21
Comment
Here is our metadata structure. As I have already mentioned, it has been established combining two accepted ESS structure: ESMS and ESQRS
The Quality Reports on our website include only the numbers rounded in red. Unfortunately, it is available only in Croatian.
We are doing are best to publish quality reports in English.
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

7. DBQI for STS- Industrial Production (2/2)
Monthly Survey on Industrial Production and Persons Employed (IND-1/KPS/M form) – 12/2016
Handbook for Calculation of Quality Indicators
Dimensions Quality and Performance Indicators (QPIs) Key/auxiliary indicators tag Data Description
Relevance
R1 Data completeness rate
Punctuality
On this slide, we can see the second part of the database, the numerical information.
In order to fulfill as well as possible, we wrote a Handbook for Calculation of QIs, where for each of the indicator are provided: the definition of the indicator, calculations procedure and example(s)
We also had several workshops on Quality Reporting and Quality Assessment Methods and Tools
A2 Over-coverage rate
A4 Unit non-response rate
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

8. DBQI Metadata Structure (QPIs)
Relevance
R1 Data completeness rate
Accuracy
A1 Sampling error indicators
A8 Bias due to selection process
A2 Over-coverage rate
A4 Unit non-response rate
A5 Item non-response rate
A6 Data revision- average size
A7 Imputation rate
A9 Editing rate
A10 Edit failure rate
A11 Hit rate
A12 Misclassification rate
Timeliness and punctuality
TP1 Time lag – first results
TP2 Time lag – final results
TP3 Punctuality
Comparability
CC1 Length of comparable time series
CC2 Asymmetry for mirror flows statistics- coefficient
Coherence
CH1 Coherence- administrative sources
CH2 Coherence- short-term and structural data
CH3 Coherence- national accounts
Here is the list of quality and performance indicators available in our Quality Database. Only the two of the listed indicators are not in the quality report.
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

9. Quality Reports on CBS website
Reporting on Quality of Statistical Surveys
Quality reporting template
Handbook for calculating key quality indicators
On our website under the domain Quality/ Quality Reporting, we have published 30 quality reports in Croatian. Currently, we are planning to publish 5 quality reports of industry department (such as Industrial Production, Industrial Turnover…) hopefully in Croatian and English.
Survey results
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

10. Conclusion and future developments
The system for disseminating information about the quality of statistics to their users should continue to be developed.
Room for improving existing quality database
The main challenge- data access for calculating QPIs
The system for disseminating information about the quality of statistics to their users should continue to develop.
There is always room to improve existing quality database so that published quality reports could become even better.
The main challenge is to have the possibility of finding the data not to fill in and publish quality reports. That calls a great IT sector support and development of our applications for data processing so we could fill in all QPIs.
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017

11. Thank you for your attention!
Slavica Rogar
ESTP "Advanced course on Quality Reporting" Eurostat, Luxembourg, 21 – 22 March 2017