1. Quality report contents: Metadata on quality
Remi Prual
Estonia
ESTP Training Course
“Advanced course on quality reporting”
Luxembourg, June 2018

2. Contents Concept / Definition / Guidelines / Examples
Metadata on quality
Confidentiality
Release policy
Dissemination format, accessibility and clarity
Relevance
Timeliness and punctuality
Coherence and comparability
Comment

3. Confidentiality Concept Descriptions ESS Guidelines Confidentiality
A property of data indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interest of the source or other relevant parties. -
Confidentiality - policy
Legislative measures or other formal procedures which prevent unauthorised disclosure of data that identify a person or economic entity either directly or indirectly.
The European and national legislations (or any other formal provision) related to statistical confidentiality applied for the data set in question should be described. It means the assurance that all necessary methods assuring confidentiality have been applied to the data.
Confidentiality - data treatment
Rules applied for treating the microdata and macrodata (including tabular data) to ensure statistical confidentiality and prevent unauthorised disclosure.
The rules applied for treating the microdata and macrodata (including tabular data) with regard to statistical confidentiality should be described (e.g. controlled rounding, cell suppression, aggregation of disclosed information, aggregation rules on aggregated confidential data, primary confidentiality with regard to single data values, etc.).

4. Confidentiality: examples
Concept
Descriptions
ESS Guidelines
Confidentiality
A property of data indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interest of the source or other relevant parties. -
Confidentiality - policy
Legislative measures or other formal procedures which prevent unauthorised disclosure of data that identify a person or economic entity either directly or indirectly.
The European and national legislations (or any other formal provision) related to statistical confidentiality applied for the data set in question should be described. It means the assurance that all necessary methods assuring confidentiality have been applied to the data.
Confidentiality - data treatment
Rules applied for treating the microdata and macrodata (including tabular data) to ensure statistical confidentiality and prevent unauthorised disclosure.
The rules applied for treating the microdata and macrodata (including tabular data) with regard to statistical confidentiality should be described (e.g. controlled rounding, cell suppression, aggregation of disclosed information, aggregation rules on aggregated confidential data, primary confidentiality with regard to single data values, etc.).
Text -
The dissemination of data collected for the purpose of producing official statistics is guided by the requirements provided for in § 34 and § 35 of the Official Statistics Act.
The treatment of confidential data is regulated by the Procedure for Protection of Data Collected and Processed by Statistics Estonia:

5. Confidentiality: lessons learned
Concept
Descriptions
ESS Guidelines
Confidentiality
A property of data indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interest of the source or other relevant parties. -
Confidentiality - policy
Legislative measures or other formal procedures which prevent unauthorised disclosure of data that identify a person or economic entity either directly or indirectly.
The European and national legislations (or any other formal provision) related to statistical confidentiality applied for the data set in question should be described. It means the assurance that all necessary methods assuring confidentiality have been applied to the data.
Confidentiality - data treatment
Rules applied for treating the microdata and macrodata (including tabular data) to ensure statistical confidentiality and prevent unauthorised disclosure.
The rules applied for treating the microdata and macrodata (including tabular data) with regard to statistical confidentiality should be described (e.g. controlled rounding, cell suppression, aggregation of disclosed information, aggregation rules on aggregated confidential data, primary confidentiality with regard to single data values, etc.).
Text -
The dissemination of data collected for the purpose of producing official statistics is guided by the requirements provided for in § 34 and § 35 of the Official Statistics Act.
The treatment of confidential data is regulated by the Procedure for Protection of Data Collected and Processed by Statistics Estonia:
More harmonisation could be used.
In some cases too generic descriptions. Guidelines require more.

6. Release policy Concept Descriptions ESS Guidelines Release policy
Rules for disseminating statistical data to all interested parties. -
Release calendar
The schedule of statistical release dates.
The policy regarding the release of statistics in question should be described, in particular if it follows a preannounced schedule. It should also be mentioned if a release calendar for the data set in question exists and if this calendar is publicly accessible.
Release calendar access
Access to the release calendar information.
The link or reference to the release calendar should be given.
User access
The policy for release of the data to users, the scope of dissemination, how users are informed that the data are being released, and whether the policy determines the dissemination of statistical data to all users.
The general policy of the organisation for data release to users should be described. This includes the scope of dissemination (e.g. to the public, to selected users), how users are informed that the data is being released, and whether the release policy determines the dissemination of statistical data to all users at the same time. For Eurostat only: Reference is also made to the impartiality protocol linked to the European Statistics Code of Practice, principle 6, where the responsible for the statistical domain should state all kinds of pre-releases.

7. Release policy: examples
Concept
Descriptions
ESS Guidelines
Release policy
Rules for disseminating statistical data to all interested parties. -
Release calendar
The schedule of statistical release dates.
The policy regarding the release of statistics in question should be described, in particular if it follows a preannounced schedule. It should also be mentioned if a release calendar for the data set in question exists and if this calendar is publicly accessible.
Release calendar access
Access to the release calendar information.
The link or reference to the release calendar should be given.
User access
The policy for release of the data to users, the scope of dissemination, how users are informed that the data are being released, and whether the policy determines the dissemination of statistical data to all users.
The general policy of the organisation for data release to users should be described. This includes the scope of dissemination (e.g. to the public, to selected users), how users are informed that the data is being released, and whether the release policy determines the dissemination of statistical data to all users at the same time. For Eurostat only: Reference is also made to the impartiality protocol linked to the European Statistics Code of Practice, principle 6, where the responsible for the statistical domain should state all kinds of pre-releases.
Text -
Notifications about the dissemination of statistics are published in the release calendar, which is available on the website. On 1 October each year, the release times of the Statistical Database, news releases, main indicators by IMF SDDS and publications are announced in the release calendar (in case of publications – the release month).
All users have been granted an equal access to official statistics: this means that the dissemination dates of official statistics have to be announced in advance and no user category (incl. Eurostat, state authorities and mass media) can have access to the official statistics (results of official statistical surveys) before other users. Statistical information is first published in the Statistical Database. In case a news release is published based on the same data, the information provided in the relevant news release is simultaneously published in the Statistical Database. Official statistics are available on the website at 8.00 a.m. on the date announced in the release calendar.

8. Release policy: lessons learned
Concept
Descriptions
ESS Guidelines
Release policy
Rules for disseminating statistical data to all interested parties. -
Release calendar
The schedule of statistical release dates.
The policy regarding the release of statistics in question should be described, in particular if it follows a preannounced schedule. It should also be mentioned if a release calendar for the data set in question exists and if this calendar is publicly accessible.
Release calendar access
Access to the release calendar information.
The link or reference to the release calendar should be given.
User access
The policy for release of the data to users, the scope of dissemination, how users are informed that the data are being released, and whether the policy determines the dissemination of statistical data to all users.
The general policy of the organisation for data release to users should be described. This includes the scope of dissemination (e.g. to the public, to selected users), how users are informed that the data is being released, and whether the release policy determines the dissemination of statistical data to all users at the same time. For Eurostat only: Reference is also made to the impartiality protocol linked to the European Statistics Code of Practice, principle 6, where the responsible for the statistical domain should state all kinds of pre-releases.
Text -
Notifications about the dissemination of statistics are published in the release calendar, which is available on the website. On 1 October each year, the release times of the Statistical Database, news releases, main indicators by IMF SDDS and publications are announced in the release calendar (in case of publications – the release month).
All users have been granted an equal access to official statistics: this means that the dissemination dates of official statistics have to be announced in advance and no user category (incl. Eurostat, state authorities and mass media) can have access to the official statistics (results of official statistical surveys) before other users. Statistical information is first published in the Statistical Database. In case a news release is published based on the same data, the information provided in the relevant news release is simultaneously published in the Statistical Database. Official statistics are available on the website at 8.00 a.m. on the date announced in the release calendar.
Mostly OK.
Mostly OK.
Mostly OK.

9. Accessibility and clarity 1/3
Concept
Descriptions
ESS Guidelines
Frequency of dissemination
The time interval at which the statistics are disseminated over a given time period.
The frequency with which the data is disseminated (e.g. monthly, quarterly, yearly) should be stated. The frequency can also be expressed by using the codes released in the harmonised code list available for the European Statistical System as long as it is easily understandable.
Accessibility and clarity
Media, various means and formats by which statistical data and metadata are disseminated to users and their accessibility. Accessibility and clarity refer to the simplicity and ease, the conditions and modalities by which users can access, use and interpret statistics, with the appropriate supporting information and assistance. -
News release
Regular or ad-hoc press releases linked to the data.
Regular or ad-hoc press releases linked to the data set in question should be described.
Publications
Regular or ad-hoc publications in which the data are made available to the public.
The titles of publications using the data set in question should be listed, with publisher, year and link to on-line documents if available.
On-line database
Information about on-line databases in which the disseminated data can be accessed.
The on-line database available for the data set in question should be described. This includes the domain names as released on the website and link to the on-line database.
AC1. Data tables - consultations
Number of consultations of data tables within a statistical domain for a given time period displayed in a graph.
QPI: AC1 Data tables - consultations

10. Accessibility and clarity 1/3: examples
Concept
Descriptions
ESS Guidelines
Frequency of dissemination
The time interval at which the statistics are disseminated over a given time period.
The frequency with which the data is disseminated (e.g. monthly, quarterly, yearly) should be stated. The frequency can also be expressed by using the codes released in the harmonised code list available for the European Statistical System as long as it is easily understandable.
Accessibility and clarity
Media, various means and formats by which statistical data and metadata are disseminated to users and their accessibility. Accessibility and clarity refer to the simplicity and ease, the conditions and modalities by which users can access, use and interpret statistics, with the appropriate supporting information and assistance. -
News release
Regular or ad-hoc press releases linked to the data.
Regular or ad-hoc press releases linked to the data set in question should be described.
Publications
Regular or ad-hoc publications in which the data are made available to the public.
The titles of publications using the data set in question should be listed, with publisher, year and link to on-line documents if available.
On-line database
Information about on-line databases in which the disseminated data can be accessed.
The on-line database available for the data set in question should be described. This includes the domain names as released on the website and link to the on-line database.
AC1. Data tables - consultations
Number of consultations of data tables within a statistical domain for a given time period displayed in a graph.
QPI: AC1 Data tables - consultations
Text
Yearly
Monthly -
The news release “Producer price index of industrial output, export and import price index” once a month. The news release can be viewed on the website.
„Eesti statistika aastaraamat. Statistical Yearbook of Estonia”
„Eesti Statistika Kvartalikiri. Quarterly Bulletin of Statistics Estonia”
Data are published under the heading „Economy/ Prices” in the Statistical Database in
QPI

11. Accessibility and clarity 1/3: lessons learned
Concept
Descriptions
ESS Guidelines
Frequency of dissemination
The time interval at which the statistics are disseminated over a given time period.
The frequency with which the data is disseminated (e.g. monthly, quarterly, yearly) should be stated. The frequency can also be expressed by using the codes released in the harmonised code list available for the European Statistical System as long as it is easily understandable.
Accessibility and clarity
Media, various means and formats by which statistical data and metadata are disseminated to users and their accessibility. Accessibility and clarity refer to the simplicity and ease, the conditions and modalities by which users can access, use and interpret statistics, with the appropriate supporting information and assistance. -
News release
Regular or ad-hoc press releases linked to the data.
Regular or ad-hoc press releases linked to the data set in question should be described.
Publications
Regular or ad-hoc publications in which the data are made available to the public.
The titles of publications using the data set in question should be listed, with publisher, year and link to on-line documents if available.
On-line database
Information about on-line databases in which the disseminated data can be accessed.
The on-line database available for the data set in question should be described. This includes the domain names as released on the website and link to the on-line database.
AC1. Data tables - consultations
Number of consultations of data tables within a statistical domain for a given time period displayed in a graph.
QPI: AC1 Data tables - consultations
Text
Yearly
Monthly -
The news release “Producer price index of industrial output, export and import price index” once a month. The news release can be viewed on the website.
„Eesti statistika aastaraamat. Statistical Yearbook of Estonia”
„Eesti Statistika Kvartalikiri. Quarterly Bulletin of Statistics Estonia”
Data are published under the heading „Economy/ Prices” in the Statistical Database in
QPI
More harmonisation could be used.
More harmonisation could be used.
Mostly OK.
Mostly OK.

12. AC1. Data tables - consultation (Eurostat level)
Number of consultations of data tables within a statistical domain for a given time period. By "number of consultations" it is meant number of data tables views, where multiples views in a single session count only once.
AC2 = #CONS
The frequency of collection of the figures for this indicator should be monthly. This indicator contributes also to the assessment of the relevance of subject matter domains. It is interesting to monitor the trend of this indicator over time. It could be done producing a graph with the months in the horizontal axis and the number of consultation on the vertical axis. 12

13. Accessibility and clarity 2/3
Concept
Descriptions
ESS Guidelines
Micro-data access
Information on whether micro-data are also disseminated.
Describe if and how the data set is accessible as micro-data (e.g. for researchers). Also the micro-data anonymisation rules should be described in short.
Other
References to the most important other data dissemination done.
The most important other data dissemination means should be described (e.g. within other publications, policy papers, etc.) and an overview of the different aspects of the dissemination practice and their impact on accessibility and clarity of the data should be stated. For Member States: Pricing policies and registration for database access and their likely effect on access should be described together with the limits on access set by confidentiality provisions and any other restrictions; dissemination of data to Eurostat and other international organisations (IMF, OECD, ... if applicable and not described under "S.7.1 Legal acts and other agreements"), and internal dissemination of data to other statistical activities within the NSI.
AC 2. Metadata - consultations
Number of metadata consultations within a statistical domain for a given time period.
QPI: AC2 Metadata - consultations

14. Accessibility and clarity 2/3: examples
Concept
Descriptions
ESS Guidelines
Micro-data access
Information on whether micro-data are also disseminated.
Describe if and how the data set is accessible as micro-data (e.g. for researchers). Also the micro-data anonymisation rules should be described in short.
Other
References to the most important other data dissemination done.
The most important other data dissemination means should be described (e.g. within other publications, policy papers, etc.) and an overview of the different aspects of the dissemination practice and their impact on accessibility and clarity of the data should be stated. For Member States: Pricing policies and registration for database access and their likely effect on access should be described together with the limits on access set by confidentiality provisions and any other restrictions; dissemination of data to Eurostat and other international organisations (IMF, OECD, ... if applicable and not described under "S.7.1 Legal acts and other agreements"), and internal dissemination of data to other statistical activities within the NSI.
AC 2. Metadata - consultations
Number of metadata consultations within a statistical domain for a given time period.
QPI: AC2 Metadata - consultations
Text
The dissemination of data collected for the purpose of producing official statistics is guided by the requirements provided for in § 34, § 35, § 36, § 37, § 38 of the Official Statistics Act.
Data serve as input for statistical activities „National accounts (annual)”, „National accounts (quarterly)”, „Regional GDP”, „Supply and use tables”, „Foreign trade” and „Producer price index of industrial output”.
There is the option of requesting customised information from the INE Customer Service Area. Limitations to confidentiality or precision are borne in mind at the time of processing said requests. This service may be accessed via the following link: www
QPI

15. More harmonisation could be used.
Accessibility and clarity 2/3: lessons learned
Concept
Descriptions
ESS Guidelines
Micro-data access
Information on whether micro-data are also disseminated.
Describe if and how the data set is accessible as micro-data (e.g. for researchers). Also the micro-data anonymisation rules should be described in short.
Other
References to the most important other data dissemination done.
The most important other data dissemination means should be described (e.g. within other publications, policy papers, etc.) and an overview of the different aspects of the dissemination practice and their impact on accessibility and clarity of the data should be stated. For Member States: Pricing policies and registration for database access and their likely effect on access should be described together with the limits on access set by confidentiality provisions and any other restrictions; dissemination of data to Eurostat and other international organisations (IMF, OECD, ... if applicable and not described under "S.7.1 Legal acts and other agreements"), and internal dissemination of data to other statistical activities within the NSI.
AC 2. Metadata - consultations
Number of metadata consultations within a statistical domain for a given time period.
QPI: AC2 Metadata - consultations
Text
The dissemination of data collected for the purpose of producing official statistics is guided by the requirements provided for in § 34, § 35, § 36, § 37, § 38 of the Official Statistics Act.
Data serve as input for statistical activities „National accounts (annual)”, „National accounts (quarterly)”, „Regional GDP”, „Supply and use tables”, „Foreign trade” and „Producer price index of industrial output”.
There is the option of requesting customised information from the INE Customer Service Area. Limitations to confidentiality or precision are borne in mind at the time of processing said requests. This service may be accessed via the following link: www
QPI
Mostly OK.
More harmonisation could be used.

16. AC2. Metadata - consultation (Eurostat level)
Number of metadata consultations (ESMS) within a statistical domain for a given time period. By "number of consultations" it is meant the number of times a metadata file is viewed.
AC1 = #ESMS
#ESMS denotes the absolute number of elements in the set of ESMS files consulted for a specific subject-matter domain for a given time period
It is interesting to monitor the trend of this indicator over time. It could be done producing a graph with the months in the horizontal axis and the number of consultation on the vertical axis. 16

17. Accessibility and clarity 3/3
Concept
Descriptions
ESS Guidelines
Documentation on methodology
Descriptive text and references to methodological documents available.
Describe the availability of national reference metadata files, important methodological papers, summary documents or other important handbooks. Title, publisher, year and links to on-line documents if possible should be described.
AC 3. Metadata completeness - rate
The ratio of the number of metadata elements provided to the total number of metadata elements applicable.
QPI: AC3 Metadata completeness - rate
Quality documentation
Documentation on procedures applied for quality management and quality assessment.
Describe the availability of all quality related documents (quality reports, studies, etc). For Eurostat: The responsible of the statistical domain should also describe the availability of national quality reports. More detailed information about quality processes should be described in S.13.1 and S.13.2.

18. Accessibility and clarity 3/3: examples
Concept
Descriptions
ESS Guidelines
Documentation on methodology
Descriptive text and references to methodological documents available.
Describe the availability of national reference metadata files, important methodological papers, summary documents or other important handbooks. Title, publisher, year and links to on-line documents if possible should be described.
AC 3. Metadata completeness - rate
The ratio of the number of metadata elements provided to the total number of metadata elements applicable.
QPI: AC3 Metadata completeness - rate
Quality documentation
Documentation on procedures applied for quality management and quality assessment.
Describe the availability of all quality related documents (quality reports, studies, etc). For Eurostat: The responsible of the statistical domain should also describe the availability of national quality reports. More detailed information about quality processes should be described in S.13.1 and S.13.2.
Text
Handbook on Industrial Producer Price Indices (PPI), Eurostat (2012)
A detailed description may be viewed at:
QPI
Quality reports are compiled and sent to Eurostat, in compliance with Commission Regulation 698/2006, every four years (when information is sent to Eurostat). The last available report refers to the survey with reference year 2008 and can be consulted at: html

19. Accessibility and clarity 3/3: lessons learned
Concept
Descriptions
ESS Guidelines
Documentation on methodology
Descriptive text and references to methodological documents available.
Describe the availability of national reference metadata files, important methodological papers, summary documents or other important handbooks. Title, publisher, year and links to on-line documents if possible should be described.
AC 3. Metadata completeness - rate
The ratio of the number of metadata elements provided to the total number of metadata elements applicable.
QPI: AC3 Metadata completeness - rate
Quality documentation
Documentation on procedures applied for quality management and quality assessment.
Describe the availability of all quality related documents (quality reports, studies, etc). For Eurostat: The responsible of the statistical domain should also describe the availability of national quality reports. More detailed information about quality processes should be described in S.13.1 and S.13.2.
Text
Handbook on Industrial Producer Price Indices (PPI), Eurostat (2012)
A detailed description may be viewed at:
QPI
Quality reports are compiled and sent to Eurostat, in compliance with Commission Regulation 698/2006, every four years (when information is sent to Eurostat). The last available report refers to the survey with reference year 2008 and can be consulted at: html
Mostly OK.
Mostly OK.

20. AC3. Rate of Metadata Completeness (Eurostat level)
The ratio of the number of metadata elements provided to the total number of metadata elements applicable.
L is the set of applicable metadata elements under consideration
ML is the subset of L of available metadata elements
The reference is ESMS structure. The target value is 1 meaning that all the Euro-SDMX Metadata applicable concepts are provided.
Computed separately for three subgroups of concepts: metadata on statistical outputs, statistical processes and quality. 20

21. Relevance Concept Descriptions ESS Guidelines Relevance
The degree to which statistical information meet current and potential needs of the users. -
User needs
Description of users and their respective needs with respect to the statistical data.
Provide: - a classification of users with some indication of their importance; - an indication of the uses for which they want the statistical outputs; - an assessment regarding the key outputs/indicators desired by different categories of users and any shortcomings in outputs for important users; - information on unmet user needs, the reasons why certain needs cannot be fully satisfied, - any plans to satisfy needs more completely in the future ; and - details of definitions which differ from requirements.
User satisfaction
Measures to determine user satisfaction.
Describe how the views and opinions of the users are regularly collected (e.g. user satisfaction surveys, other user consultations, ...). In addition the main results regarding investigation of user satisfaction should be shown (in the form of a user satisfaction index if available) and the date of most recent user satisfaction survey.
R1. Data completeness - rate for U
The extent to which all statistics that are needed are available.
Provide qualitative information on completeness compared with relevant regulations/ guidelines. Applicable for Eurostat: if any Member State is not transmitting all necessary data items.
R1. Data completeness - rate for P
The ratio of the number of data cells provided to the number of data cells required.
QPI: R1, Data completeness - rate for P, with different CALCULATION FORMULA for U and P

22. Relevance: examples Text Concept Descriptions ESS Guidelines Relevance
The degree to which statistical information meet current and potential needs of the users. -
User needs
Description of users and their respective needs with respect to the statistical data.
Provide: - a classification of users with some indication of their importance; - an indication of the uses for which they want the statistical outputs; - an assessment regarding the key outputs/indicators desired by different categories of users and any shortcomings in outputs for important users; - information on unmet user needs, the reasons why certain needs cannot be fully satisfied, - any plans to satisfy needs more completely in the future ; and - details of definitions which differ from requirements.
User satisfaction
Measures to determine user satisfaction.
Describe how the views and opinions of the users are regularly collected (e.g. user satisfaction surveys, other user consultations, ...). In addition the main results regarding investigation of user satisfaction should be shown (in the form of a user satisfaction index if available) and the date of most recent user satisfaction survey.
R1. Data completeness - rate for U
The extent to which all statistics that are needed are available.
Provide qualitative information on completeness compared with relevant regulations/ guidelines. Applicable for Eurostat: if any Member State is not transmitting all necessary data items.
R1. Data completeness - rate for P
The ratio of the number of data cells provided to the number of data cells required.
QPI: R1, Data completeness - rate for P, with different CALCULATION FORMULA for U and P
Text -
Ministry of Economic Affairs and Communications
Users’ suggestions and information about taking them into account are available on the SE website
The key users of the RPI are: Central Bank of Malta (CBM); Economic Policy Division (EPD) within the Ministry for Finance (MFIN); The public in general for queries related to rents and maintenance agreements.
The INE has carried out general user satisfaction surveys in 2007, 2010 and 2013, and it plans to continue doing so every three years. In general, users are satisfied.
Since 1996 SE conducts reputation surveys and user surveys. All results are available on the website
Applicable for Eurostat.
QPI

23. Relevance: lessons learned
Concept
Descriptions
ESS Guidelines
Relevance
The degree to which statistical information meet current and potential needs of the users. -
User needs
Description of users and their respective needs with respect to the statistical data.
Provide: - a classification of users with some indication of their importance; - an indication of the uses for which they want the statistical outputs; - an assessment regarding the key outputs/indicators desired by different categories of users and any shortcomings in outputs for important users; - information on unmet user needs, the reasons why certain needs cannot be fully satisfied, - any plans to satisfy needs more completely in the future ; and - details of definitions which differ from requirements.
User satisfaction
Measures to determine user satisfaction.
Describe how the views and opinions of the users are regularly collected (e.g. user satisfaction surveys, other user consultations, ...). In addition the main results regarding investigation of user satisfaction should be shown (in the form of a user satisfaction index if available) and the date of most recent user satisfaction survey.
R1. Data completeness - rate for U
The extent to which all statistics that are needed are available.
Provide qualitative information on completeness compared with relevant regulations/ guidelines. Applicable for Eurostat: if any Member State is not transmitting all necessary data items.
R1. Data completeness - rate for P
The ratio of the number of data cells provided to the number of data cells required.
QPI: R1, Data completeness - rate for P, with different CALCULATION FORMULA for U and P
Text -
Ministry of Economic Affairs and Communications
Users’ suggestions and information about taking them into account are available on the SE website
The key users of the RPI are: Central Bank of Malta (CBM); Economic Policy Division (EPD) within the Ministry for Finance (MFIN); The public in general for queries related to rents and maintenance agreements.
The INE has carried out general user satisfaction surveys in 2007, 2010 and 2013, and it plans to continue doing so every three years. In general, users are satisfied.
Since 1996 SE conducts reputation surveys and user surveys. All results are available on the website
Applicable for Eurostat.
QPI
In some cases too generic descriptions. Guidelines require more.
In some cases too generic descriptions. Guidelines require more.

24. R1. Data completeness rate (Eurostat level)
The ratio of the number of data cells (entities to be specified by the Eurostat domain manager) provided to the number of data cells required by Eurostat or relevant. The ratio is computed for a chosen dataset and a given period.
For a specific key variable
For producers:
It refers to the set of available data cells on the required ones (i.e. excl. derogations/confidentiality)
For users:
It refers to the set of available data cells on the relevant data cells (e.g. excluding fishing fleet in Hungary) 24

25. Timeliness and punctuality
Concept
Descriptions
ESS Guidelines
Timeliness
Length of time between data availability and the event or phenomenon they describe.
Provide, for annual or more frequent releases, the average production time for each release of data and the reasons for possible long production times and efforts to improve the situation described, together with the TP1 and TP2 indicators explained for users. Applicable for Eurostat: - National data deliveries: the agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Describe the TP2 indicator for users.
TP1. Time lag - first results
The number of days (or weeks or months) from the last day of the reference period to the day of publication of first results.
QPI: TP1. Time lag - first results
TP2. Time lag - final results for P
The number of days (or weeks or months) from the last day of the reference period to the day of publication of complete and final results.
QPI: TP2. Time lag - final results for P, with DIFFERENT LEVEL OF DETAILS for U and Pa
Punctuality
Time lag between the actual delivery of the data and the target date when it should have been delivered.
Provide, for annual or more frequent releases: - The percentage of releases delivered on time, based on scheduled release dates. - The reasons for non-punctual releases explained and efforts to improve the situation described and the TP3 indicator, calculated and described for users. *National data deliveries to Eurostat: The agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Where there are several stages of publication (e.g., preliminary and final results) all should be included.
TP3. Punctuality - delivery and publication for P
The number of days between the delivery/ release date of data and the target date on which they were scheduled for delivery/ release.
QPI: TP3. Punctuality - delivery and publication for P, with different CALCULATION FORMULA for U and P

26. Timeliness and punctuality: examples
Concept
Descriptions
ESS Guidelines
Timeliness
Length of time between data availability and the event or phenomenon they describe.
Provide, for annual or more frequent releases, the average production time for each release of data and the reasons for possible long production times and efforts to improve the situation described, together with the TP1 and TP2 indicators explained for users. Applicable for Eurostat: - National data deliveries: the agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Describe the TP2 indicator for users.
TP1. Time lag - first results
The number of days (or weeks or months) from the last day of the reference period to the day of publication of first results.
QPI: TP1. Time lag - first results
TP2. Time lag - final results for P
The number of days (or weeks or months) from the last day of the reference period to the day of publication of complete and final results.
QPI: TP2. Time lag - final results for P, with DIFFERENT LEVEL OF DETAILS for U and Pa
Punctuality
Time lag between the actual delivery of the data and the target date when it should have been delivered.
Provide, for annual or more frequent releases: - The percentage of releases delivered on time, based on scheduled release dates. - The reasons for non-punctual releases explained and efforts to improve the situation described and the TP3 indicator, calculated and described for users. *National data deliveries to Eurostat: The agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Where there are several stages of publication (e.g., preliminary and final results) all should be included.
TP3. Punctuality - delivery and publication for P
The number of days between the delivery/ release date of data and the target date on which they were scheduled for delivery/ release.
QPI: TP3. Punctuality - delivery and publication for P, with different CALCULATION FORMULA for U and P
Text
The data are released 20 days upon the end of the reference month (T+20) or on the working day following it (except in February).
The results referring to year t are published in t+1+7 months.
QPI
The data have been published at the time announced in the release calendar.
Pre-announced schedules are usually observed. If the date and time of a release change; it is announced on the NSO website. Considering News Releases related to Retail Price Index, disseminated between June 2012 and June 2015, there was just one instance out of a possible 37 which has been issued after 11.10am.

27. Timeliness and punctuality: lessons learned
Concept
Descriptions
ESS Guidelines
Timeliness
Length of time between data availability and the event or phenomenon they describe.
Provide, for annual or more frequent releases, the average production time for each release of data and the reasons for possible long production times and efforts to improve the situation described, together with the TP1 and TP2 indicators explained for users. Applicable for Eurostat: - National data deliveries: the agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Describe the TP2 indicator for users.
TP1. Time lag - first results
The number of days (or weeks or months) from the last day of the reference period to the day of publication of first results.
QPI: TP1. Time lag - first results
TP2. Time lag - final results for P
The number of days (or weeks or months) from the last day of the reference period to the day of publication of complete and final results.
QPI: TP2. Time lag - final results for P, with DIFFERENT LEVEL OF DETAILS for U and Pa
Punctuality
Time lag between the actual delivery of the data and the target date when it should have been delivered.
Provide, for annual or more frequent releases: - The percentage of releases delivered on time, based on scheduled release dates. - The reasons for non-punctual releases explained and efforts to improve the situation described and the TP3 indicator, calculated and described for users. *National data deliveries to Eurostat: The agreed time frame for deliveries should be included as well as the achieved dates for deliveries during a past period. Where there are several stages of publication (e.g., preliminary and final results) all should be included.
TP3. Punctuality - delivery and publication for P
The number of days between the delivery/ release date of data and the target date on which they were scheduled for delivery/ release.
QPI: TP3. Punctuality - delivery and publication for P, with different CALCULATION FORMULA for U and P
Text
The data are released 20 days upon the end of the reference month (T+20) or on the working day following it (except in February).
The results referring to year t are published in t+1+7 months.
QPI
The data have been published at the time announced in the release calendar.
Pre-announced schedules are usually observed. If the date and time of a release change; it is announced on the NSO website. Considering News Releases related to Retail Price Index, disseminated between June 2012 and June 2015, there was just one instance out of a possible 37 which has been issued after 11.10am.
More harmonisation could be used.
More harmonisation could be used.

28. TP1. Time lag – first results
The number of days (or weeks or months) from the last day of the reference period to the day of publication of first results.
T1= dfrst - drefp
dfrst Release date of first (preliminary) results;
drefp Last day (date) of the reference period of the statistics
TP2. Time lag – final results
The number of days (or weeks or months) from the last day of the reference period to the day of publication of complete and final results.
T2= dfinl - drefp
dfinl Release date of final results 28

29. TP3. Punctuality: delivery and publication (Eurostat level)
Punctuality is the time lag between the delivery/release date of data and the target date for delivery/release as agreed for delivery or announced in an official release calendar, laid down by Regulations or previously agreed among partners.
Punctuality of data delivery (For producers)
P3 =dact - dsch
dact Actual date of the effective provision of the statistics
dsch Scheduled date of the effective provision of the statistics
The target value for this indicator is 0. 29

30. Coherence and comparability 1/2
Concept
Descriptions
ESS Guidelines
Comparability
Measurement of the impact of differences in applied statistical concepts, measurement tools and procedures where statistics are compared between geographical areas or over time. -
Comparability - geographical
The extent to which statistics are comparable between geographical areas.
Describe any problems of comparability between countries or regions. The reasons for the problems should be described and as well an assessment (preferably quantitative) of the possible effect of each reported difference on the output values should be done. Information on discrepancies from the ESS/ international concepts and definitions should be included. Differences between the statistical process and the corresponding European regulation/standard and/or international standard (if any) should be reported. Also asymmetries for statistical mirror flows should be described.
CC1. Asymmetry for mirror flows statistics - coefficient
The difference or the absolute difference of inbound and outbound flows between a pair of countries divided by the average of these two values.
QPI: CC1 Asymmetry for mirror flows statistics - coefficient
CC2. Length of comparable time series for U
The extent to which statistics are comparable or reconcilable over time.
Provide information on possible limitations in the use of data for comparisons over time. In assessing comparability over time the first step is to determine (from the metadata) the extent of the changes in the underlying statistical process that have occurred from one period to the next. (continued)
CC2. Length of comparable time series for P
The number of reference periods in time series from last break.
QPI: CC2 Length of comparable time series for P, with different LEVEL OF DETAILS for U and P

31. Coherence and comparability 1/2: examples
Concept
Descriptions
ESS Guidelines
Comparability
Measurement of the impact of differences in applied statistical concepts, measurement tools and procedures where statistics are compared between geographical areas or over time. -
Comparability - geographical
The extent to which statistics are comparable between geographical areas.
Describe any problems of comparability between countries or regions. The reasons for the problems should be described and as well an assessment (preferably quantitative) of the possible effect of each reported difference on the output values should be done. Information on discrepancies from the ESS/ international concepts and definitions should be included. Differences between the statistical process and the corresponding European regulation/standard and/or international standard (if any) should be reported. Also asymmetries for statistical mirror flows should be described.
CC1. Asymmetry for mirror flows statistics - coefficient
The difference or the absolute difference of inbound and outbound flows between a pair of countries divided by the average of these two values.
QPI: CC1 Asymmetry for mirror flows statistics - coefficient
CC2. Length of comparable time series for U
The extent to which statistics are comparable or reconcilable over time.
Provide information on possible limitations in the use of data for comparisons over time. In assessing comparability over time the first step is to determine (from the metadata) the extent of the changes in the underlying statistical process that have occurred from one period to the next. (continued)
CC2. Length of comparable time series for P
The number of reference periods in time series from last break.
QPI: CC2 Length of comparable time series for P, with different LEVEL OF DETAILS for U and P
Text -
Countries can implement different data collection methods (surveys, use of administrative data) and different calculation procedures to ensure better use of the data in the country. This may result in reduced comparability of data between the countries.
QPI
The results broken down by CNAE-09 sections and divisions are comparable over time, as of the year For the total economic activities, it must be borne in mind that, as of 2008, information has been collected on section O (division 84) of CNAE-09.
The length of the comparable time series : 14.

32. Coherence and comparability 1/2: lessons learned
Concept
Descriptions
ESS Guidelines
Comparability
Measurement of the impact of differences in applied statistical concepts, measurement tools and procedures where statistics are compared between geographical areas or over time. -
Comparability - geographical
The extent to which statistics are comparable between geographical areas.
Describe any problems of comparability between countries or regions. The reasons for the problems should be described and as well an assessment (preferably quantitative) of the possible effect of each reported difference on the output values should be done. Information on discrepancies from the ESS/ international concepts and definitions should be included. Differences between the statistical process and the corresponding European regulation/standard and/or international standard (if any) should be reported. Also asymmetries for statistical mirror flows should be described.
CC1. Asymmetry for mirror flows statistics - coefficient
The difference or the absolute difference of inbound and outbound flows between a pair of countries divided by the average of these two values.
QPI: CC1 Asymmetry for mirror flows statistics - coefficient
CC2. Length of comparable time series for U
The extent to which statistics are comparable or reconcilable over time.
Provide information on possible limitations in the use of data for comparisons over time. In assessing comparability over time the first step is to determine (from the metadata) the extent of the changes in the underlying statistical process that have occurred from one period to the next. (continued)
CC2. Length of comparable time series for P
The number of reference periods in time series from last break.
QPI: CC2 Length of comparable time series for P, with different LEVEL OF DETAILS for U and P
Text -
Countries can implement different data collection methods (surveys, use of administrative data) and different calculation procedures to ensure better use of the data in the country. This may result in reduced comparability of data between the countries.
QPI
The results broken down by CNAE-09 sections and divisions are comparable over time, as of the year For the total economic activities, it must be borne in mind that, as of 2008, information has been collected on section O (division 84) of CNAE-09.
The length of the comparable time series : 14.
More harmonisation could be used.
More harmonisation could be used.

33. CC1. Asymmetry for mirror flows statistics (Eurostat level)
Discrepancies between data related to flows, e.g. for pairs of countries.
It is computed as the difference or the absolute difference of Inbound and Outbound Flows between a pair of countries divided by the average of these two values.
Bilateral mirror statistics (A=country A, B=country B):
OFAB - outbound flow going from country A to country B
m IFAB –mirror inbound flow 33

34. CC2. Length of comparable time series (Eurostat level)
Number of reference periods in time series from last break.
Breaks in statistical time series may occur when there is a change in the definition of the parameter to be estimated (e.g. variable or population) or the methodology used for the estimation. Sometimes a break can be prevented, e.g. by linking.
The reference periods are numbered
CC1 = Jlast – Jfirst +1
Jlas tnumber of the last reference period with disseminated statistics.
Jfirst number of the first reference period with comparable statistics. 34

35. Coherence and comparability 2/2
Concept
Descriptions
ESS Guidelines
Coherence - cross domain
The extent to which statistics are reconcilable with those obtained through other data sources or statistical domains.
Describe the differences of the statistical outputs in question to other related statistical outputs (incl. main differences in concepts and definitions, statistical unit or object, classification (nomenclature) used, geographical breakdown, reference period, correction methods etc). The order of magnitude of the effects of the differences should be assessed as well. For each output the report should contain an assessment of incoherence in terms of possible sources and their impacts.
Coherence - sub annual and annual statistics
The extent to which statistics of different frequencies are reconcilable.
Coherence between subannual and annual statistical outputs is a natural expectation but the statistical processes producing them are often quite different. Compare subannual and annual estimates and, eventually, describe reasons for lack of coherence between subannual and annual statistical outputs.
Coherence- National Accounts
The extent to which statistics are reconcilable with National Accounts.
Where relevant, the results of comparisons with the National Account framework and feedback from National Accounts with respect to coherence and accuracy problems should be reported and should be a trigger for further investigation.
Coherence - internal
The extent to which statistics are consistent within a given data set.
Each set of outputs should be internally consistent: if statistical outputs within the data set in question are not consistent, any lack of coherence in the output of the statistical process itself should be stated as well as the reasons for publishing such results. For example it may occur that the process actually comprises data from different sources. In above circumstances a brief explanation should be given.

36. Coherence and comparability 2/2: examples
Concept
Descriptions
ESS Guidelines
Coherence - cross domain
The extent to which statistics are reconcilable with those obtained through other data sources or statistical domains.
Describe the differences of the statistical outputs in question to other related statistical outputs (incl. main differences in concepts and definitions, statistical unit or object, classification (nomenclature) used, geographical breakdown, reference period, correction methods etc). The order of magnitude of the effects of the differences should be assessed as well. For each output the report should contain an assessment of incoherence in terms of possible sources and their impacts.
Coherence - sub annual and annual statistics
The extent to which statistics of different frequencies are reconcilable.
Coherence between subannual and annual statistical outputs is a natural expectation but the statistical processes producing them are often quite different. Compare subannual and annual estimates and, eventually, describe reasons for lack of coherence between subannual and annual statistical outputs.
Coherence- National Accounts
The extent to which statistics are reconcilable with National Accounts.
Where relevant, the results of comparisons with the National Account framework and feedback from National Accounts with respect to coherence and accuracy problems should be reported and should be a trigger for further investigation.
Coherence - internal
The extent to which statistics are consistent within a given data set.
Each set of outputs should be internally consistent: if statistical outputs within the data set in question are not consistent, any lack of coherence in the output of the statistical process itself should be stated as well as the reasons for publishing such results. For example it may occur that the process actually comprises data from different sources. In above circumstances a brief explanation should be given.
Text
The HICP and RPI (retail Price Index) are two separate measures of inflation. The main difference between the two indices is that the HICP takes into account every Euro spent in Malta (domestic concept) whereas the national CPI takes into account every Euro spent by the Maltese in Malta and Gozo.
Fully coherent (NSA) - annual data are averages of the four quarters.
The use of the same national classification of economic activities allows for the possibility of contrasting the information with other economic statistics on common variables, such as Annual Business Surveys or National Accounts, etc.
Harmonised indices of consumer prices are internally coherent. Higher level aggregations are derived from detailed data according to pre-defined procedures.
The sum of lower-level expenditures may not always equal the corresponding higher-level expenditure, since some expenditures have been coded on a higher level.

37. Coherence and comparability 2/2: lessons learned
Concept
Descriptions
ESS Guidelines
Coherence - cross domain
The extent to which statistics are reconcilable with those obtained through other data sources or statistical domains.
Describe the differences of the statistical outputs in question to other related statistical outputs (incl. main differences in concepts and definitions, statistical unit or object, classification (nomenclature) used, geographical breakdown, reference period, correction methods etc). The order of magnitude of the effects of the differences should be assessed as well. For each output the report should contain an assessment of incoherence in terms of possible sources and their impacts.
Coherence - sub annual and annual statistics
The extent to which statistics of different frequencies are reconcilable.
Coherence between subannual and annual statistical outputs is a natural expectation but the statistical processes producing them are often quite different. Compare subannual and annual estimates and, eventually, describe reasons for lack of coherence between subannual and annual statistical outputs.
Coherence- National Accounts
The extent to which statistics are reconcilable with National Accounts.
Where relevant, the results of comparisons with the National Account framework and feedback from National Accounts with respect to coherence and accuracy problems should be reported and should be a trigger for further investigation.
Coherence - internal
The extent to which statistics are consistent within a given data set.
Each set of outputs should be internally consistent: if statistical outputs within the data set in question are not consistent, any lack of coherence in the output of the statistical process itself should be stated as well as the reasons for publishing such results. For example it may occur that the process actually comprises data from different sources. In above circumstances a brief explanation should be given.
Text
The HICP and RPI (retail Price Index) are two separate measures of inflation. The main difference between the two indices is that the HICP takes into account every Euro spent in Malta (domestic concept) whereas the national CPI takes into account every Euro spent by the Maltese in Malta and Gozo.
Fully coherent (NSA) - annual data are averages of the four quarters.
The use of the same national classification of economic activities allows for the possibility of contrasting the information with other economic statistics on common variables, such as Annual Business Surveys or National Accounts, etc.
Harmonised indices of consumer prices are internally coherent. Higher level aggregations are derived from detailed data according to pre-defined procedures.
The sum of lower-level expenditures may not always equal the corresponding higher-level expenditure, since some expenditures have been coded on a higher level.
Mostly OK.
More harmonisation could be used.
More harmonisation could be used.
More harmonisation could be used.

38. Comment Mostly not used. Concept Descriptions ESS Guidelines Comment
Supplementary descriptive text which can be attached to the data or metadata. -
Mostly not used.

39. Any questions?

40. References Eurostat quality framework Eurostat quality reporting
SIMS 2.0 – ESMS 2.0 and ESQRS 2.0 – 2015
SIMS and its Technical Manual
ESS Handbook for Quality reports 2014
ESS Quality and Performance Indicators 2014