HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS Pedro Abad Banco de España European Conference on Quality in Official Statistics Viena, 2-5/06/2014

STATISTICS DEPARTMENT 2 HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS  Introduction. Main elements of the Statistics Quality Framework: towards high-quality output  Quality monitoring and reporting on financial accounts  Quantitative quality indicators  Output enhancement from the ECB quality monitoring reports: the experience of Banco de España  Conclusions

STATISTICS DEPARTMENT 3 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT  The ECB statistical system rests on three pillars: the regulations setting obligations on economic agents the guidelines given to NCBs detailing the datasets to be transmitted and other statistical requirements, and the quality framework  The ESCB Public Commitment on European Statistics set the principles that govern the collection, compilation and dissemination of statistics by the ECB.  The ECB Statistics Quality Framework combines these principles and others of good governance with the quality assurance procedure developed by the ECB.  Quality reports are at the core of the system as they review main quality elements and draw some conclusions/recommendations in order to enhance statistics.

STATISTICS DEPARTMENT 4 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT Quality is defined in relation to how well statistics are fit for purpose, and, therefore, meet users´ and other stakeholders´ expectations. This is a multidimensional concept. Guiding principles from the Public Commitment on European Statistics by the ESCB Institutional environmentStatistical processesHigh output quality 1 Professional independence 7 Sound methodology11 Relevance 2 Mandate for data collection 8 Appropriate statistical procedures 12 Accuracy and reliability (including stability) 3 Adequacy of resources 9 Minimisation of the reporting burden 13 Timeliness (including punctuality) 4 Commitment to quality10 Cost-effectiveness14 Consistency and comparability 5 Statistical confidentiality 15 Accessibility and clarity 6 Impartiality and objectivity

STATISTICS DEPARTMENT 5 1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT Data Quality Framework: High Output Quality PrinciplesQuality assurance proceduresPrinciplesQuality assurance procedures Relevance  Medium-term strategy / annual work programmes  Merit and cost exercises  Seminars and consultations with users Consistency and comparability  Reporting schemes follow international accounting standards and best practices  Regular internal quality monitoring reports Accuracy and reliability (including stability)  Reporting schemes follow international accounting standards and best practices  Completeness and plausibility checks  Revision analysis and consistency analysis between different datasets  Regular internal quality monitoring reports Accessibility and clarity  Intensive use of up-to-date electronic system for disseminating data  Press releases, regular publications  Data warehouse, joint country- tables  International standards in the denomination of variables  On-line documentation Timeliness (including punctuality)  Transmission calendars  Release calendars  Compliance with IMF SDDS standards

STATISTICS DEPARTMENT 6 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS – QUALITY REPORTS  The main two procedures implemented by the SQF to report on quality to the ECB senior management are: The audit reports prepared by the Internal Auditors Committee (IAC) Quality reports prepared by the Statistics Committee (STC)  Quality reports prepared by the STC review main quality elements with the help of some usually pre-defined indicators and draw some conclusions and/or recommendations in order to enhance the quality of statistics  Several ECB statistical guidelines require the preparation of quality reports, mainly focused on the compilation of euro aggregates: Balance of Payments and International Investment Position Statistics (biannual), Government Finance Statistics (internal), Monetary and Financial Statistics, Centralised Securities Database (internal) Financial Accounts.  All these reports are based on a common framework, although they are tailored to each specific statistical domain and so their content is not fully homogenous.  The preparation of these reports by the ECB with the assistance of the expert groups can be seen as the final step of the quality monitoring process, which also includes other internal quality reports.

STATISTICS DEPARTMENT 7 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS  ECB compiles euro area financial accounts on the basis of: national financial accounts transmitted under Guideline ECB/2013/24, and datasets already available in the ECB data warehouse from other ECB guidelines. The most important are: banking statistics, securities statistics, government finance, and external statistics.  ECB Guideline requires the transmission of: stocks, transactions, other flows and counterpart information for selected items information on major events and revisions the preparation of an annual quality report including at least: coverage, compliance with relevant definitions and revisions  The quality monitoring process carried out by the ECB with the assistance of the expert group on financial accounts also includes several internal quality reports.

STATISTICS DEPARTMENT 8 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS Table 2. Coverage of quality monitoring reports on financial accounts PeriodicityOrientation Production CollectionCompilationDissemination Internal quality monitoring reports Feedback country report QuarterlyCompilers X Regular transmission report (EA level, information by country) QuarterlyCompilers XX Integration report (EA level) Quarterly Compilers/ internal users X Methodological notes (EA level) Ad-hoc Compilers/ internal users X Quality report (EA level, public) Annual Internal users/ public XXX

STATISTICS DEPARTMENT 9 2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS  It is inspired by the basic principles of the SQF  Public (since 2013)  Coverage: Institutional and legal framework Quality assurance procedures and user involvement Coverage Revision analysis Consistency and comparability Timeliness of national contributions Data accessibility Recent developments NCB-specific assessment, recommendations and non- compliance cases (only internal version)  Comprises a set of quantitative indicators at EA level EURO AREA QUARTERLY FINANCIAL ACCOUNTS REPORT

STATISTICS DEPARTMENT QUANTITATIVE QUALITY INDICATORS Revision analysis  Errors or revisions are defined as the difference between the current values of a variable (as transactions) and the values when they were released for the first time. The quality report presents four indicators to assess accuracy and reliability. Positive revisions (R) as % of positive revisions in relation to total observations Directional reliability (Q) as % of first release of data with the same sign as the last release MACE [Mean Absolute Comparable Errors] which is the mean of absolute errors on transactions in relation to stocks, and RMSRE [Root Mean Square Relative Errors] which is a relative measure of errors in relation to the variability series Consistency analysis  Statistical discrepancies arise in the compilation process due to the integration of different data sources. The quality report assesses discrepancies in financial instruments (horizontal inconsistencies), and between the capital and the financial accounts (vertical inconsistencies) by means of the RMSE indicator. RMSE [Root Mean Square Errors]

STATISTICS DEPARTMENT QUANTITATIVE QUALITY INDICATORS Euro area quality reliability indicators for households

STATISTICS DEPARTMENT OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA  The BdE contribution is fully compliant with the ECB data requirements  The BdE takes note of the conclusions and recommendations to enhance, as far as possible, the Spanish financial accounts and therefore the national contributions to the ECB  In recent years, changes have been introduced in several areas following quality monitoring reports: to reduce inconsistencies with securities issues statistics due to the treatment of own securities used as collateral at the ECB to reduce inconsistencies with money and banking statistics due to the treatment of factoring operations related to general government transactions to harmonise the treatment of repos transactions across euro area countries other discrepancies between data sources are regularly monitored  Discrepancies between net lending/borrowing and net financial transactions of non-financial corporations and households are monitored by the BdE and the INE

STATISTICS DEPARTMENT OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA Spain. Discrepancies between net lending/borrowing and net financial transactions

STATISTICS DEPARTMENT CONCLUSIONS  Monitoring and reporting on quality is an essential element of the Statistics Quality Framework.  Several ECB Guidelines require the preparation of annual quality reports, which to some extent are the final step of the quality monitoring process carried out by the ECB with the assistance of the expert groups.  By means of ECB quality reports the Statistics Committee can draw the attention of the highest decision-making bodies of the ECB to relevant quality problems on euro area statistics, and related national contributions, so that the ECB and national central banks can resolve underlying problems.  The quality report on financial accounts is an example of this strategy. This and other internal quality monitoring reports cover the compliance by countries with the ECB regulations and other quality aspects, in particular revisions and consistency with other statistics, which are examined on the basis of a set of quantitative indicators.  Finally, the paper provides some information about the changes introduced in the Spanish financial accounts following the conclusions and recommendations of the ECB quality reports.

STATISTICS DEPARTMENT THANK YOU FOR YOUR ATTENTION Pedro Abad