1. ESTP Course
Balance of Payments – Introductory course Paris, May 2014 Quality issues

2. L14 - Quality issues Assessing the quality of the BOP
1. What is quality ? What are the challenges for external statistics compilers?
2. Illustration : the Data Quality Framework in Europe (EU and Euro Area)

3. What is quality? Institutional environment Independence accountability
Mandate for data collection
Impartiality
Confidentiality
Coordination and cooperation
Resources and efficiency
Serviceability (Dissemination, service to users …)
Statistical processes Statistical output
Institutional environment 4

4. What is quality? Statistical processes Sound methodology
Appropriate statistical procedures
Cost effectiveness
Minimisation of burden for reporting agents

5. What is quality? Statistical output Relevance Accuracy, reliability
Consistency, comparability
Timeliness
Accessibility, clarity

6. Challenges for compilers
A moving environment
Context of liberalisation of the market for financial services and the need to develop alternative sources of information
From an International Transaction Reporting System - ITRS (or “settlement based reporting” system linked) to a “mixed” reporting data collection system in a context of reduced resources and limitation to statistical burden on respondents. From 2000 onwards, new data collection and compilation systems: estimates, sample surveys
In EU/Euro areas, regional BoP statistics with the implementation of common DQMs and common tools (such as Centralised Security database, Foreign direct investment network…)

7. Challenges for compilers
An increasing role played by BoP and External statistics, quality is crucial …
Following the financial crisis, new requirements from the “statistics’ users” for a better granularity and quality (G20 – Financial stability board, IMF, BIS, OECD …) in order to detect imbalances or measure interconnections
Increasing number of indicators are based on BoP, IIP and external statistics: assessing macroeconomic imbalances in the EU for instance, European dashboard implemented …
BoP usually are compiled as aggregates of a variety of primary statistics : sample survey, information derived from administrative data or resulting from models, estimates

8. Challenges for compilers
The legal framework
Council regulation, European Treaty (ESCB/ECB statute) and guidelines, ESS Handbook for Quality Report
Memorandum of Understanding in 2003 between the ESCB and Eurostat
Definition of and obligations on reporting population
Data requirements and methodology
Protection of confidentiality
Quality assurance
Convergence with BPM6 and SNA2008: concepts, data transmission, release calendar, common data requests, common revision policy and practice …

9. The Data Quality Framework in Europe (EU and Euro Area)
European Union (EU BoP)
 Mainly focused on goods, services and direct investments items
 Annual Quality report (public document)
 Annual country quality report
 Main items :
Coverage and availability
Accuracy: stability (mean values of revisions)
Timeliness and punctuality
Accessibility and clarity
Comparability (over space and over time)
Coherence: internal (E&O) and external consistencies (FTS)

10. The Data Quality Framework in Europe (EU and Euro Area)
Illustration of the quality challenge: Macroeconomic Imbalance Procedure (MIP) (EU Regulation 1176/2011 on the prevention and correction of macroeconomic imbalances)
 A surveillance mechanism that aims to identify potential risks, prevent the emergence of macroeconomic imbalances and correct the imbalances
 Annual starting point is the Alert Mechanism Report based on a scoreboard of indicators to identify countries and issues for which a closer analysis (in-depth review) is deemed necessary.

11. The Data Quality Framework in Europe (EU and Euro Area)
External statistics are used as primary data for 4 of the 11 indicators
 A preventive and a corrective arm. The latter is made operational by the Excessive Imbalance Procedure: possible sanctions for Member States if they repeatedly fail to meet their obligations.
 Data quality management is therefore a crucial point for the credibility of the procedure. The European Statistics System (ESS) and the European System of Central Banks continue to work together on improving the underlying statistics and to ensure comparability => Task force on quality

12. The Data Quality Framework in Europe (EU and Euro Area)
Euro Area (BoP and IIP)
Compilation and publication of euro area BoP and IIP
Quality assessment mainly focused on financial transactions and positions
Bi-annual BoP and IIP Quality Report (public document)
Annual Quality report on country contributions (for the information of the Governing Council of the ECB)

13. The Data Quality Framework in Europe (EU and Euro Area)
Euro Area (BoP and IIP)
Main items of the country contributions quality report :
Transmission compliance (timeliness, completeness, validity of data …)
Methodological issues: recent developments in countries, deviation from the standards
Stability of the data: revision of current and financial accounts
3. Illustration : the Data Quality Framework in Europe (Euro Area)

14. European Union (EU BoP) 11 Differences between first and final estimates for EU-27 current account (M€)

15. European Union (EU BoP) Goods: difference between Foreign Trade Statistics (FTS) and Balance of Payments statistics (EU-27 and extra EU-27 partners) (in M€)

16. The Data Quality Framework in Europe (EU and Euro Area)
Euro Area (BoP and IIP)
Main items of the country contributions quality report :
Internal consistency: net E&O in national BoP
External consistency:
-With MFI balance sheet data
-With foreign trade statistics

17. Issues concerning data collection systems
Main characteristics of data collection system (surveys, international transactions reporting system)?
Mains concerns or “gaps” in data collection system?
What is the approximate share of estimated data? Objective in terms of coverage?
Data quality management:
 Nature and content of your control process (tools, frequency …)?
 How to detect a “bad quality” individual reporting?
 Regular and direct relationship with the reporting institutions?
 Correction of data collected?

18. Measure of quality of BOP data
Control process and tools used in this respect?
Internal consistency (within the dataset):
 Specific monitoring regarding size/volatility of errors and omissions? on which counterpart sector and instrument?
 Main concerns to ensure consistency over time?
 External consistency (across datasets):
 Cross checking with other external sector datasets (international merchandise trade, reserve assets, external debt, IIP statistics) or other macroeconomic statistics (monetary and financial statistics)?
 Process of reducing errors and omissions in close cooperation with national accountants?
 Usefulness of “mirror” data such as bilateral data, BIS data, IMF surveys (CPIS, CDIS …)? for which items?

19. Revision policy
“Stability” of data over time as an indicator of data quality?
Usefulness of limits concerning the size of revisions?
Use of relative measures of revisions (gross or net transactions / balances between assets and liabilities)?
Publication of revision policy and analysis?