ESSnet on the use of administrative and accounts data in business statistics Development of Quality Indicators (WP6) John-Mark Frost (ONS, UK), Humberto Pereira (INE, PT), Sofia Rodrigues (INE, PT), Ana Chumbau (INE, PT), Jorge Mendes (INE, PT) and Sarah Green (ONS, UK), Q2010, 4 th May 2010

Overview Work package (WP) partners Aims of the WP Why this work is important Work done so far Next steps

WP partners

Aims of the WP (2009 – 2013) “To collect and analyse information on existing methods used in NSIs for quality assessment when administrative data are used. To develop quantitative quality indicator(s) for business statistics produced using administrative data, and To develop qualitative indicators to complement the quantitative one(s).”

Why this work is important … Increasing use of administrative data in business statistics Dimensions of quality apply but … - Quality reporting is not entirely the same - CVs cannot be used when solely using administrative data Best practice in other areas will depend on appropriate and effective measures of quality

Work done so far … Focussed on: “collecting and analysing information on existing methods used in NSIs for quality assessment when administrative data are used.”

Phase 1: Methodology Developed Questionnaire -Use of administrative data in business statistics -Use of quality checks Circulated to 34 NSIs: -27 Member States -4 EFTA -3 Non-European 90+% response rate

Phase 1: Results (1) Administrative data used extensively in business statistics Quality considered an important issue Lots of generic checks conducted during the process of statistics production

Phase 1: Results (2) But … … checks are not necessarily formal and very few are published as quality indicators … only half the NSIs indicated that they produce any kind of quantitative quality measure

Phase 2: Methodology Identified 16 NSIs that showed the most experience in the area of quality indicators Sent more extensive questionnaires, specifically asking about quantitative quality indicators in the areas of SBS, STS, Business Registers and Prodcom 100% response rate

Phase 2: Results Consistent with Phase 1: -NSIs check quality -Checks are generally made as part of the statistical production process -But … the checks are not necessarily produced on a regular or formal basis.

Phase 3: Methodology Identified 7 ‘more experienced’ NSIs and requested to meet with them Engaged in face-to-face interviews with relevant staff within the NSIs to: - Better understand their use of administrative data in business statistics -Gain clarity on their responses in Phase 2 -Get more detailed information on their use of quality indicators

Phase 3: Results (1) NSIs engaged in similar quality checks: -Accuracy -e.g. % of units with correct activity code -Coverage -e.g. comparison of units included in administrative source with units in the BR to estimate under / over- coverage -Missing data/non-response, -e.g. % of turnover accumulated at publication of first estimate

Phase 3: Results (2) -Revisions -e.g. differences between first and final estimates -Matching (more relevant for NSIs without unique identifiers) -e.g. % of matched units from both sources -Coefficients of Variation (when combining administrative and survey data) -e.g. using the jack-knife method However, very few of these checks were produced as formal, quality indicators

Summary of work so far … Administrative data are widely used in business statistics Quality is seen as important Various checks are conducted during the statistical production process but … -they are not necessarily formal or regular -they are rarely published On the whole, NSIs do not produce quality indicators in the same way as when using survey data Development of list of quality indicators welcomed by NSIs

The next steps … Build on the results of the stock-take research Further develop the list of quality indicators -including user testing Investigate composite quality indicators Throughout, ensure that we: -adopt a pragmatic approach -develop a user-friendly list of indicators -consider the limitations on NSIs (resource and data availability)

Some areas for consideration Indicators that apply to both survey and administrative data Not all NSIs will have access to the same level or type of information (either administrative or process related) Qualitative as well as quantitative indicators

Thank you for listening …