1. Quality Criteria
Near Final

2. Establishing the Basics
ESS recommend quality reports should comment on 9 areas:
Relevance
Accuracy
Timeliness
Accessibility & clarity
Coherence & comparability
Trade off between output quality components
Assessment of user needs and perceptions
Performance, cost and respondent burden
Confidentiality, transparency and security

3. Scope
We’re focusing on quality of the flash estimate, not the quality of the GDP estimate
We’re also looking at setting the quality criteria that need to be met before first publication of a EU28 and EA18 flash estimate
For this reason the horizontal working group have focused on:
Accuracy
Timeliness
At the June 2014 task force meeting it was also suggested we add a communication plan criteria

4. Near Final Paper
A near final paper has been presented to the task force
This paper was agreed in principal and presented to the wider National Accounts Working Group in May 2015
NAWG did not suggest any particular change to the paper or criteria
As such the quality group would like to present the paper and four sign off criteria to the task force for final or almost final approval

5. Criterion 1 – Bias Criteria
The aim of this criterion is to test whether the flash estimate provides an unbiased estimate of what T+45 growth will be:
T+30 should be an unbiased estimate of GDP growth at T+45, with an average revision between and 0.05, and no more than 66.7% of revisions in the same direction
This is probably the most desirable property of a ‘flash’ estimate of a statistic
Our main difficulty will be testing this with only 16 quarters of data available (8 real, 8 retrospective).
Statistical tests were ruled out due to the limited number of quarters being available (see Annex A)

6. Criterion 2 – Scale/Size
This criterion compares the size of revision between T+30 & T+45, and T+30 & T+65
The average absolute revision for T+30 GDP growth should be within 0.10 ppt compared with T+45 GDP growth estimate, and 0.13 ppt compared with T+65 GDP growth estimate.
As with criteria one, using statistical tests was ruled out due to insufficient number of data points
When setting this criteria consideration was given to the revisions performance of the US Flash estimate (presented in paper)
There was also discussion around the fact that the EU28 estimate is a developing statistic and that one could assume revisions may improve as members states method evolve

7. Criterion 3 – Timeliness
This criterion is designed to ensure enough member states can deliver to deadline consistently to produce quality results at the EU28 and EA18 level
For the final two quarters of 2015 Eurostat needs to receive 70% of last year’s EU Member States GDP by the T+29 deadline
This criteria is set to safeguard against the publication of a T+30 flash before it has been proven the timeliness can be adhered to on a consistent basis
In addition to this criterion there should also be no known reasons why the criteria wouldn’t be met for each quarter of 2016 and beyond.

8. Criterion 4 – Communication
This criterion is designed to ensure users are well informed about the estimate and it’s strengths and weaknesses
Key milestones of the communication plan should be met or plans in place to meet before publication of the T+30 flash estimate
This criteria is set to safeguard against the publication of a T+30 flash before users understand how it should be interpreted
Outstanding Work
Linked to the communication working group

9. Decision Time
There has been a large amount of discussion around quality criteria since the second working group meeting
Various issues have been raised and considered when developing the 4 criteria
The objective for the quality working group was to propose quality acceptance criteria that would be used to inform a ‘go/no go’ decision for publication of a T+30 flash estimate
With the task force’s approval these four criteria would become the ‘go/no go’ quality criteria

10. Questions?