1. Prodcom ESTP course 20 - 22 October 2010
Data Quality
Prodcom ESTP course October 2010
20-22 October 2010

2. Main problems in getting accurate Prodcom data
Accuracy in the general statistical sense denotes the closeness of computations or estimates to the exact or true values.
It is impossible to know whether the Prodcom data is accurate.
The main problems which prevent us to asses the exact Prodcom data accuracy are:
20-22 October 2010
Publication of the Prodcom Figures

3. 1. The coverage  90% rule (1/2)
impossible to assess
no statistical registry contains information on commodities but on activity
only rough estimation based on turnover
it does not guarantee that the biggest enterprises by turnover actually also cover 90% of the industrial activities
20-22 October 2010
Publication of the Prodcom Figures

4. 1. The coverage  90% rule (2/2)
Reporting countries can do their best to identify all the enterprises producing a product, but
they may miss some,
some may fail to respond,
the enterprise may report inaccurate data,
enterprises not covered, such as those with fewer than 20 employees, may make up a significant part of the national production,
the national survey may be based on a sample requiring grossing-up techniques to arrive at the national total.
20-22 October 2010
Publication of the Prodcom Figures

5. 2. Measurement errors
A measurement error by the respondent may concern:
wrong industrial product codes;
wrong volume unit or reporting a volume unit inconsistent with the Prodcom list;
which heading has to reported (sold and/or total);
reporting of work under subcontracting;
making a clear distinction between production and industrial services (i.e. production activity or assembly activity).
The measurement errors are impossible to assess but the reporting countries do their best to minimise them.
20-22 October 2010
Publication of the Prodcom Figures

6. Quality checks made by Eurostat
Member States perform a lot of checks at micro and macro level, at different aggregation levels, between historical data and actual data and with other sources (i.e. business surveys, STS and SBS especially)
Eurostat has the advantage of being able to compare the data across countries. It contributes to further improvement on data quality.
Quality checks made by Eurostat do not discover errors! They only detect possible anomalies.
anomalies need to be reported back to the reporting country; it is up to them either to give an explanation for the anomaly, (e.g. a big factory closed down) or to confirm that it is in fact an error, and hopefully provide a correction.
20-22 October 2010
Publication of the Prodcom Figures

7. 1. Validation checks consistency with the Prodcom List
completeness of Prodcom headings,
production type,
measurement unit and so on
correctness of formatting
If some validations checks fail, the data might be ignored and not used in EU aggregates. So, validation is a first gate which might drop some data if they are considered inconsistent or wrong. The reporting countries are informed about errors and asked to correct them.
20-22 October 2010
Publication of the Prodcom Figures

8. 2. Plausibility checks (1/3)
After the validation of reporting countries’ inputs, the following plausibility checks are applied:
The value is zero but the volume is non-zero (or vice-versa)
The value and volume are zero but they are marked confidential
The number of enterprises is blank or zero but production is reported
Visual checks. For instance:
production of olive oil in a Nordic country;
production of ships in a land-locked country;
no dairy production in a country, even though there are a lot of cows in the fields
20-22 October 2010
Publication of the Prodcom Figures

9. 2. Plausibility checks (2/3)
Compliance Score (1/2)
Once the validation and plausibility checks are done, the compliance score is computed. Eurostat applies a combined compliance score for both punctuality and completeness of the data.
The formula is
m = the percentage of missing data
e = the percentage of data which were estimated by the country
d = the number of days late.
20-22 October 2010
Publication of the Prodcom Figures

10. 2. Plausibility checks (3/3)
Compliance Score (2/2)
Countries with a score of
20 or less are assessed as VG (Very Good),
as G (Good),
as P (Partially compliant) and
over 100 as N (Not compliant).
The most favourable compliance score is used.
20-22 October 2010
Publication of the Prodcom Figures

11. 3. Comparisons across time
The evolution of values or volumes across years is hard to evaluate because there may be a lot of natural variation.
But if production is fairly stable over several years and then changes significantly, there may be a problem (or a large factory opening or closing).
Such comparisons are useful both for detecting big changes from one year to another, but also for showing the differences between successive revisions of the same year's data.
20-22 October 2010
Publication of the Prodcom Figures

12. 4. Comparisons across countries
Unit value
For 2002 data onwards the unit values for each country have been calculated and compared with those for other countries.
To achieve this, Eurostat calculates the ratio of each country's unit value to the median unit value of all countries for each product.
Whichever was larger of the country/median ratio and the median/country ratio is taken so that a large distance from the median is always represented by a high value.
This is the most useful automated check. Compared with values or volumes alone, the unit value should be much more consistent over time and between countries, so it is worth detecting outliers. Even then, there can be valid explanations for anomalies.
20-22 October 2010
Publication of the Prodcom Figures

13. 5. Comparisons across time and countries
EU Aggregates
Prioritise the detected anomalies based on their impact on EU aggregates
Eurostat use its advantage to detect possible anomalies and to signal those with the biggest impact on EU aggregates. In this way Eurostat avoids to put a too much pressure/burden on reporting countries by asking them to check all possible detected anomalies.
20-22 October 2010
Publication of the Prodcom Figures

14. 5. Comparisons across time and countries
If production at EU level is fairly stable over several years and then changes significantly, there may be a problem.
For those EU aggregates where big changes in production are observed, Eurostat analyses the production and unit vales of concerned reporting countries across time.
If there are multiple indices which indicate possible anomalies, they are reported back to the reporting country for explanations or correction.
20-22 October 2010
Publication of the Prodcom Figures

15. 5. Comparisons across time and countries
Following such analysis the possible anomalies might concern previous years. In this case, depending on the revision policy set up by each Member State, the reporting country might or not revise their previous data.
Anyway, the reporting countries may submit revisions at any time, so Eurostat also republishes the data in the middle of any month when new data has been received since the last publication. In practice this means that data is republished in most months, sometimes even for earlier years.
Conventions used in charts (examples): the thickest mark and line shows the reporting country where a possible anomaly is detected.
20-22 October 2010
Publication of the Prodcom Figures

16. 6. Other checks
"Manual" checks can often be more effective. The Eurostat Prodcom team is capable of identifying some anomalies, based on general knowledge.
But it is useful to encourage users to give feedback. In particular the FEBIs (trade associations) have a detailed knowledge of their own sector and can recognise implausible production levels. Such feedback should also be reported back to the Member State concerned; it often results in corrections to the data.
20-22 October 2010
Publication of the Prodcom Figures

17. Thank you for your attention
20-22 October 2010
Publication of the Prodcom Figures