OECD STESTWP June 2007 Towards a n omenclature of reasons for revisions

Task Force proposal Terms of Reference (V) “Define the different causes of revisions, and at what time interval (i.e. length of time after first published data) they are expected to occur. A goal would be to break the different causes of revisions into three or four homogenous groups which could then be analysed in a quantitative manner, where such decomposition is of relevance to users.”.

Some questions (1) Are all STS amenable to the same nomenclature? –suggests that an ideal nomenclature should be applicable to the revision process for all STS How broad/detailed do we want the nomenclature? i.e. what is useful? –suggests an hierarchical structure may be helpful –users can then identify a level appropriate to their purpose –this approach may require that revisions within each level of the hierarchy are additive

Some questions (1) Are all STS amenable to the same nomenclature? –suggests that an ideal nomenclature should be applicable to the revision process for all STS How broad/detailed do we want the nomenclature? i.e. what is useful? –suggests an hierarchical structure may be helpful –users can then identify a level appropriate to their purpose –this approach may require that revisions within each level of the hierarchy are additive

Some questions (2) What will it be used for? Who by? e.g. some users may whish to forecast likely future revisions, others may simply require a ‘feel’ for how close to final the current vintage is. –suggests an ideal nomenclature should encompass technical (e.g. revisions to time series filters) and non- technical causes

Some questions (3) Should the ideal nomenclature discriminate ‘updates’ from ‘revisions’? –The term ‘updates’ was introduced in the UK to delineate revisions arising from regular processes (‘updates’) from those arising from one-off causes e.g. changes to international standards or even the correction of gross errors –The importance of ‘one-off’ revisions compared with ‘routine’ or ‘expected revisions may be relevant, particularly to forecasters

Importance of timing of revisions The notion of a ‘revisions cycle’, which relates to most STS, may also be helpful (IMF refers to this as ‘revisions classified by timing’). This might range from: –revisions to very early (often partially forecast) estimates –through revisions to ‘adolescent’ estimates –onto revisions arising from incorporation of annual benchmark data, and beyond (revs to SNA etc). Do we want the nomenclature to reflect this cycle? This ‘timing’ approach to classification may provide a useful additional dimension for users to anticipate the evolution of revisions over a typical cycle.

A survey of some recent nomenclatures (1) STATS Canada (J Taillion) 1. Methodological changes (conceptual changes, change in classifications, definitions) 2. Benchmarking (the quantity of data increases) –annual surveys, tax data… –re-basing, annual chain-linking (routine recalculations) –population census 3. Evolution of data ( the quantity of data increases) data replacing forecasts higher response rates revised source data SA (routine recalculations) 4. Compilation/ balancing of methods improved 5. Alignment of accounts with conceptual targets: capitalisation of software; SNA93, ESA95

A survey of some recent nomenclatures (2) OECD (outcome of workshop Oct 2004 – as proposed by UK) A. As indicator data evolve (= increasing quantity of data) –data replacing forecasts –increased response rates –revised source data –seasonal adjustment B. As benchmark data available (= increasing quantity of data) –Annual surveys, tax data, government expenditure, etc. –re-basing - 5-yearly, annual chain-linking –population census C. As data sources improved (= one-off improvements) –new survey or admin data sources appear/developed –improved survey forms D. As compilation/balancing methods improved (= one-off) –moving to use of supply-use tables on quarterly basis E. As accounts brought in line with conceptual targets (= one-off) –SNA93, ESA95 –allocation of FISIM –capitalisation of software

A survey of some recent nomenclatures (3) UK (as used in current UK estimates of GDP(P)) 1Forecast data for proxy series replaced by actual data 2Forecast data for deflator series replaced by actual data 3Firmer actual data for proxy series received from supplier 4Firmer actual data for deflator series received from supplier 5Seasonal adjustment (from later data) 6Changes to 2-digit data quality adjustments (automatically assessed) 7Changes to 2-digit quarterly coherence adjustments (automatically assessed) 8Changes to MIDSS adjustments 9Other 10Changes to weights (automatically assessed) 11Seasonal adjustment review 12Methodological changes, ie. Industry review 13Changes to annual coherence adjustments (automatically assessed) 14Errors - Source error 15Errors - Processing error

A survey of some recent nomenclatures (4) ECB (STESWP paper June 2006) 1. New information 2. Conceptual changes (improved sources, better deflators etc.) 3. Seasonal and working day adjustment 4. Different geographical or institutional layers ((.e.g. revision to EU aggregates caused by compilation from country estimates) 5. Correction of errors

A survey of some recent nomenclatures (5) IMF (Working paper “Revisions policy for official statistics: A Matter of Governance” August 2003): 1. Incorporation of more complete or otherwise better source data: –Incorporation of source data with more complete or otherwise better reporting. –Incorporation of source data that more closely match the concepts. –Replacement of first estimates derived from judgmental or statistical techniques when data become available or as a result of benchmarking. 2. Routine recalculation: Incorporation of updated seasonal factors. Updating of the base period. 3. Improvements in methodology: Changes in statistical methods. Changes in concepts, definitions, and classifications. 4. Error correction: Correction of errors in source data and computations.

A survey of some recent nomenclatures (6) ISTAT (Working paper for the current OECD Revisions TF) 1. Current revisions 2. Periodic revisions 3. Occasional or extraordinary revisions 4. Revisions from update of time series adjustment in addition, ISTAT suggest that a triangle can be derived by adding the additional dimension of time of revision.

Summary of nomenclatures in current use (1) All are more or less exhaustive of all types of causes. Some are semi-hierarchical, others are ‘flat’. Subtle differences: –e.g. IMF class SA revisions as ‘Routine recalculation’ –Stats Canada and OECD see these as a ‘evolution of data’ (which IMF class as ‘Incorporation of more complete data’ which is broadly akin to the Stats Canada concept of ‘evolution’). UK’s nomenclature is driven by the compilation processes rather than conceptual reasons for revision

Summary of nomenclatures in current use (2) Most distinguish ‘errors’ from other causes –This suggests that that an ideal nomenclature may define ‘error’ as a ‘prime cause’, for example, an error in data and an error in processing might be classed under ‘error’ rather than ‘late data’ Most distinguish between ‘standard revisions’ arising from improved information from other (‘one- off’) causes of revision A time dimension is suggested by both the IMF and ISTAT’s ‘triangle’.

Attributes of a desirable nomenclature Based on the foregoing, it is suggested that an ideal nomenclature should have the following properties: be applicable to all STS be exhaustive of all revisions encompass a hierarchy to allow: –users to select a level of detail appropriate to their use –compilers to select one consistent with available resource identify ‘routine’ revisions separately from ‘one-off’ type revisions separate out revision arising from errors if possible, be extensible to allow time (or the revision cycle) to be added as main additional dimension of the hierarchy

A putative hierarchical nomenclature Motivation for a hierarchical approach: Users can use the level most appropriate to their needs. Statistical agencies can decide which level is appropriate to their needs and to available resources –More detail is likely to require more sophisticated and costly processes to compile regular statistics based on the nomenclature. Hierarchy can be arranged to identify technical causes from non-technical causes

A putative hierarchical nomenclature (1) The foregoing suggests a possible structure might look like (based broadly on ISTAT’s approach): Level 1 1. Routine revisions 2. Periodic revisions 3. Extraordinary revisions

A putative hierarchical nomenclature (2) Levels 2/3 (for example): 2. Periodic revisions: –2.1 Time series adjustments Calendar adjustments Working day adjustments Trend component Seasonal component Errors associated with time series adjustments –2.2 Benchmarking Incorporation of data more closely related to the concept being measured Revisions to weights (e.g. GVA weights in SNA) Errors in benchmark source of data Errors in compilation of previous benchmarking etc.

Time as an additional dimension? might the ideal hierarchy be mapped onto a classification by timing (as suggested by the IMF and ISTAT)? Perhaps as a matrix of –[time x cause] or –[stage of revisions cycle x cause] this may be useful for those who wish to anticipate future revisions as well as for general analytical purposes (e.g. identifying ways of reducing of revisions)

Next Steps seek feedback from STESWP delegates on principles outlined here seek input from other agencies, initially ECB, BEA. agree proposals within the Revisions TF wider STESWP to review TF proposals

Issues for STESWP The Revisions Task Force would welcome the views of STEWP representatives, in particular on: –the appropriateness of an hierarchical classification for revisions; –the particular outline putative hierarchy presented here; –the treatment of revisions arising from errors within the hierarchy; –the issue of whether time or the revisions cycle might be added as a further dimension in the nomenclature.