1. OECD STESWP Paris 26 June 2007 DRAFT EUROSTAT GUIDELIENS ON SEASONAL ADJUSTMENT Cristina Calizzani - Gian Luigi Mazzi

2. General Scheme 0.Seasonal Adjustment: advantages and costs 1.Pre-Treatment 2.Seasonal Adjustment 3.Revision Policies 4.Quality of Seasonal Adjustment 5.Specific issues on Seasonal Adjustment

3. Seasonal Adjustment: advantages and costs Advantages: -Provide more smoothed and understandable series for analysts -Facilitate comparisons of long/short term movements -Supply users with necessary input for BC analysis, TC decomposition and turning points detection Cautions: -SA depends on ‘a priori’ hypothesis -Quality of SA depends on quality of raw data -Lower degree of comparability of data among countries and across statistical domains if clear policies are not defined -SA data are inappropriate for econometric modelling purposes Costs: -Time consuming, significant computer/human resources required -Common and defined IT structure is needed -Inappropriate or low quality SA can give misleading results

4. 1 Pre-Treatment 1.1 Graphical analysis of the series 1.2 National and EU/Euro-area calendars 1.3 Methods for trading/working day adjustment 1.4 Correction for moving holidays 1.5 Outlier detection and correction 1.6 Decomposition model 1.7 Model selection

5. 1.1 Graphical analysis of the series  Graphical analysis of the series provides useful information (SA, parameters, problems on data)  Information on: - length of the series - presence strange values (zero, outliers..) - structure of the series (trend, seasonal component, volatility) - presence of breaks in seasonal behaviour - decomposition model More sophisticated graphs (spectrum, autocorrelogram) provide information on the presence of the seasonal component and/or trading-day effect

6. 1.1 Graphical analysis of the series  Options:  Use of basic graph in the time domain  Use of sophisticated graphs (spectrum, autocorrelograms)  Use default run of the SA software  Graphical analysis of the most important series  Evaluation of alternatives: A.Detailed graphical analysis based on graphs, autocorrelograms, spectra. The analysis could be complemented with a first explanatory run of the SA software B.First graphical analysis of the most important series (explanatory first run of the SA software) C.No graphical analysis is done

7. 1.2 National and EU/Euro-zone calendars  Can be used for working/trading-day adjustment  Availability of national calendars under DEMETRA  Tramo-Seats, TSW, X12Arima allow integrating national calendars through regressors  Construction of EU/Euro-area calendar by the ECB :  Weighted average of national calendars  Weights derived from the added value of the economic sector for which the specific calendar must be used

8. 1.2 National and EU/Euro-zone calendars  Options:  Use of default calendars  Use of national calendars or the Euro-area one as appropriate  Definition of series for which calendar adjustment is not required  Evaluation of alternatives: A. B.Use of default calendars (within the tool without reference to national/euro area specificities) C.No calendar correction despite diagnostic evidence (in this case the ARIMA modelling of the series will be affected) Consequences on decomposition may be important European aggregates (Direct appraoch)EU/Euro-zone calendars Indirect geographical adjustmentNational calendars

9. 1.3 Methods for trading/working day adjustment  Removal of all effects related to calendar effects  Length of the month  Number of working days per month  Composition of working days in terms of number of Monday, Tuesday, etc…  February length is not constant over years (leap year effect)  Working/trading-days effects as a source of non linearity of time series  Obtaining series with single-point values independent on calendar  Non Seasonally adjusted  Seasonally adjusted

10. 1.3 Methods for trading/working day adjustment  Options  No correction  Proportional methods on the bases of the number of days in the month  Regression methods in a multivariate regression framework -With or without correction for the length of the month or leap year -Identification of the most appropriate number of regressors  RegArima correction (as before but with ARIMA residuals)  Evaluation of alternatives A.RegArima approach (in case of economic rationale for the TDE) - All pre-test for number of regressors, length and composition of month -Checking for plausibility of effects B.Regression-based approach C.Purely proportional methods, other adjustments or no adjustment

11. 1.4 Correction for moving holidays  Moving holidays occur irregularly in the course of the year  Absence of periodicity  Not removed by standard seasonal filters  Examples of moving holidays: Easter, Pentecost, Ramadan  Moving holidays effect typically defined on a time varying span  Among months  Among quarters  When non negligible, such effects should be estimated and removed  Aim: obtaining a seasonally adjusted series whose single-point values are independent of particular calendar effects which occur irregularly across years

12. 1.4 Correction for moving holidays  Options  No correction  Correction within proportional adjustment  Automatic correction  Correction based on an estimation of the duration of the moving holidays effects  Evaluation of alternatives A.RegArima approach -pre-test for Easter, other moving holidays effect -Definition of the length of Easter effect on the basis of results of pre- tests -Checking for plausibility of effects B.Regression-based approach C.No tests/correction despite diagnostic evidence of such effects

13. 1.5 Outlier detection and correction  Outliers are abnormal values of the series  Classification of main outliers  Impulse outliers: abnormal values in isolated points of the series  Transitory changes: series of innovation outliers with transitory effects on the level of the series  Level shifts: series of innovation outliers with constant and permanent effect on the level of the series  Presence of outliers affects model identification and seasonal adjustment  Outliers have to be removed and reintroduced after having estimated calendar/seasonal component

14. 1.5 Outlier detection and correction  Options  Types of outliers to be considered for pre-testing  Removal or not of outliers before seasonal adjustment  Including or not important outliers in the regression model as intervention variables  Evaluation of alternatives A.The series should be checked for different outliers - Remove outliers due to data errors - Adjust out of the series other outliers before seasonal adjustment/ re-introduce outliers after estimation calendar/seasonal component - Outliers should be explained using all available information Outliers with a clear interpretation (severe strikes, changes in government policy..) are included as regressors in the model B.As before, but complete automatic procedure according to available tools C.No preliminary treatment of outliers

15. 1.6 Decomposition model  How the various components (TC, S, I) combine to form the original series  Default decomposition model is multiplicative in most economic time series  Tramo-Seats only proposes additive/log-additive decomposition X-12-RegARIMA adds a pseudo-additive and a multiplicative decomposition  For series with trends in mean and in variance the log-additive decomposition seems to be the most appropriate one; When only trend in mean is present, the multiplicative decomposition is recommended  For series with zero or negative values, the additive decomposition is automatically selected by SA procedures  The choice of the decomposition model and the differencing orders aim to achieve a stationary series  These 2 decisions impact forecasts, model-based seasonal adjustments and trend-cycle estimates

16. 1.6 Decomposition model  Options  Automatic decomposition model selection  Manual decomposition scheme after graphical inspection  For series with zero or negative values adding a constant and select the most appropriate scheme  For second order weak-stationary series additive decomposition  Evaluation of alternatives A.Automatic decomposition model selection using appropriate information criteria after graphical inspection of the series; Special treatment for non positive series (adding a constant and checking the impact on the seasonally adjusted series); Manual selection for more problematic series A.Fully automatic decomposition model using information criteria B.Use of a fixed decomposition model (multiplicative for positive series, additive for non positive series)

17. 1.7 Model selection  Model selection:  Criteria to select the appropriate model for pre-adjustment, seasonal adjustment, forecast extension fro SA  Log versus non-log specification of the model  Order of differencing for seasonal and non seasonal part  Use of additive or multiplicative components  Statistical checking of the adequacy of the estimated model  Analysis of decomposition on the basis of the chosen model  …  Different relevance for model based methods or non parametric ones

18. 1.7 Model selection  Options  Automatic model selection  Model selection based on a set of predefined models  Manual model selection  Evaluation of alternatives A.Automatic selection within a large number of models: additive/multiplicative, extended order Arima models,... -check for model adequacy using standard statistical tests (normality, heteroskedasticity, serial correlation, …) and spectrum diagnostics -use of manual model selection for most important / more problematic series B.As before, but complete automatic procedure C.Selection based on a restricted number of pre-defined models

19. 2 Seasonal Adjustment 2.1 Choice of SA approach 2.2 Consistency between raw and SA data 2.3 Geographical aggregation: direct versus indirect approach 2.4 Sectoral aggregation: direct versus indirect approach 2.5 Data presentation issues

20. 2.1 Choice of seasonal adjustment approach  Most commonly used seasonal adjustment packages  Tramo-Seats  X12ARIMA  Tramo-Seats: model-based approach based on Arima decomposition techniques  X12ARIMA: non parametric approach based on a set of linear filters (moving averages)  X13-AS: new package containing Tramo-Seats and X12- RegARIMA  Univariate or multivariate structural time series models

21. 2.1 Choice of seasonal adjustment approach  Options  X12ARIMA  Tramo-Seats or TSW  X13-AS  Structural time series models  Evaluation of alternatives A.Tramo-Seats and X12ARIMA / X13-AS -Choice on the basis of past experiences, subjective appreciation, characteristics of the series, etc. -Production tools updated on a regular basis after a sufficiently long testing period B.Structural time series models based on simultaneous representation of the unobserved components of the series within software that can estimate calendar and outliers effects with diagnostics for all components and effects C.Other production tools

22. 2.2 Consistency between raw and SA data  Aggregation of SA data coincide with aggregation of NSA data over the year  Cumulative seasonality is zero over the year  No trading day or calendar effects  Unrealistic assumption especially in the case of multiplicative models  Strong requirements from many users  Quarterly National Accounts  Balance of Payments  No theoretical justification for this constraint  Consequence: bias on seasonally adjusted data

23. 2.2 Consistency between raw and SA data  Options  Do not apply any constraint  Apply default constraining techniques (X12ARIMA)  Constrain seasonally data to sum up to original data  Constrain seasonally adjusted data to sum to trading day ONLY adjusted data  Evaluation of alternatives A.Do not impose that the sum (average) of raw and SA/calendar adjusted data should coincide B.Consistency between raw/working days adjusted and seasonally/working days adjusted data can be accepted under particular circumstances, i.e. requirements from users. benchmarking methods should be used C.Impose consistency between seasonally/calendar adjusted data and raw data

24. 2.3 Geographical aggregation: direct versus indirect approach  Performing SA at different geographocal aggregation levels:  SA either by NSI or Eurostat on national series with same method and software and European totals derived by aggregating SA national figures (decentralised or centralised indirect approach)  SA directly on European total obtained by aggregation of national not SA/WDA (direct approach)  Each series is SA with different methods and software and the SA European totals derived as aggregation of the SA components (Mixed indirect approach)  National and European figures are independently SA and aggregation constrains imposed ex-post  Neither theoretical nor empirical evidence in favour of one of the approaches  Often strong requirements of consistency between MS data and European figures, especially when additive data (QNA, External Trade, Empl/Unemploiment)

25. 2.3 Geographical aggregation: direct versus indirect approach  Options  Indirect approach: SA national components series in a centralized/decentralized way with the same software and then derive totals by aggregation of SA components  Mixed indirect approach: SA of national data at NSI level using different approaches and software  Direct approach: aggregation of NSA data and SA of the aggregates  Direct approach with distribution of discrepancies -Use of benchmarking techniques

26. 2.3 Geographical aggregation: direct vs indirect approach  Evaluation of alternatives A.Direct approach for transparency reasons and in case of lack of harmonization of national approaches; Centralized indirect approach is also recommended when subsidiarity principle does apply (external trade, unemployment); B.Decentralized indirect approach within a common quality assessment framework (where there is a strong user requirement of geographical consistency between national and European aggregates i.e. additivity) Check for the presence of any residual seasonality in the indirectly adjusted European aggregates C.Mixed indirect approach

27. 2.4 Sectoral aggregation: direct vs indirect approach Different sectoral aggregation levels  Indirect approach: SA sectoral components with same methods and software and totals derived by aggregating SA sectoral figures  Direct approach: aggregation of raw data and SA of the aggregates  Mixed indirect approach: each sectoral series is SA with different methods and software and the SA totals derived by aggregation of SA components  Aggregated and disaggregated figures are independently SA and aggregation constraints imposed ex-post  For several statistical domains there is often a strong requirement of consistency at various level of classification (QNA, Balance of payments, External trade, Employment)

28. 2.4 Sectoral aggregation: direct vs indirect approach  Evaluation of alternatives A.Both direct or indirect approaches either in case of direct adjustment for the geographical aggregation or no previous geographical aggregation. Choice based on: - characteristics of data (correlation of single components, quality of basic data, etc.) - users' requests (e.g. consistency of components and aggregates) Otherwise only indirect adjustment B.Direct approach at any level with benchmarking techniques to obtain consistency of disaggregated and aggregated estimates (if the adjustments arising from benchmarking have adequate quality) C.Use of a mixed indirect approach

29. 2.5 Data presentation issues  SA versus trend-cycle data  Main difference: presence of irregular component  SA data often considered more informative  univariate and multivariate analysis  Trend-cycle data usually preferred for high volatile series and graphical representations  Choice of growth rates to show in press releases  Standard versus annualised  Period on period versus annual

30. 2.5 Data presentation issues  Options  Include only raw data in press releases  Extend the informative content of press releases with one or more of the following transformations: SA series, SA plus WDA series, Trend-cycle series  Present only levels or different kinds of growth rates

31. 2.5 Data presentation issues  Evaluation of alternatives A.Avoid the presentation of trend-cycle data in press releases -Always raw and SA data -raw data, SA and trend-cycle series should be available to users through Eurostat website -Trend-cycle data can be used in press releases for high volatile series (include only graphs) -“period on period” growth rates have to be computed on SA data -annual growth rates have to be computed on NON SA data -avoid annualised growth rates B.Present only seasonally adjusted data C.Presentation of trend-cycle data only; computation of yearly growth rates on SA data

32. 3 Revisions Policies 3.1General revision policy 3.2Timing of revision: Tramo-Seats approach 3.3Timing of revision: X12Arima approach

33. 3.1 3.1 General revision policy  SA data usually revised  revision of raw data  Improved information set  Better estimates of the seasonal pattern Revision are welcomed because they derive from improved information set In SA one or more observation can lead to revise several past observations Trade off between precision in SA data and stability of seasonal adjustment pattern  Revision should be scheduled in a regular way  No revision should take place between two consecutive releases

34. 3.1 3.1 General revision policy  Options  Revise SA data according to the release calendar of the unadjusted data  Revise SA data whenever revisions of raw data occur or at least once a year  Revise SA data more often than scheduled raw data releases  Evaluation of alternatives A) Every time a new data is added, internal check against the best possible seasonally adjusted data. If results computed with forecasted factors differ perceptibly for the best possible adjusted data, then seasonal factors have to be recalculated and SA data revise back in time. Revise only last 3-4 years B) Revise SA data once a year or whenever revisions of raw data occur. It is recommended to freeze data as in A); C) Scheduling revision only once a year or even at lower frequency without any a priori evaluation, as well as revising data more often than scheduled raw data releases

35. 3.2 Timing of revision: Tramo-Seats approach  Important to define the timing for the re- identification and re-estimation of Arima models  Normally, the re-estimation of parameters is carried out more frequently than the re- identification re-identification of models  Re-identification takes place usually once per year or even less frequently.

36. 3.2 Timing of revision: Tramo-Seats approach  Options  To re-identify and re-estimate models once a year;  To re-identify models once a year and re-estimate parameters each time seasonal adjustment is performed;  To specify other timing between re-identification and re- estimation.  Evaluation of alternatives A) Models are re-identified once per year and parameters are re- estimated every time seasonal adjustment is performed according to the release calendar of raw data (concurrent adjustment). To restrict revisions only to the last three/four years and to freeze previous seasonally adjusted data unless major revisions on raw data occur; B) To re-identify and re-estimate models once a year or whenever unexpected events and/or revisions on raw data occur. It is recommended to freeze data as described in A); C) No re-identification/estimation of models or different timing from those under A) and B), as well as revising data more often than scheduled raw data releases

37. 3.2 Timing of revision: X12Arima approach  For the current year seasonally adjusted data can be computed:  by running every month/quarter the seasonal adjustment procedure (data are revised every month/quarter)  by using extrapolated coefficients computed once a year (data are not revised within the year but only once a year)  The first approach increases data accuracy  The second one is preferred by users which do not like continuous revision  The use of extrapolated seasonal factors can lead to biased results when unexpected events occur  Revisions should be scheduled in a regular way and possibly according to the raw data release calendar  No revision should take place between two consecutive releases.

38. 3.2 Timing of revision: X12Arima approach  Options  Concurrent adjustment;  Concurrent adjustment with only the preceding month and same month of the previous year revised until December adjustment, when all past adjustment values are updated. This could avoid complaints from some users  Use of extrapolated seasonal factors.  Evaluation of alternatives  A) Re-estimate seasonal factors according to the release calendar of raw data (concurrent adjustment). It is recommended to restrict revisions only to the last three/four years and to freeze previous seasonally adjusted data unless major revisions on raw data occur;  B) Concurrent adjustment with updating of only a few past values until December or use of extrapolated seasonal factors with the possibility of modifying them whenever unexpected events and/or revisions in raw data take place. It is recommended to freeze data as in A)  C) Use of purely extrapolated seasonal factors as well as revising seasonal factors more often than scheduled raw data releases

39. 4 Quality of Seasonal Adjustment 4.1 Validation of seasonal adjustment 4.2 Common quality measures for seasonal adjustment 4.3 Eurostat Quality Report for seasonal adjustment 4.4 Template for seasonal adjustment metadata

40. 4.1 Validation of seasonal adjustment Accurate monitoring needed before acceptance results Availability of a wide range of quality measures  Absence of residual seasonality  Absence of residual calendar effects  Stability of the seasonal adjusted pattern Validation by means of:  Graphical analysis  Descriptive statistics  Non parametric criteria  Parametric criteria

41. 4.1 Validation of seasonal adjustment  Options  Use an integrate set of graphical, descriptive, non parametric/parametric criteria to check the suitable characteristics of SA data;  Restrict validation to the use of standard measures proposed by different seasonal adjustment tools;  Use only graphical inspection and descriptive statistics to validate the seasonal adjustment  Evaluation of alternatives A) Use an integrate set of graphical, descriptive, non parametric and parametric criteria to validate the seasonal adjustment and run again the SA with a different set of options in case of non acceptance of results. Particular attention to: -absence residual seasonality/calendar effects - absence over-smoothing - absence autocorrelation of the irregular component - stability of the seasonal component B) Use only default criteria defined within different tools to validate the results and run again the seasonal adjustment as in alternative A) if validation fails; C) No validation of performed seasonal adjustment

42. 4.2 Common quality measures for SA  Specific quality measures developed for Tramo-Seats and X12  Reflecting, at least partially, their different philosophy  Possibility of extending X12 measures to Tramo-Seats and vice versa  Not all quality measures can be generalised  Eurostat contribution to define common quality measures  X13-AS as an ideal framework for a set of common quality measures

43. 4.2 Common quality measures for SA  Options  To use specific quality measures for each approach  To use common diagnostics for both approaches  Evaluation of alternatives A.Use of common measures/diagnostics for the analysis of the quality of seasonal adjustment performed with different tools B.Use of standard quality measures/diagnostics provided by tools C.No use of quality measures/diagnostics to evaluate seasonal adjustment

44. 4.3 Eurostat Quality Report for SA  Development of a Eurostat quality report for SA  In the context of a general quality assessment of infra- annual statistics  Eurostat quality report defined both for massive SA treatment and small scale analysis  Short version  Detailed version  Improvement to the quality report needed in the light of recent developments on SA tools and methods

45. 4.3 Eurostat Quality Report for SA  Options  Use Eurostat quality report  Identify further improvement to Eurostat quality report  Use only the quality measures available in standard tools for seasonal adjustment  Evaluation of alternatives A.Use an improved (to be finalised) version of Eurostat quality report B.Use the existing Eurostat quality report C.No use of any quality framework for the evaluation of seasonal adjustment

46. 4.4 Template for seasonal adjustment metadata  Clarity of SA: appropriate harmonized standard documentation  Special Data Dissemination Standard format (SDDS) of IMF  Development of a SA template for metadata  ECB-Eurostat Task Force on Quarterly National Accounts  Improvement of such template according to recent developments in the field of metadata and SA  SDMX

47. 4.4 Template for seasonal adjustment metadata  Options  Use the existing standard template for SA metadata  Improve the standard template for SA metadata  Include SA information into the general SDDS files  Evaluation of alternatives A.Use of standard (or improved) template for SA metadata B.Include SA information into the general SDDS files C.No methodological information supplied for SA

48. 5 Specific issues on seasonal Adjustment 5 Specific issues on seasonal Adjustment 5.1 Seasonal adjustment of short time series 5.2 Treatment of problematic series

49. 5.1 Seasonal adjustment of short time series 5.1 Seasonal adjustment of short time series  Impossibility of performing standard SA on too short series  Use of non standard software  Do not perform any SA  Stability and reliability problems of Tramo-Seats and X12 when series are long enough to use the tools, but shorter than 10 years  Several empirical studies analysing the behaviour of Tramo- Seats and X12 on short time series  Adopt a transparent policy to inform users about all problems related to the SA treatment of short time series

50. 5.1 Seasonal adjustment of short time series 5.1 Seasonal adjustment of short time series  Options  No adjustment of series shorter than the minimum requirement for Tramo-Seats and X12  Use of alternative procedures to SA of short time series  Re-specify all parameters involved in pre-treatment and seasonal adjustment more often than in the standard case  Comparative studies on the relative performance of Tramo- Seats and X12 for series shorter than 7-10 years  Inform users about instability problems for series shorter than 7-10 years

51. 5.1 Seasonal adjustment of short time series 5.1 Seasonal adjustment of short time series  Evaluation of alternatives A.Use standard tools whenever possible -Extension of the sample and stabilisation of SA using non official back-recalculated time series -Simulations on relative performances of the existing standard tools for short series SA -Inform users on the greater instability of SA data and on used methods -Clear publication policy -Tuning of parameters checked more than once per year B.SA not performed on too short series C.Use of non standard tools on short time series

52. 5.2 Treatment of problematic series  Strange features in time series  Impossible to find model with acceptable diagnostics  Absence of a clear signal due to the presence of a dominant irregular component  Unstable seasonality  Large number of outliers  Heteroskedasticity in the series/components  Impossibility of a standard treatment for such series  Ad hoc treatment -Software -Options  Quality of SA problematic series  appropriateness of the adopted strategy

53. 5.2 Treatment of problematic series  Options  Seasonally adjust only recent years of the series  Perform ad hoc SA on all problematic series  Perform ad hoc SA only on relevant problematic series  No ad hoc SA  Evaluation of alternatives A.SA is performed for problematic series -Prefer a case by case approach to a standard one -Inform users on the adopted strategy -Consult literature/manual/experts B.Perform SA only on relevant problematic series and treat other problematic series in a standard way C.Automatic SA for all series

54. Thanks for your attention!