Presentation is loading. Please wait.

Presentation is loading. Please wait.

Theory and Application of Benchmarking in Business Surveys Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007.

Similar presentations


Presentation on theme: "Theory and Application of Benchmarking in Business Surveys Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007."— Presentation transcript:

1 Theory and Application of Benchmarking in Business Surveys Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007

2 2 Content Introduction Notation Benchmarking methods Timeliness issue Implied forecasts and annual growth rates Other uses: Seasonally adjusted data Linking problem Conclusions

3 3 Introduction Main references Dagum, E.B. and Cholette, P. (2006) Benchmarking, Temporal Distribution, and Reconciliation Methods for Time Series, New York: Springer-Verlag, Lecture Notes in Statistics 186. Bloem, A. M., R. J. Dippelsman, and N. Ø. Mæhel (2001): Quarterly National Accounts Manual, Concepts, Data Sources and Compilation. International Monetary Fund, Washington DC.

4 4 Introduction Benchmarking : Combining a series of high-frequency data with a series of less frequent data into a consistent time series. Monthly/Quarterly Annual Explicit information about the short-term movement Reliable information on the overall level and long-term movement The “indicator” seriesThe “benchmarks”

5 5 Introduction Issues in Benchmarking : Preserve period to period movement of the indicator (monthly/quarterly) series while simultaneously attaining the level of the benchmarks (annual). Consider the timeliness of the benchmarks.

6 6 Introduction Example of a quarterly series

7 7 Introduction A quarterly series with its auxiliary source

8 8 Introduction Timeliness issue

9 9 Introduction Benchmarked series

10 10 Notation Methodological details : Indicator (monthly/quarterly) Benchmarks (annual) DATA mySeries; INPUT @01 year 4. @06 period 1. @08 value; CARDS; 2000 1 1851 2000 2 2436 2000 3 3115 2000 4 2205 2001 1 1987 … ; RUN; DATA myBenchmarks; INPUT @01 startYear 4. @06 startPeriod 1. @08 endYear 4. @13 endPeriod 1. @15 value; CARDS; 2000 1 2000 4 10324 2001 1 2001 4 10200 … ; RUN;

11 11 Notation Methodological details : With binding benchmarking, the benchmarked series is such that

12 12 Notation A bias parameter can be estimated and used to pre-adjust the indicator series: A bias corrected series is obtained as:

13 13 Notation Alternatively, the bias can be expressed in terms of a ratio: The bias corrected series is then:

14 14 Notation Bias correction is a preliminary adjustment to reduce, on average, the discrepancies between the two sources of data. Useful for periods not covered by benchmarks.

15 15 Notation Effect of the Bias Correction (ratio)

16 16 Methods : Pro-rating A simple way to respect the constraints is to use This is the well-known formula for pro-rating.

17 17 Methods : Pro-rating Benchmarked series with pro-rating

18 18 Methods : Pro-rating BI ratio with pro-rating

19 19 Methods : Pro-rating Growth rates with pro-rating

20 20 Methods : Pro-rating Growth rates with pro-rating DATE Indicator Series Benchmarked Series Growth Rate in Indicator Series (%) Growth Rate in Benchmarked Series (%) 2000-0118511989.15.. 2000-0224362617.8131.60 2000-0331153347.4827.87 2000-0422052369.57-29.21 2001-0119871945.42-9.89-17.90 2001-0226352579.8632.61 2001-0334353363.1230.36 2001-0423612311.60-31.27 2002-0121832059.05-7.54-10.93

21 21 Methods : Proportional Denton Benchmarked series with Prop. Denton

22 22 Methods : Proportional Denton BI ratio with Prop. Denton

23 23 Methods : Proportional Denton Growth rates with Prop. Denton

24 24 Methods : Proportional Denton Growth rates with Prop. Denton DATE Indicator Series Benchmarked Series Growth Rate in Indicator Series (%) Growth Rate in Benchmarked Series (%) 2000-0118511989.15.. 2000-0224362617.8131.6030.86 2000-0331153347.4827.8726.18 2000-0422052369.57-29.21-30.86 2001-0119871945.42-9.89-12.66 2001-0226352579.8632.6129.04 2001-0334353363.1230.3627.62 2001-0423612311.60-31.27-32.14 2002-0121832059.05-7.54-8.16. 31.60 27.87 -29.21 -17.90 32.61 30.36 -31.27 -10.93 Pro-rating

25 25 Based on Dagum and Cholette (2006). Generalization of many well-known methods: Pro-rating Denton (and proportional Denton) Implemented at Statistics Canada with a user- defined SAS procedure: PROC BENCHMARKING Forillon Project Forillon Software Demo Main method

26 26 Main method : Formula The benchmarked series can be obtained as the solution of a minimization problem. For given parameters and find the values that minimize the following function of : subject to

27 27 Solution when : Solution: Main method : Formula “Regression-based” model from Dagum & Cholette

28 28 Solution when : Solution: where W is the T x M upper-right corner matrix from : Main method : Formula

29 29 Main method : Formula We can obtain pro-rating with the general formula with and : minimise under gives

30 30 Main method : Effect of Consider the case where and. The function to be minimized under the constraints which aims at preserving the period-to-period change in the original series. Modified Denton Modified Denton method

31 31 Main method : Effect of Consider the case where and. The function to be minimized under the constraints Proportional Denton which seeks to minimize the change in the ratios (not to preserve the growth rates but a fairly close approx). Variant of Proportional Denton method with positive data!

32 32 Main method : Effect of 3 parameters at play:  : model adjustment parameter  : “smoothing” parameter  bias (implied with ) subject to

33 33 Main method : Effect of

34 34 Main method : Effect of bias Benchmarking without bias ( )

35 35 Main method : Effect of bias Benchmarking with bias ( )

36 36 Main method : Effect of bias Benchmarking without bias ( )

37 37 Main method : Effect of bias Benchmarking with bias ( )

38 38 Timeliness issues Adjustments for periods without benchmarks: Benchmarked series give an implicit forecast for the unknown annual values. The better the forecast, the lesser the revision! Proportional Denton (ρ=1, λ=1)Benchmarking with bias (ρ=0.9 3, λ=1)

39 39 Timeliness issues 2 implicit forecasts for 2006: explicit Enhanced benchmarking method with explicit forecasts YearBenchmarkIndicatorBenchmarked (bias) Benchmarked (prop Denton) 2004 11,582 4.37% 11,891 1.98% 11,582 4.37% 11,582 4.37% 2005 11,092 -4.23% 12,399 4.27% 11,092 -4.23% 11,092 -4.23% 2006 n/a 12,196 -1.64% 11,352 2.35% 10,689 -3.64%

40 40 Timeliness issues One possibility for explicit forecast: Use the annual growth rate from the indicator series on the last known benchmark. YearBenchmarkIndicatorBenchmarked (bias) Benchmarked (prop Denton) 2004 11,582 4.37% 11,891 1.98% 11,582 4.37% 11,582 4.37% 2005 11,092 -4.23% 12,399 4.27% 11,092 -4.23% 11,092 -4.23% 2006 10,910 -1.64% 12,196 -1.64% 11,352 2.35% 10,689 -3.64%

41 41 Timeliness issues With explicit forecast ( )

42 42 Timeliness issues With explicit forecast ( )

43 43 Timeliness issues With ″recent″ bias(, bias=0.94)

44 44 Timeliness issues With ″recent″ bias(, bias=0.94)

45 45 Timeliness issues Minimize revision? Bias Explicit forecast (based on indicator) Will change annual growth rate of indicator series Preserve annual growth rate of indicator when nothing else is available Could be ″infected″ with non-representative historical data Annual discrepancies based only on one year

46 46 Methods : Summary so far! Summary of methods presented: Pro-rating Denton (and proportional Denton) Regression-based (Dagum and Cholette) with or without bias correction Denton with explicit forecast Results from all of the above can be obtained by PROC BENCHMARKING.

47 47 Methods Other methods Other numerical methods revolve around different minimisation functions. Statistical model-based approaches See annex 6.1 in Bloem, Dippelsman, and Mæhel (2001) for variants and references See also Chen and Wu (2006) for link between numerical, regression based and signal extraction methods. Future version of PROC benchmarking ?

48 48 Syntax : PROC Benchmarking PLEASE SEE SOFTWARE DEMO !! PROC BENCHMARKING BENCHMARKS=myBenchmarks SERIES=mySeries OUTBENCHMARKS=outBenchmarks OUTSERIES=outSeries OUTGRAPHTABLE=outGraph RHO=0.729 LAMBDA=1BIASOPTION=3; RUN;

49 49 In SAS Enterprise Guide® (Demo)

50 50 Other uses : S easonal adjustment Seasonally adjusted series can be required to ″match″ given annual totals : System of National Accounts (typical cases) X-12-ARIMA version 0.3+ FORCE spec (table D11 A ) With argument Type=regress : same methodology as PROC BENCHMARKING

51 51 Other uses : S easonal adjustment X-12-ARIMA V0.3 Bias parameter option is replaced with argument target, which specifies which series is used as the target for forcing the totals of the seasonally adjusted series. The choices are: Original Caladjust (Calendar adjusted series) Permprioradj (Original series adjusted for permanent prior adjustment factors) Both (Original series adjusted for calendar and permanent prior adjustment factors)

52 52 X-12-ARIMA V0.3 By default, the FORCE spec implies that the calendar year totals in the SA = calendar year totals of the target series. Alternative starting period for the annual total can be specified with start argument. Annual total starting at any other period other than start may not be equal. Other uses : S easonal adjustment

53 53 Other uses : S easonal adjustment X-12-ARIMA V0.3 : example spec series{… save = A18} transform{function=log} regression{ variables=(TD easter[8])} outlier{ …} arima{…} forecast{…} x11{… save = D11} force{ type=regress lambda=1 rho=0.9 target=calendaradj save=SAA }

54 54 Other uses : S easonal adjustment Canadian Department Stores Sales SA (D11) and SA with forced annual totals (D11 A)

55 55 Other uses : S easonal adjustment Canadian Department Stores Sales

56 56 Other uses : S easonal adjustment Canadian Department Stores Sales

57 57 Other uses : S easonal adjustment Annual total starting at any other period other than start may not be equal.

58 58 Other uses : Linking (bridging) Linking segments of time series with different levels or ranges. Used to minimize breaks caused by survey redesign, reclassification, change in concept… Challenges: Estimation of the potential break (parallel run, forecasting, backcasting, …) Preserve data coherence.

59 59 Other uses : Linking (bridging) Can usually be achieved with PROC BENCHMARKING: If the two segments overlap (if not, use a model to extend one of the two segments) With proper identification of “anchor” points as benchmarks The smoothing parameter can gradually “bridge” the gap between the two levels

60 60 Other uses : Linking (bridging) Two segments of a series

61 61 Other uses : Linking (bridging) Adjusted as a level shift (λ=1, ρ=0.9, bias)

62 62 Other uses : Linking (bridging) Adjusted as a level shift (λ=1, ρ=0.9, bias)

63 63 Other uses : Linking (bridging) Adjusted as a gradual level shift (λ=1, ρ=0.9, no bias)

64 64 Other uses : Linking (bridging) BI ratio for a gradual level shift (λ=1, ρ=0.9, no bias)

65 65 Conclusions Summary : Many numerical methods can be achieved through PROC BENCHMARKING Different uses of benchmarking Future developments in PROC BENCHMARKING Simplify the use of explicit forecasts Improve bias estimation Enhance batch processing (VAR and BY statements) Include more options provided in Dagum and Cholette (2006): more generalised autocorrelation structure of the residuals, measurement errors in the input series, variance estimation of the results.

66 66 For more information please contact Pour plus d’information veuillez contacter Susie.Fortier@Statcan.ca Benoit.Quenneville@Statcan.ca


Download ppt "Theory and Application of Benchmarking in Business Surveys Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007."

Similar presentations


Ads by Google