Presentation is loading. Please wait.

Presentation is loading. Please wait.

Theory and Application of Benchmarking in Business Surveys

Similar presentations


Presentation on theme: "Theory and Application of Benchmarking in Business Surveys"— Presentation transcript:

1 Theory and Application of Benchmarking in Business Surveys
Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007

2 Content Introduction Notation Benchmarking methods Timeliness issue
Implied forecasts and annual growth rates Other uses: Seasonally adjusted data Linking problem Conclusions

3 Introduction Main references
Dagum, E.B. and Cholette, P. (2006) Benchmarking, Temporal Distribution, and Reconciliation Methods for Time Series, New York: Springer-Verlag, Lecture Notes in Statistics 186. Bloem, A. M., R. J. Dippelsman, and N. Ø. Mæhel (2001): Quarterly National Accounts Manual, Concepts, Data Sources and Compilation. International Monetary Fund, Washington DC.

4 Introduction Benchmarking :
Combining a series of high-frequency data with a series of less frequent data into a consistent time series. Monthly/Quarterly Annual Explicit information about the short-term movement Reliable information on the overall level and long-term movement The “indicator” series The “benchmarks”

5 Introduction Issues in Benchmarking :
Preserve period to period movement of the indicator (monthly/quarterly) series while simultaneously attaining the level of the benchmarks (annual). Consider the timeliness of the benchmarks.

6 Example of a quarterly series
Introduction Example of a quarterly series

7 A quarterly series with its auxiliary source
Introduction A quarterly series with its auxiliary source

8 Introduction Timeliness issue

9 Introduction Benchmarked series

10 Indicator (monthly/quarterly)
Notation Methodological details : Indicator (monthly/quarterly) Benchmarks (annual) DATA mySeries; year 4. @06 period 1. @08 value; CARDS; ; RUN; DATA myBenchmarks; startYear 4. @06 startPeriod 1. @08 endYear 4. @13 endPeriod 1. @15 value; To introduce notation and “input requirement”, i.e. coverage periods for each benchmarks.

11 Notation Methodological details :
With binding benchmarking, the benchmarked series is such that With the coverage periods for each benchmark, we can easily identify which periods t correspond to a given m. We will use a shortcut with the “t in m” notation We will only (mostly??) consider binding cases for this IOL, that is that respecting the annual benchmarks is imperative

12 Notation A bias parameter can be estimated and used to pre-adjust the indicator series: A bias corrected series is obtained as: Bias can be computed on all m (years) or only the most recent

13 Notation Alternatively, the bias can be expressed in terms of a ratio:
The bias corrected series is then: Bias can be computed on all m (years) or only the most recent

14 Notation Bias correction is a preliminary adjustment to reduce, on average, the discrepancies between the two sources of data. Useful for periods not covered by benchmarks. Bias can be computed on all m (years) or only the most recent

15 Effect of the Bias Correction (ratio)
Notation Effect of the Bias Correction (ratio) Re-scaling always the same direction even if the final discrepancies to correct may vary.

16 Methods : Pro-rating A simple way to respect the constraints is to use
This is the well-known formula for pro-rating. To explain the choice of the minimisation formulae s* ?

17 Benchmarked series with pro-rating
Methods : Pro-rating Benchmarked series with pro-rating s* ?

18 BI ratio with pro-rating
Methods : Pro-rating BI ratio with pro-rating

19 Growth rates with pro-rating
Methods : Pro-rating Growth rates with pro-rating

20 Growth rates with pro-rating
Methods : Pro-rating Growth rates with pro-rating DATE Indicator Series Benchmarked Series Growth Rate in Indicator Series (%) Growth Rate in Benchmarked Series (%) 1851 . 2436 31.60 3115 27.87 2205 -29.21 1987 -9.89 -17.90 2635 32.61 3435 30.36 2361 -31.27 2183 -7.54 -10.93

21 Methods : Proportional Denton
Benchmarked series with Prop. Denton

22 Methods : Proportional Denton
BI ratio with Prop. Denton

23 Methods : Proportional Denton
Growth rates with Prop. Denton

24 Methods : Proportional Denton
Growth rates with Prop. Denton DATE Indicator Series Benchmarked Series Growth Rate in Indicator Series (%) Growth Rate in Benchmarked Series (%) 1851 . 2436 31.60 30.86 3115 27.87 26.18 2205 -29.21 -30.86 1987 -9.89 -12.66 2635 32.61 29.04 3435 30.36 27.62 2361 -31.27 -32.14 2183 -7.54 -8.16 . 31.60 27.87 -29.21 -17.90 32.61 30.36 -31.27 -10.93 Pro-rating s* ?

25 Main method Based on Dagum and Cholette (2006).
Generalization of many well-known methods: Pro-rating Denton (and proportional Denton) Implemented at Statistics Canada with a user-defined SAS procedure: PROC BENCHMARKING Project Forillon Software Demo

26 Main method : Formula The benchmarked series can be obtained as the solution of a minimization problem. For given parameters and find the values that minimize the following function of : subject to s* to be explained… Explain choice of minimization function (later)

27 Main method : Formula Solution when : Solution: “Regression-based”
model from Dagum & Cholette For rho < 1

28 Main method : Formula Solution when : Solution:
where W is the T x M upper-right corner matrix from : For rho < 1

29 Main method : Formula We can obtain pro-rating with the general formula with and : minimise under gives How to obtain the pro-rating with the minimisation formulae.

30 Main method : Effect of Consider the case where and .
The function to be minimized under the constraints which aims at preserving the period-to-period change in the original series. Modified Denton method To motivate the choice of the minimisation formulae

31 Main method : Effect of Consider the case where and .
The function to be minimized under the constraints which seeks to minimize the change in the ratios (not to preserve the growth rates but a fairly close approx). Variant of Proportional Denton method with positive data! To motivate the choice of the minimisation formula (We suppose data is positive

32 Main method : Effect of 3 parameters at play: subject to
: model adjustment parameter : “smoothing” parameter bias (implied with ) subject to s* to be explained… Explain choice of minimization function (later)

33 Main method : Effect of Also to introduce bias…

34 Main method : Effect of bias
Benchmarking without bias ( )

35 Main method : Effect of bias
Benchmarking with bias ( )

36 Main method : Effect of bias
Benchmarking without bias ( ) Coverge to 1

37 Main method : Effect of bias
Benchmarking with bias ( ) Coverge to the bias ( Overlay the two graphs?)

38 Proportional Denton (ρ=1, λ=1)
Timeliness issues Adjustments for periods without benchmarks: Benchmarked series give an implicit forecast for the unknown annual values. The better the forecast, the lesser the revision! Proportional Denton (ρ=1, λ=1) Benchmarking with bias (ρ=0.93, λ=1)

39 Timeliness issues 2 implicit forecasts for 2006: 2004 2005 2006
Enhanced benchmarking method with explicit forecasts Year Benchmark Indicator Benchmarked (bias) (prop Denton) 2004 11,582 4.37% 11,891 1.98% 2005 11,092 -4.23% 12,399 4.27% 2006 n/a 12,196 -1.64% 11,352 2.35% 10,689 -3.64%

40 Timeliness issues One possibility for explicit forecast: 2004 2005
Use the annual growth rate from the indicator series on the last known benchmark. Year Benchmark Indicator Benchmarked (bias) (prop Denton) 2004 11,582 4.37% 11,891 1.98% 2005 11,092 -4.23% 12,399 4.27% 2006 10,910 -1.64% 12,196 11,352 2.35% 10,689 -3.64%

41 With explicit forecast ( )
Timeliness issues With explicit forecast ( )

42 With explicit forecast ( )
Timeliness issues With explicit forecast ( )

43 With ″recent″ bias( , bias=0.94)
Timeliness issues With ″recent″ bias( , bias=0.94) Bias computed with last 3 years

44 With ″recent″ bias( , bias=0.94)
Timeliness issues With ″recent″ bias( , bias=0.94) Bias computed with last 3 years

45 Timeliness issues Minimize revision? Bias Explicit forecast
(based on indicator) Will change annual growth rate of indicator series Preserve annual growth rate of indicator when nothing else is available Could be ″infected″ with non-representative historical data Annual discrepancies based only on one year Minimize revision? See Bloem

46 Methods : Summary so far!
Summary of methods presented: Pro-rating Denton (and proportional Denton) Regression-based (Dagum and Cholette) with or without bias correction Denton with explicit forecast Results from all of the above can be obtained by PROC BENCHMARKING.

47 Methods Other methods Future version of PROC benchmarking?
Other numerical methods revolve around different minimisation functions. Statistical model-based approaches See annex 6.1 in Bloem, Dippelsman, and Mæhel (2001) for variants and references See also Chen and Wu (2006) for link between numerical, regression based and signal extraction methods. Future version of PROC benchmarking?

48 Syntax : PROC Benchmarking
PLEASE SEE SOFTWARE DEMO !! PROC BENCHMARKING BENCHMARKS=myBenchmarks SERIES=mySeries OUTBENCHMARKS=outBenchmarks OUTSERIES=outSeries OUTGRAPHTABLE=outGraph RHO= LAMBDA=1 BIASOPTION=3; RUN;

49 In SAS Enterprise Guide®(Demo)
Provide input information to the procedure Mainly for training and demonstration Easy access to results in graphical format Available graphs for Benchmarking Original scale Adjustment scale ( Benchmarked to Indicator ratio) Growth rates

50 Other uses : Seasonal adjustment
Seasonally adjusted series can be required to ″match″ given annual totals : System of National Accounts (typical cases) X-12-ARIMA version 0.3+ FORCE spec (table D11 A) With argument Type=regress : same methodology as PROC BENCHMARKING

51 Other uses : Seasonal adjustment
X-12-ARIMA V0.3 Bias parameter option is replaced with argument target, which specifies which series is used as the target for forcing the totals of the seasonally adjusted series. The choices are: Original Caladjust (Calendar adjusted series) Permprioradj (Original series adjusted for permanent prior adjustment factors) Both (Original series adjusted for calendar and permanent prior adjustment factors)

52 Other uses : Seasonal adjustment
X-12-ARIMA V0.3 By default, the FORCE spec implies that the calendar year totals in the SA = calendar year totals of the target series. Alternative starting period for the annual total can be specified with start argument. Annual total starting at any other period other than start may not be equal.

53 Other uses : Seasonal adjustment
X-12-ARIMA V0.3 : example spec series{… save = A18} transform{function=log} regression{ variables=(TD easter[8])} outlier{ …} arima{…} forecast{…} x11{… save = D11} force{ type=regress lambda=1 rho=0.9 target=calendaradj save=SAA }

54 Other uses : Seasonal adjustment
Canadian Department Stores Sales SA (D11) and SA with forced annual totals (D11 A)

55 Other uses : Seasonal adjustment
Canadian Department Stores Sales

56 Other uses : Seasonal adjustment
Canadian Department Stores Sales

57 Other uses : Seasonal adjustment
Annual total starting at any other period other than start may not be equal.

58 Other uses : Linking (bridging)
Linking segments of time series with different levels or ranges. Used to minimize breaks caused by survey redesign, reclassification, change in concept… Challenges: Estimation of the potential break (parallel run, forecasting, backcasting, …) Preserve data coherence.

59 Other uses : Linking (bridging)
Can usually be achieved with PROC BENCHMARKING: If the two segments overlap (if not, use a model to extend one of the two segments) With proper identification of “anchor” points as benchmarks The smoothing parameter can gradually “bridge” the gap between the two levels

60 Other uses : Linking (bridging)
Two segments of a series

61 Other uses : Linking (bridging)
Adjusted as a level shift (λ=1, ρ=0.9, bias)

62 Other uses : Linking (bridging)
Adjusted as a level shift (λ=1, ρ=0.9, bias)

63 Other uses : Linking (bridging)
Adjusted as a gradual level shift (λ=1, ρ=0.9, no bias)

64 Other uses : Linking (bridging)
BI ratio for a gradual level shift (λ=1, ρ=0.9, no bias)

65 Conclusions Summary : Future developments in PROC BENCHMARKING
Many numerical methods can be achieved through PROC BENCHMARKING Different uses of benchmarking Future developments in PROC BENCHMARKING Simplify the use of explicit forecasts Improve bias estimation Enhance batch processing (VAR and BY statements) Include more options provided in Dagum and Cholette (2006): more generalised autocorrelation structure of the residuals, measurement errors in the input series, variance estimation of the results.

66 Susie.Fortier@Statcan.ca Benoit.Quenneville@Statcan.ca
For more information please contact Pour plus d’information veuillez contacter


Download ppt "Theory and Application of Benchmarking in Business Surveys"

Similar presentations


Ads by Google