1 Appendix B: A Primer of Time Series Forecasting Models B.1 A Primer of Time Series Forecasting Models

2 The Universal Time Series Model TREND SEASONAL ERROR(Irregular) TRANSFORMATION

3 Additive Decomposition of the Airline Data T: Linear Trend S: Seasonal Average I: Irregular Component

4 Types of Models Stationary Only Trend and Stationary Seasonal and Stationary Trend, Seasonal, and Stationary

5 Exponential Smoothing Models (ESM) Stationary Only –Simple Exponential Smoothing (one parameter) Trend and Stationary –Simple Exponential Smoothing (one parameter) –Linear (Holt) Exponential Smoothing (two parameters) –Damped-Trend Exponential Smoothing (three parameters) continued...

6 Exponential Smoothing Models (ESM) Seasonal and Stationary –Seasonal Exponential Smoothing (two parameters) (Both additive and multiplicative types are supported.) Trend, Seasonal, and Stationary –Holt-Winters Additive (three parameters) –Holt-Winters Multiplicative (three parameters)

7 Exponential Smoothing Premise Weighted averages of past values can produce good forecasts of the future. The weights should emphasize the most recent data. Forecasting should require only a few parameters. Forecast equations should be simple and easy to implement.

8 ESM as Weighted Averages Weights Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8 Weights applied to past values to predict Y 9 Y7Y7 Y8Y8 Sample Mean Random Walk

9 ESM as Weighted Averages Weights Sample Mean The mean is a weighted average where all weights are the same. Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8

10 ESM as Weighted Averages Random Walk A random walk forecast is a weighted average where all weights are 0 except the most recent, which is 1. Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8

11 The Exponential Smoothing Coefficient Forecast Equation

12 Simple Exponential Smoothing As the parameter grows larger, the most recent values are emphasized more. Weights Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8 Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5 Y6Y6 Y7Y7 Y8Y8 Weights applied to past values to predict Y 9

13 ESM for Seasonal Data Weights decay with respect to the seasonal factor. Weights Jan 00 Jan 01 Jan 02 Jan 03 Jan 04 Feb 00 Feb 01 Feb 02 Feb 03 Feb 04 …

14 ESM Seasonal Factors First seasonal factor s 1 is always “natural.” First season: January, Monday, Q1 Additive model: factors average to 0 Multiplicative model: factors average to 1 Monthly Seasonal Factors

15 Smoothing Weights The choice of a Greek letter is arbitrary. The software uses names rather than Greek symbols. Level smoothing weight Trend smoothing weight Seasonal smoothing weight Trend damping weight

16 ESM Parameters and Keywords ESMParametersName in Repository Simple  simple Double  double Linear (Holt) ,  linear Damped-Trend , ,  damptrend Seasonal ,  seasonal Additive Winters , ,  addwinters Multiplicative Winters , ,  winters

17 Box-Jenkins ARIMAX Models ARIMAX: AutoRegressive Integrated Moving Average with eXogenous variables. AR: Autoregressive  Time series is a function of its own past. MA: Moving Average  Time series is a function of past shocks (deviations, innovations, errors, and so on). I: Integrated  Differencing provides stochastic trend and seasonal components, so forecasting requires integration (undifferencing). X: Exogenous  Time series is influenced by external factors. (These input variables can actually be endogenous or exogenous.)

18 Box-Jenkins ARMA Models Theory: Given a stationary time series, there exists an ARMA model that approximates the true model arbitrarily closely  universal approximator. Reality: Given a stationary time series, there is no guarantee that you can find the best ARMA approximator. Theory: Apply differencing operators until what remains is a stationary time series. Reality: Differencing might not be the best way to model trend and seasonality. After differencing, the time series could still be nonstationary.

19 Box-Jenkins Forecasting Myths Myth: Box and Jenkins invented ARIMA models. Fact: Box and Jenkins brought together existing theory and added some of their results, and thus popularized the use of ARIMA models. Myth: Box-Jenkins forecasting only works for stationary time series. Fact: Box-Jenkins forecasting provides a general methodology for forecasting any time series. ARIMA models are nonstationary models that can be decomposed into the usual trend, seasonal, and stationary components.

20 Historical Impediments to Box-Jenkins Modeling History Models are sophisticated and require training and experience to use them successfully. Modelers are prone to overfitting the data, which leads to poor forecasts. Software is unavailable, unreliable, or too slow for forecasting many time series. Today Techniques exist for automatic model selection. Honest assessment techniques prevent overfitting. Modern software is reliable and fast.

21 ARIMA Model Specification ARIMA(p, d, q)(P, D, Q) p indicates a simple autoregressive order. P indicates a seasonal autoregressive order.

22 AR(1): The Toothpaste Series

23 ARIMA Model Specification ARIMA(p, d, q)(P, D, Q) d indicates a simple difference of the series. D indicates a seasonal difference.

24 ARIMA (1, 1, 0)(0, 0, 0): The Crocs Series

25 ARIMA Model Specification ARIMA(p, d, q)(P, D, Q) q indicates a simple moving average order. Q indicates a seasonal moving average order.

26 ARIMA (0, 0, 1)(0, 1, 0): The Pork Bellies Series Summer Peaks; “BLT effect”

27 Types of ARIMA Models Stationary Only Trend and Stationary Seasonal and Stationary Trend, Seasonal, and Stationary ARIMAX models accommodate exogenous variables.

28 Intermittent Demand Models (IDM) Intermittent time series have a large number of values that are zero. These types of series commonly occur in Internet, inventory, sales, and other data where the demand for a particular item is intermittent. Typically, when the value of the series associated with a particular time period is nonzero, demand occurs. When the value is zero (or missing), no demand occurs. Source: SAS ® 9 Online Help and Documentation

29 Intermittent Demand Data Time Demand Mostly Zeros

30 Intermittent Demand Models (IDM) Demand Time Size Interval

31 Intermittent Demand Models (IDM) Demand Size Demand Interval Index Average Demand=Demand Size divided by Demand Interval

32 Two IDM Choices Croston’s Method = Two smoothing models –The Interval component is fit with an ESM. –The Size component is fit with an ESM. –The forecast of Average Demand is Forecast Size/Forecast Interval. Average Demand Method = One smoothing model –Average demand is calculated directly from the data and forecast with an ESM.

33 Unobserved Components Models (UCMs) Unobserved components models are also called structural models in the time series literature. A UCM decomposes the response series into components such as trend, seasonals, cycles, and the regression effects due to predictor series. The components in the model are supposed to capture the salient features of the series that are useful in explaining and predicting its behavior. Source: SAS ® 9 Online Help and Documentation

34 Unobserved Components Models (UCMs) also known as structural time series models decomposed time series into four components: –trend –season –cycle –Irregular General form: Y t = Trend + Season + Cycle + Regressors

35 UCMs Each component captures some important feature of the series dynamics. Components in the model have their own models. Each component has its own source of error. The coefficients for trend, season, and cycle are dynamic. The coefficients are testable. Each component has its own forecasts.

36 Types of UCM Models Stationary Only Trend and Stationary Seasonal and Stationary Trend, Seasonal, and Stationary UCM models accommodate exogenous variables.

37 Types of Models UCM StatementModel Types irregular Stationary Only (White Noise) level, slope, irregular Trend and Stationary season (or cycle ), irregular Seasonal and Stationary all statements Trend, Seasonal, and Stationary

38 Specifying UCMs Unobserved components models are available through the HPFDIAGNOSE and HPFUCMSPEC procedures. The syntax used by these procedures is similar to that used by the UCM procedure in SAS/ETS software.

39 Which Model Type? Performance: time required to derive coefficients and create forecasts Accuracy Usability: ease of going from data to forecasts and interpreting results

40 Performance Best to worst: 1.ESM 2.IDM 3.ARIMAX 4.UCM

41 Accuracy Best to Worst: 1.ARIMAX, UCM 2.ESM Intermittent Demand - Best to Worst: 1.IDM (when appropriate). 2.Others can be used, but they generally provide unacceptable accuracy.

42 Usability Best to worst: 1.ESM 2.UCM 3.ARIMAX

43 Mean Absolute Percent Error (MAPE) Absolute percent error for one time point: 100%  |Actual-Forecast|/Actual MAPE is one of the most common accuracy measures in business forecasting. As a selection criterion, choose the model with the smallest value of MAPE. Interpretationthe size of forecast error relative to the magnitude of the actual value Mean absolute percent error the average of all of the individual absolute percent errors

44 Mean Absolute Error (MAE) Absolute error for one time point: |Actual-Forecast| MAE is not commonly used as an accuracy measure in business forecasting. As a selection criterion, choose the model with the smallest value of MAE. Interpretationthe size of the forecast error Mean absolute error the average of all of the individual absolute errors

45 MAPE versus MAE Holiday Sales Day Low Sales Day Actual1, Forecast APE10%33.3% AE100 Mean 21.65% 100 An error of 100 on a large sales day is usually not as serious as an error of 100 on a low sales day, but MAE weights both equally. MAPE MAE

46 Root Mean Square Error (RMSE) Squared error for one time point: (Actual-Forecast) 2 RMSE is commonly used as an accuracy measure in industrial, economic, and scientific forecasting. As a selection criterion, choose the model with the smallest value of RMSE. InterpretationThe squared size of the forecast error Mean squared error (MSE) The average of all of the individual squared errors, adjusted for the number of estimated model parameters Root mean square error The square root of MSE

47 Classes of Models Exponential smoothing models ARIMAX models UCM models Simple regression models –are predefined trend components: linear, quadratic, cubic, log-linear, exponential, and so on –are predefined seasonal dummies –include a combination of one or more simple predefined components Simple models –the mean –a random walk –a random walk with drift

48 Performance Simple models have no performance issues. Exponential smoothing models can be constructed quickly and easily, so they always have good performance. ARIMAX models require many more computer cycles than simple or exponential smoothing models, but are based on algorithms that were refined over the past 30 years. Thus, creating a custom fit ARIMAX model is feasible even for large numbers of series. UCM models are very computer intensive and should be tried only on small data sets or individual time series.

49 Forecasting with SAS Forecast Studio Functionality: Only automatically generated and custom ARIMAX or UCM models accommodate event, input, and outlier (exogenous) variables. Pre-existing ESM models and ARIMA models (for example, those shipped in the default catalog) do not accommodate exogenous variables. Automatically generated ARIMAX models can select best combinations of exogenous variables for each series diagnosed (identified). Custom, user-defined ARIMAX models must be specified to explicitly accommodate exogenous variables.

50 Static Linear Regression with Two Variables Y =  0 +  1 X 1 +  2 X 2 +  Y is the target (response/dependent) variable. X 1 and X 2 are input (predictor/independent) variables.  is the error term.  0,  1, and  2 are parameters.  0 is the intercept or constant term.  1 and  2 are partial regression coefficients.

51 Time Series Regression Static Regression Time Series Regression with Ordinary Regressors Time Series Regression with Dynamic Regressors

52 Common Transfer Functions Contemporaneous Regression Model

53 Common Transfer Functions Dynamic Regression: One Lag Term Model

54 Common Transfer Functions Dynamic Regression: One Shifted Term Model

55 Common Transfer Functions Dynamic Regression: One Shifted and One Lag Term Model