1. Judgment in Forecasting, Forecast Accuracy, Moving Averages and Decomposition Lecture 2 February 23, 2010

2. (c) Martin L. Puterman2 When should we use judgment in forecasting? When there’s no data When determining how much data to use When choosing a forecasting model When dealing with structural changes Adjusting outliers Adjusting quantitative forecasts when; –the consequences are great –they don’t make sense –additional information is available that is not included in the model –the penalties differ between over and under forecasting

3. (c) Martin L. Puterman3 Why are judgmental forecasts so poor? Forecasting is hard Forecasters have biases Recent events are given more weight Forecasters often rely on anecdotal information Ignoring data Justification of sunk costs Difficult to process large amounts of data Difficult to determine which data is usual and relevant Financial markets are efficient

4. Forecast the Next 5 years Makadakis et. al p.493 (c) Martin L. Puterman4 Sales (1000’s) Time 50

5. Forecast the Next 5 years Makadakis et. al p.495 (c) Martin L. Puterman5 Sales Time

6. (c) Martin L. Puterman6 Judgmental Forecasting Critique Manager’s forecasts tend to be overly optimistic. –Studies shows they are outperformed by statistical models!! Case Study (Makadakis et. al. p.493) –Mangers produced different forecasts from the same data when told it was their companies product and when told it was a competitors product or when told it was a new, a mature or an old product. –Most neutral parties forecasting from same data used extrapolation (trend model) and overestimated accuracy of the forecast.

7. (c) Martin L. Puterman7 Empirical evidence suggests that “expert” forecasts do not perform well Financial forecasts: Analysts and mutual funds consistently under perform markets or fail to predict market turns. Mutual fund advisor incentive systems encourage erratic behavior. Sales force forecasts: Influenced by incentives. Rewards for exceeding targets so set low targets or forecasts. Sales managers set high targets for motivation. In general such forecasts rely on anecdotal information So use inexpensive experts!

8. (c) Martin L. Puterman8 So why use judgment? In some cases you have to.  Little data available.  Long term forecasts required.  Rapid technological or marketplace change  New products  Relationships between predictors change  Adjustment of outliers  Consequence great

9. (c) Martin L. Puterman9 Consensus Forecasting Informal discussion and argument (committee?) Weighted average of all forecasters Delphi method –form individual forecasts and deliver to oracle –oracle summarizes forecasts and returns average, high, low, etc to individuals + reasons –individuals revise their forecasts –repeat process

10. (c) Martin L. Puterman10 How to improve judgmental forecasting Review historical data Practice; calibrate performance and give feedback Provide ranges –high (95 th percentile) –midpoint (median) –low point (5 th percentile) Provide scenarios with probabilities Clearly separate out forecasting and decision making

11. (c) Martin L. Puterman11 These calculations assume that we are forecasting k periods ahead and can assess its quality using a holdout sample Y t+k = Value in Period t+k; F t (k) = k period ahead forecast in period t n = forecast horizon Period t+k forecast error = Actual - Forecast = Y t+k - F t (k) Mean squared error (MSE) –Square the individual forecast errors over next n periods –Sum the squared errors and divide by n Accuracy Measures

12. (c) Martin L. Puterman12 Mean absolute error (MAE) –Take absolute values of forecast errors –Sum absolute values and divide by n Mean absolute percent error (MAPE) –Take absolute values of forecast percent errors –Sum percent errors and divide by n Forecasting Accuracy Measures

13. (c) Martin L. Puterman13 Assessing Out of Sample Forecasts

14. 14 Moving Averages F t (1) is average of last m observations Issue is to choose m Most appropriate if series is random variation around a mean –This is the case if all autocorrelations are near zero Not intended as a forecasting method - best for smoothing a series and determining patterns Lags behind an increasing series Calculated in a spreadsheet using Average function or using the MAVk transformation in NCSS. –Note NCSS averages past k observations. Alternatives are median smooth or LOESS smooth which are available in NCSS graphical routines.

15. 15 Moving Average Example

16. 16 Decomposition Method The decomposition method represents series –Additively as Y t = T t + S t + C t + I t –Multiplicatively as Y t = T t S t C t I t where T t is the trend component value at time t S t is the seasonal component value at time t C t is the cyclical component value at time t I t is the irregular or noise or “error” component at time t

17. 17 Decomposition Methods Some comments –Cyclical components not usually included since they cannot be forecasted and are hard to determine (may not exist) –The decomposition method provides a plausible approach for understanding time series behavior –Its suggests the following general forecasting approach; Deseasonalize data – use a forecasting method for stationary or trending series on the deseasonalized data and then reseasonalize.