1. Prediction and model selection
Chapter 3
Prediction and model selection
Introduction to time series (2008)

2. Introduction to time series (2008)
Chapter 3. Contents.
3.1.  Properties of MMSE of prediction.
3.2.  The computation of ARIMA forecasts.
      
3.3.  Interpreting the forecasts from ARIMA models.
3.4.  Prediction confidence intervals.
Forecasting updating.
Model selection criteria.
Introduction to time series (2008)

3. Introduction to time series (2008)
We assume an observed sample:
and want to generate predictions of future values given the observations,
T is the forecast origin and k the forecast horizon.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

4. Introduction to time series (2008)
Three components
1. Estimation of new values: prediction.
2. Measure of the uncertainty: prediction intervals.
3. Arrival of new data: updating.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

5. Chapter 3. Prediction and model selection.
3.1. Properties of MMSE of prediction.
Introduction to time series (2008)

6. Properties of MMSE of prediction
Prediction by the conditional expectation
We have T observations of a zero mean stationary time series , and we want to forecast the value of
In order to compare alternative forecasting procedures, we need a criterion of optima- lity.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

7. Properties of MMSE of prediction
Minimum Mean Square Error Forecasts (MMSF). Forecasts that minimize this criterion can be computed as follows.
Let be the forecast we want to generate, this forecast must minimize
Where the expected value is taken over the joint distribution of and
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

8. Properties of MMSE of prediction
Using the well-known property of
we obtain
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

9. Properties of MMSE of prediction
and taking the derivative, we obtain
This result indicates that, conditioning to the observed sample, the MMSEF is obtained by computing the conditional expectation of the random variable given the available information.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

10. Properties of MMSE of prediction
Linear predictions
Conditional expectations can be, in some cases, difficult to compute.
Restrict our search to forecasting functions that are linear functions of the observations.
General equation for a linear predictor .
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

11. Properties of MMSE of prediction
calling MSEL to the mean square error of a linear forecast
minimizing this expression with respect to the parameters, we have
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

12. Properties of MMSE of prediction
Which implies that the best linear forecast must be such that the forecast error is uncorrelated with the set of observed variables.
This property suggests the interpretations of the linear predictor as projections.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

13. Properties of MMSE of prediction
that is, finding the coefficients of the best linear predictor is equivalent to regress,
then,
where is the covariance matrix of and is the covariance vector between and
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

14. Chapter 3. Prediction and model selection.
3.2. The computation of ARIMA forecasts.
Introduction to time series (2008)

15. The computation of ARIMA forecasts
Suppose we want to forecast a time series that follows an ARIMA(p,d,q) model. First, we will assume that the parameters are known and the prediction horizon is 1 (k=1)
where h=p+d
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

16. The computation of ARIMA forecasts
The one-step-ahead forecast will be,
and because the expected value for the observed sample data or the errors are themselves, and the only unknown is
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

17. The computation of ARIMA forecasts
Therefore, the one-step prediction error is,
remember this is considering that the parameters are known, and therefore, the innovations are also known because we can compute them recursively from the observations
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

18. The computation of ARIMA forecasts
Multiple steps ahead forecast.
where
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

19. The computation of ARIMA forecasts
This expression has two parts:
The first one, which depends on the AR coefficients, will determine the form of the long run forecast (eventual forecast equation).
The second one, which depends on the moving average coefficients, will dissapear for k>q
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

20. The computation of ARIMA forecasts
AR(1) model
for large k, the term , and therefore, the long-run forecast (for any ARMA(p,q)) will go to the mean of the process.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

21. The computation of ARIMA forecasts
Random walk with constant.
The forecasts follow a straight line with slope c. If c=0, all forecasts are equal to the last observed value.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

22. Chapter 3. Prediction and model selection.
3.3. Interpreting the forecasts from ARIMA models.
Introduction to time series (2008)

23. Interpretation of the forecasts
Nonseasonal models.
The eventual forecast function of a nonseasonal ARIMA model verifies for k>q
where
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

24. Interpretation of the forecasts
Espasa and Peña (1995) proved that the general solution for this equation can be written as,
where, the permanent component is,
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

25. Interpretation of the forecasts
and the transitory component is,
Permanent component will be given by
With determined by the mean of the stationary process
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

26. Interpretation of the forecasts
whereas the rest of the parameters, depend on the initial values and change with the forecast origin.
Examples:
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

27. Interpretation of the forecasts
1. will be constant for all horizons.
2. deterministic linear trend with slope , if , then the permanent component is just a constant.
3. the solution is a quadratic trend with the leading term determined by . If the equation reduces to a linear trend, but now the slope depends on the origin of the forecast.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

28. Interpretation of the forecasts
In summary, the long-run forecast from an ARIMA model is the mean if the series is stationary and a polynomial for nonstatio-nary models.
In this last case, the leading term of the polynomial is a constant (when the mean is zero), whereas it depends on the forecast origin (adaptative) if the mean is different from zero.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

29. Interpretation of the forecasts
Transitory component. Can be given by
where are the roots of the AR polyno-mial and are coefficientes depending on the forecast origin.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

30. Interpretation of the forecasts
Example. Consider the model,
then , and the forecasts must have the form,
where ,the constant that appears as the solution of and , the constant in the transitory equation must
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

31. Interpretation of the forecasts
be determined from the initial conditions and can be obtained by
and the solution of these two equations is
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

32. Interpretation of the forecasts
and,
these results indicate that the forecasts are slowly approaching the long run forecast
note that as goes to zero, the adjust-ment made by the transitory decreases exponentially. Cases for
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

33. Interpretation of the forecasts
Seasonal models. For seasonal processes the forecast will satisfy the equation
Let us assume that D=1, then the seasonal difference
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

34. Interpretation of the forecasts
and therefore,
which has the property that all the operators involved do not share roots in common. The solution is given by
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

35. Interpretation of the forecasts
Permanent component has been splitted into two terms, trend component
and the seasonal component
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

36. Interpretation of the forecasts
Finally the transitory component, which will die out for large horizon is,
the trend component has the same form as for nonseasonal data, but the order is d+1 and therefore the last term is
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

37. Interpretation of the forecasts
the seasonal component will be given by
and the solution of this equation is a function of period s and values summing zero each s lags. The coefficients are called seasonal coefficients and depend on the forecasting origin.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

38. Interpretation of the forecasts
Example: the airline model.
The equation of the forecast is
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

39. Interpretation of the forecasts
This equation can be written,
that is a linear trend plus a seasonal compo-nent with coefficients that are changing over time. In order to determine the parameters, we need 13 initial conditions
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

40. Interpretation of the forecasts
With , we obtain that the slope is,
and calling
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

41. Interpretation of the forecasts
we have that
The seasonal ceofficients are
and will be given by the deviations of the forecast from the trend component.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

42. Prediction confidence intervals
Known parameter values. Let us write
then, we can write
taking expected values conditional to data
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

43. Prediction confidence intervals
The forecast error is
with variance
this equation indicates that the uncertainty of the long run forecast is different for stationary and nonstationary models.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

44. Prediction confidence intervals
For a stationary model the series converge since
For an AR(1) model, for instance
The long run forecast goes to the mean, and the uncertainty is finite.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

45. Prediction confidence intervals
When the model is nonstationary, the variance of the forecast grows without bounds. This means that we cannot make useful long run forecasts.
If the distribution of the forecast error is known, we can compute confidence intervals for the forecast.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

46. Prediction confidence intervals
Assuming normality, the 95% confidence interval for the random variable
We may also need the covariances, for h>0
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

47. Prediction confidence intervals
Unknown parameter values. It can be shown that the uncertainty introduced in the forecast for this additional source is small for moderate sample size, and can be ignored in practice.
Suppose an AR(1) model,
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

48. Prediction confidence intervals
the true forecast error , is related to the observed forecast error
assuming that is fixed, and using that
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

49. Prediction confidence intervals
we have that
This equation indicates that the forecast error has two components:
uncertainty due to the random behavoir of the observation.
The second measures the parameter uncertainty becuse the parameter are estimated from the sample. (order 1/n - can be ignored for large n).
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

50. Chapter 3. Prediction and model selection.
3.5. Forecasting updating.
Introduction to time series (2008)

51. Introduction to time series (2008)
Forecasting updating.
Computing updated forecasts. When new observations become available, forecasts are updated by
which leads to
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

52. Introduction to time series (2008)
Forecasting updating.
where , is the one-step-ahead forecast error.
and so the forecasts are updated by
Note that the forecasts are updated by adding some part of the observed last forecast error to the previous forecast, and the coefficients for forecast updating are the weights.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

53. Introduction to time series (2008)
Forecasting updating.
Testing model stability Box and Tiao(1976) If the model is correct, we have that the statistic,
because we need an estimate of the variance
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

54. Chapter 3. Prediction and model selection.
3.6. Model selection criteria.
Introduction to time series (2008)

55. Model selection criteria.
The FPE and AIC criteria. Suppose we want to select the order of an AR(p) model in such a way that the out-of-sample one-step- ahead prediction mean-square error is minimized. This MSE is given by
With
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

56. Model selection criteria.
The forecast error can be decomposed as
and so
which decomposes the forecast error as the sum of the variable uncertainty and the parameter uncertainty.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

57. Model selection criteria.
This expectation can be approximated by,
An unbiased estimate of is, Inserting this, we have an estimation of the out-of-sample forecast error. If we want to minimize this value, it implies that the order p must be chosen by minimizing the FPE criterion
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

58. Model selection criteria.
The Final Prediction Error (FPE) combines fitting with parsimony, due to the penalty introduced by the term (n+p)/(n-p).
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

59. Model selection criteria.
An equivalent form for this criterion is
Multiplying for n, we ontain the AIC criterion
Tends to overstimate the number of parameters
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)

60. Model selection criteria.
Bayesian Information Criterion (BIC).
In this criteria the penalty is grater than in AIC, so BIC tends to select simpler models.
theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time
Introduction to time series (2008)