 # Prediction and model selection

## Presentation on theme: "Prediction and model selection"— Presentation transcript:

Prediction and model selection
Chapter 3 Prediction and model selection Introduction to time series (2008)

Introduction to time series (2008)
Chapter 3. Contents. 3.1.  Properties of MMSE of prediction. 3.2.  The computation of ARIMA forecasts.        3.3.  Interpreting the forecasts from ARIMA models. 3.4.  Prediction confidence intervals. Forecasting updating. Model selection criteria. Introduction to time series (2008)

Introduction to time series (2008)
We assume an observed sample: and want to generate predictions of future values given the observations, T is the forecast origin and k the forecast horizon. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Introduction to time series (2008)
Three components 1. Estimation of new values: prediction. 2. Measure of the uncertainty: prediction intervals. 3. Arrival of new data: updating. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Chapter 3. Prediction and model selection.
3.1. Properties of MMSE of prediction. Introduction to time series (2008)

Properties of MMSE of prediction
Prediction by the conditional expectation We have T observations of a zero mean stationary time series , and we want to forecast the value of In order to compare alternative forecasting procedures, we need a criterion of optima- lity. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
Minimum Mean Square Error Forecasts (MMSF). Forecasts that minimize this criterion can be computed as follows. Let be the forecast we want to generate, this forecast must minimize Where the expected value is taken over the joint distribution of and theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
Using the well-known property of we obtain theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
and taking the derivative, we obtain This result indicates that, conditioning to the observed sample, the MMSEF is obtained by computing the conditional expectation of the random variable given the available information. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
Linear predictions Conditional expectations can be, in some cases, difficult to compute. Restrict our search to forecasting functions that are linear functions of the observations. General equation for a linear predictor . theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
calling MSEL to the mean square error of a linear forecast minimizing this expression with respect to the parameters, we have theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
Which implies that the best linear forecast must be such that the forecast error is uncorrelated with the set of observed variables. This property suggests the interpretations of the linear predictor as projections. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Properties of MMSE of prediction
that is, finding the coefficients of the best linear predictor is equivalent to regress, then, where is the covariance matrix of and is the covariance vector between and theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Chapter 3. Prediction and model selection.
3.2. The computation of ARIMA forecasts. Introduction to time series (2008)

The computation of ARIMA forecasts
Suppose we want to forecast a time series that follows an ARIMA(p,d,q) model. First, we will assume that the parameters are known and the prediction horizon is 1 (k=1) where h=p+d theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
The one-step-ahead forecast will be, and because the expected value for the observed sample data or the errors are themselves, and the only unknown is theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
Therefore, the one-step prediction error is, remember this is considering that the parameters are known, and therefore, the innovations are also known because we can compute them recursively from the observations theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
Multiple steps ahead forecast. where theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
This expression has two parts: The first one, which depends on the AR coefficients, will determine the form of the long run forecast (eventual forecast equation). The second one, which depends on the moving average coefficients, will dissapear for k>q theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
AR(1) model for large k, the term , and therefore, the long-run forecast (for any ARMA(p,q)) will go to the mean of the process. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

The computation of ARIMA forecasts
Random walk with constant. The forecasts follow a straight line with slope c. If c=0, all forecasts are equal to the last observed value. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Chapter 3. Prediction and model selection.
3.3. Interpreting the forecasts from ARIMA models. Introduction to time series (2008)

Interpretation of the forecasts
Nonseasonal models. The eventual forecast function of a nonseasonal ARIMA model verifies for k>q where theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Espasa and Peña (1995) proved that the general solution for this equation can be written as, where, the permanent component is, theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
and the transitory component is, Permanent component will be given by With determined by the mean of the stationary process theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
whereas the rest of the parameters, depend on the initial values and change with the forecast origin. Examples: theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
1. will be constant for all horizons. 2. deterministic linear trend with slope , if , then the permanent component is just a constant. 3. the solution is a quadratic trend with the leading term determined by . If the equation reduces to a linear trend, but now the slope depends on the origin of the forecast. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
In summary, the long-run forecast from an ARIMA model is the mean if the series is stationary and a polynomial for nonstatio-nary models. In this last case, the leading term of the polynomial is a constant (when the mean is zero), whereas it depends on the forecast origin (adaptative) if the mean is different from zero. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Transitory component. Can be given by where are the roots of the AR polyno-mial and are coefficientes depending on the forecast origin. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Example. Consider the model, then , and the forecasts must have the form, where ,the constant that appears as the solution of and , the constant in the transitory equation must theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
be determined from the initial conditions and can be obtained by and the solution of these two equations is theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
and, these results indicate that the forecasts are slowly approaching the long run forecast note that as goes to zero, the adjust-ment made by the transitory decreases exponentially. Cases for theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Seasonal models. For seasonal processes the forecast will satisfy the equation Let us assume that D=1, then the seasonal difference theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
and therefore, which has the property that all the operators involved do not share roots in common. The solution is given by theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Permanent component has been splitted into two terms, trend component and the seasonal component theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Finally the transitory component, which will die out for large horizon is, the trend component has the same form as for nonseasonal data, but the order is d+1 and therefore the last term is theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
the seasonal component will be given by and the solution of this equation is a function of period s and values summing zero each s lags. The coefficients are called seasonal coefficients and depend on the forecasting origin. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
Example: the airline model. The equation of the forecast is theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
This equation can be written, that is a linear trend plus a seasonal compo-nent with coefficients that are changing over time. In order to determine the parameters, we need 13 initial conditions theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
With , we obtain that the slope is, and calling theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Interpretation of the forecasts
we have that The seasonal ceofficients are and will be given by the deviations of the forecast from the trend component. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
Known parameter values. Let us write then, we can write taking expected values conditional to data theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
The forecast error is with variance this equation indicates that the uncertainty of the long run forecast is different for stationary and nonstationary models. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
For a stationary model the series converge since For an AR(1) model, for instance The long run forecast goes to the mean, and the uncertainty is finite. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
When the model is nonstationary, the variance of the forecast grows without bounds. This means that we cannot make useful long run forecasts. If the distribution of the forecast error is known, we can compute confidence intervals for the forecast. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
Assuming normality, the 95% confidence interval for the random variable We may also need the covariances, for h>0 theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
Unknown parameter values. It can be shown that the uncertainty introduced in the forecast for this additional source is small for moderate sample size, and can be ignored in practice. Suppose an AR(1) model, theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
the true forecast error , is related to the observed forecast error assuming that is fixed, and using that theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Prediction confidence intervals
we have that This equation indicates that the forecast error has two components: uncertainty due to the random behavoir of the observation. The second measures the parameter uncertainty becuse the parameter are estimated from the sample. (order 1/n - can be ignored for large n). theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Chapter 3. Prediction and model selection.
3.5. Forecasting updating. Introduction to time series (2008)

Introduction to time series (2008)
Forecasting updating. Computing updated forecasts. When new observations become available, forecasts are updated by which leads to theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Introduction to time series (2008)
Forecasting updating. where , is the one-step-ahead forecast error. and so the forecasts are updated by Note that the forecasts are updated by adding some part of the observed last forecast error to the previous forecast, and the coefficients for forecast updating are the weights. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Introduction to time series (2008)
Forecasting updating. Testing model stability Box and Tiao(1976) If the model is correct, we have that the statistic, because we need an estimate of the variance theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Chapter 3. Prediction and model selection.
3.6. Model selection criteria. Introduction to time series (2008)

Model selection criteria.
The FPE and AIC criteria. Suppose we want to select the order of an AR(p) model in such a way that the out-of-sample one-step- ahead prediction mean-square error is minimized. This MSE is given by With theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Model selection criteria.
The forecast error can be decomposed as and so which decomposes the forecast error as the sum of the variable uncertainty and the parameter uncertainty. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Model selection criteria.
This expectation can be approximated by, An unbiased estimate of is, Inserting this, we have an estimation of the out-of-sample forecast error. If we want to minimize this value, it implies that the order p must be chosen by minimizing the FPE criterion theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Model selection criteria.
The Final Prediction Error (FPE) combines fitting with parsimony, due to the penalty introduced by the term (n+p)/(n-p). theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Model selection criteria.
An equivalent form for this criterion is Multiplying for n, we ontain the AIC criterion Tends to overstimate the number of parameters theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)

Model selection criteria.
Bayesian Information Criterion (BIC). In this criteria the penalty is grater than in AIC, so BIC tends to select simpler models. theoretically stationary means that the probability density functions of (z1,z2..)(z1+k,z2+k....) are identical for any arbitrary choice of k. That is to say, the overall behavior of the series remains the same over the time Introduction to time series (2008)