# Model Building For ARIMA time series Consists of three steps 1.Identification 2.Estimation 3.Diagnostic checking.

## Presentation on theme: "Model Building For ARIMA time series Consists of three steps 1.Identification 2.Estimation 3.Diagnostic checking."— Presentation transcript:

1 Model Building For ARIMA time series Consists of three steps 1.Identification 2.Estimation 3.Diagnostic checking

2 ARIMA Model building Identification Determination of p, d and q

3 To identify an ARIMA(p,d,q) we use extensively the autocorrelation function {  h : -  < h <  } and the partial autocorrelation function, {  kk : 0  k <  }.

4 The definition of the sample covariance function {C x (h) : -  < h <  } and the sample autocorrelation function {r h : -  < h <  } are given below: The divisor is T, some statisticians use T – h (If T is large, both give approximately the same results.)

5 It can be shown that: Thus Assuming  k = 0 for k > q

6 The sample partial autocorrelation function is defined by:

7 It can be shown that:

8 Identification of an Arima process Determining the values of p,d,q

9 Recall that if a process is stationary one of the roots of the autoregressive operator is equal to one. This will cause the limiting value of the autocorrelation function to be non-zero. Thus a nonstationary process is identified by an autocorrelation function that does not tail away to zero quickly or cut-off after a finite number of steps.

10 To determine the value of d Note: the autocorrelation function for a stationary ARMA time series satisfies the following difference equation The solution to this equation has general form where r 1, r 2, r 1, … r p, are the roots of the polynomial

11 For a stationary ARMA time series Therefore The roots r 1, r 2, r 1, … r p, have absolute value greater than 1. If the ARMA time series is non-stationary some of the roots r 1, r 2, r 1, … r p, have absolute value equal to 1, and

12 stationary non-stationary

13 If the process is non-stationary then first differences of the series are computed to determine if that operation results in a stationary series. The process is continued until a stationary time series is found. This then determines the value of d.

14 Identification Determination of the values of p and q.

15 To determine the value of p and q we use the graphical properties of the autocorrelation function and the partial autocorrelation function. Again recall the following:

16 More specically some typical patterns of the autocorrelation function and the partial autocorrelation function for some important ARMA series are as follows: Patterns of the ACF and PACF of AR(2) Time Series In the shaded region the roots of the AR operator are complex

17 Patterns of the ACF and PACF of MA(2) Time Series In the shaded region the roots of the MA operator are complex

18 Patterns of the ACF and PACF of ARMA(1.1) Time Series Note: The patterns exhibited by the ACF and the PACF give important and useful information relating to the values of the parameters of the time series.

19 Summary: To determine p and q. Use the following table. MA(q)AR(p)ARMA(p,q) ACFCuts after qTails off PACFTails offCuts after pTails off Note: Usually p + q ≤ 4. There is no harm in over identifying the time series. (allowing more parameters in the model than necessary. We can always test to determine if the extra parameters are zero.)

20 Examples

21

22 The data

23

24

25

26

27

28

29

30

31 Possible Identifications 1.d = 0, p = 1, q= 1 2.d = 1, p = 0, q= 1

32

33 ACF and PACF for x t,  x t and  2 x t (Sunspot Data)

34

35

36 Possible Identification 1.d = 0, p = 2, q= 0

37

38 ACF and PACF for x t,  x t and  2 x t (IBM Stock Price Data)

39

40

41 Possible Identification 1.d = 1, p =0, q= 0

42 Estimation of ARIMA parameters

43 Preliminary Estimation Using the Method of moments Equate sample statistics to population paramaters

44 Estimation of parameters of an MA(q) series The theoretical autocorrelation function in terms the parameters of an MA(q) process is given by. To estimate  1,  2, …,  q we solve the system of equations:

45 This set of equations is non-linear and generally very difficult to solve For q = 1 the equation becomes: Thus or This equation has the two solutions One solution will result in the MA(1) time series being invertible

46 For q = 2 the equations become:

47 Estimation of parameters of an ARMA(p,q) series We use a similar technique. Namely: Obtain an expression for  h in terms  1,  2,...,  p ;  1,  1,...,  q of and set up q + p equations for the estimates of  1,  2,...,  p ;  1,  2,...,  q by replacing  h by r h.

48 Estimation of parameters of an ARMA(p,q) series Example: The ARMA(1,1) process The expression for  1 and  2 in terms of  1 and  1 are: Further

49 Thus the expression for the estimates of  1,  1, and  2 are : and

50 Hence or This is a quadratic equation which can be solved

51 Example (ChemicalConcentration Data) the time series was identified as either an ARIMA(1,0,1) time series or an ARIMA(0,1,1) series. If we use the first identification then series x t is an ARMA(1,1) series.

52 Identifying the series x t is an ARMA(1,1) series. The autocorrelation at lag 1 is r 1 = and the autocorrelation at lag 2 is r 2 = Thus the estimate of  1 is 0.495/0.570 = Also the quadratic equation becomes which has the two solutions and Again we select as our estimate of  1 to be the solution -0.48, resulting in an invertible estimated series.

53 Since  =  (1 -  1 ) the estimate of  can be computed as follows: Thus the identified model in this case is x t = 0.87 x t-1 + u t u t

54 If we use the second identification then series  x t = x t – x t-1 is an MA(1) series. Thus the estimate of  1 is: The value of r 1 = Thus the estimate of  1 is: The estimate of  1 = -0.53, corresponds to an invertible time series. This is the solution that we will choose

55 The estimate of the parameter  is the sample mean. Thus the identified model in this case is:  x t = u t u t or x t = x t-1 + u t u t This compares with the other identification: x t = 0.87 x t-1 + u t u t (An ARIMA(1,0,1) model) (An ARIMA(0,1,1) model)

56 Preliminary Estimation of the Parameters of an AR(p) Process

57 and The regression coefficients  1,  2, ….,  p and the auto correlation function  h satisfy the Yule- Walker equations:

58 and The Yule-Walker equations can be used to estimate the regression coefficients  1,  2, ….,  p using the sample auto correlation function r h by replacing  h with r h.

59 Example Considering the data in example 1 (Sunspot Data) the time series was identified as an AR(2) time series. The autocorrelation at lag 1 is r 1 = and the autocorrelation at lag 2 is r 2 = The equations for the estimators of the parameters of this series are which has solution Since  =  ( 1 -  1 -  2 ) then it can be estimated as follows:

60 Thus the identified model in this case is x t = x t x t-2 + u t +14.9

61 Maximum Likelihood Estimation of the parameters of an ARMA(p,q) Series

62 The method of Maximum Likelihood Estimation selects as estimators of a set of parameters  1,  2,...,  k, the values that maximize L(  1,  2,...,  k ) = f(x 1,x 2,..., x N ;  1,  2,...,  k ) where f(x 1,x 2,..., x N ;  1,  2,...,  k ) is the joint density function of the observations x 1,x 2,..., x N. L(  1,  2,...,  k ) is called the Likelihood function.

63 It is important to note that: finding the values -  1,  2,...,  k - to maximize L(  1,  2,...,  k ) is equivalent to finding the values to maximize l(  1,  2,...,  k ) = ln L(  1,  2,...,  k ). l(  1,  2,...,  k ) is called the log-Likelihood function.

64 Again let {u t : t  T} be identically distributed and uncorrelated with mean zero. In addition assume that each is normally distributed. Consider the time series {x t : t  T} defined by the equation: (*)x t =  1 x t-1 +  2 x t  p x t-p +  + u t +  1 u t-1 +  2 u t  q u t-q

65 Assume that x 1, x 2,...,x N are observations on the time series up to time t = N. To estimate the p + q + 2 parameters  1,  2,...,  p ;  1,  2,...,  q ; ,  2 by the method of Maximum Likelihood estimation we need to find the joint density function of x 1, x 2,...,x N f(x 1, x 2,..., x N |  1,  2,...,  p ;  1,  2,...,  q, ,  2 ) = f(x| , , ,  2 ).

66 We know that u 1, u 2,...,u N are independent normal with mean zero and variance  2. Thus the joint density function of u 1, u 2,...,u N is g(u 1, u 2,...,u N ;  2 ) = g(u ;  2 ) is given by.

67 It is difficult to determine the exact density function of x 1,x 2,..., x N from this information however if we assume that p starting values on the x-process x* = (x 1-p,x 2-p,..., x o ) and q starting values on the u-process u* = (u 1-q,u 2-q,..., u o ) have been observed then the conditional distribution of x = (x 1,x 2,..., x N ) given x* = (x 1- p,x 2-p,..., x o ) and u* = (u 1-q,u 2-q,..., u o ) can easily be determined.

68 The system of equations : x 1 =  1 x 0 +  2 x  p x 1-p +  + u 1 +  1 u 0 +  2 u  q u 1-q x 2 =  1 x 1 +  2 x  p x 2-p +  + u 2 +  1 u 1 +  2 u  q u 2-q... x N =  1 x N-1 +  2 x N  p x N-p +  + u N +  1 u N-1 +  2 u N  q u N-q

69 can be solved for: u 1 = u 1 (x, x*, u*; , ,  ) u 2 = u 2 (x, x*, u*; , ,  )... u N = u N (x, x*, u*; , ,  ) (The jacobian of the transformation is 1)

70 Then the joint density of x given x* and u* is given by:

71 Let: = “conditional likelihood function”

72 “conditional log likelihood function” =

73 The values that maximize are the values that minimize with

74 Comment: Requires a iterative numerical minimization procedure to find: The minimization of: Steepest descent Simulated annealing etc

75 Comment: for specific values of The computation of: can be achieved by using the forecast equations

76 Comment: assumes we know the value of starting values of the time series {x t | t  T} and {u t | t  T} The minimization of : Namely x* and u*.

77 Approaches: 1.Use estimated values: 2.Use forecasting and backcasting equations to estimate the values:

78 Backcasting: If the time series {x t |t  T} satisfies the equation: It can also be shown to satisfy the equation: Both equations result in a time series with the same mean, variance and autocorrelation function: In the same way that the first equation can be used to forecast into the future the second equation can be used to backcast into the past:

79 Approaches to handling starting values of the series {x t |t  T} and {u t |t  T} 1.Initially start with the values: 2.Estimate the parameters of the model using Maximum Likelihood estimation and the conditional Likelihood function. 3.Use the estimated parameters to backcast the components of x*. The backcasted components of u* will still be zero.

80 4.Repeat steps 2 and 3 until the estimates stablize. This algorithm is an application of the E-M algorithm This general algorithm is frequently used when there are missing values. The E stands for Expectation (using a model to estimate the missing values) The M stands for Maximum Likelihood Estimation, the process used to estimate the parameters of the model.

81 Some Examples using: Minitab Statistica S-Plus SAS

Similar presentations