Presentation is loading. Please wait.

Presentation is loading. Please wait.

Autoregressive Integrated Moving Average (ARIMA) models 1.

Similar presentations

Presentation on theme: "Autoregressive Integrated Moving Average (ARIMA) models 1."— Presentation transcript:

1 Autoregressive Integrated Moving Average (ARIMA) models 1

2 2 - Forecasting techniques based on exponential smoothing -General assumption for the above models: times series data are represented as the sum of two distinct components (deterministc & random) - Random noise: generated through independent shocks to the process -In practice: successive observations show serial dependence

3 - ARIMA models are also known as the Box-Jenkins methodology -very popular : suitable for almost all time series & many times generate more accurate forecasts than other methods. -limitations: If there is not enough data, they may not be better at forecasting than the decomposition or exponential smoothing techniques. Recommended number of observations at least Weak stationarity is required - Equal space between intervals 3 ARIMA Models

4 4

5 5

6 Linear Models for Time series 6

7 7 Linear Filter - It is a process that converts the input x t, into output y t -The conversion involves past, current and future values of the input in the form of a summation with different weights -Time invariant do not depend on time -Physically realizable: the output is a linear function of the current and past values of the input -Stable if In linear filters: stationarity of the input time series is also reflected in the output

8 Stationarity 8

9 A time series that fulfill these conditions tends to return to its mean and fluctuate around this mean with constant variance. Note: Strict stationarity requires, in addition to the conditions of weak stationarity, that the time series has to fulfill further conditions about its distribution including skewness, kurtosis etc. 9 -Take snaphots of the process at different time points & observe its behavior: if similar over time then stationary time series -A strong & slowly dying ACF suggests deviations from stationarity Determine stationarity

10 10

11 11

12 Infinite Moving Average Input x t stationary THEN, the linear process with white noise time series ε t Is stationary 12 Output y t Stationary, with & ε t independent random shocks, with E(ε t )=0 &

13 13 autocovariance function Linear Process Infinite moving average

14 14 The infinite moving average serves as a general class of models for any stationary time series THEOREM (World 1938): Any no deterministic weakly stationary time series yt can be represented as where INTERPRETATION A stationary time series can be seen as the weighted sum of the present and past disturbances

15 15 Infinite moving average: -Impractical to estimate the infinitely weights -Useless in practice except for special cases: i. Finite order moving average (MA) models : weights set to 0, except for a finite number of weights ii. Finite order autoregressive (AR) models: weights are generated using only a finite number of parameters iii. A mixture of finite order autoregressive & moving average models (ARMA)

16 Finite Order Moving Average (MA) process Moving average process of order q(MA(q)) MA(q) : always stationary regardless of the values of the weights 16

17 Expected value of MA(q) Variance of MA(q) Autocovariance of MA(q) Autocorelation of MA(q) 17 ε t white noise

18 18 ACF function: Helps identifying the MA model & its appropriate order as its cuts off after lag k Real applications: r(k) not always zero after lag q; becomes very small in absolute value after lag q

19 First Order Moving Average Process MA(1) Autocovariance of MA(q) Autocorelation of MA(q) 19 q=1

20 20 -Mean & variance : stable -Short runs where successive observations tend to follow each other -Positive autocorrelation -Observations oscillate successively -negative autocorrelation

21 Second Order Moving Average MA(2) process Autocovariance of MA(q) Autocorelation of MA(q) 21

22 22 The sample ACF cuts off after lag 2

23 Finite Order Autoregressive Process 23 -Worlds theorem: infinite number of weights, not helpful in modeling & forecasting -Finite order MA process: estimate a finite number of weights, set the other equal to zero Oldest disturbance obsolete for the next observation; only finite number of disturbances contribute to the current value of time series -Take into account all the disturbances of the past : use autoregressive models; estimate infinitely many weights that follow a distinct pattern with a small number of parameters

24 First Order Autoregressive Process, AR(1) Assume : the contributions of the disturbances that are way in the past are small compared to the more recent disturbances that the process has experienced Reflect the diminishing magnitudes of contributions of the disturbances of the past,through set of infinitely many weights in descending magnitudes, such as The weights in the disturbances starting from the current disturbance and going back in the past: 24 Exponential decay pattern

25 First order autoregressive process AR(1) AR(1) stationary if 25 where WHY AUTOREGRESSIVE ?

26 Mean AR(1) Autocovariance function AR(1) Autocorrelation function AR(1) 26 The ACF for a stationary AR(1) process has an exponential decay form

27 27 Observe: -The observations exhibit up/down movements

28 Second Order Autoregressive Process, AR(2) 28 This model can be represented in the infinite MA form & provide the conditions of stationarity for y t in terms of φ 1 & φ 2 WHY? 1. Infinite MA Apply

29 29 where

30 30 Calculate the weights We need

31 31 Solutions The satisfy the second-order linear difference equation The solution : in terms of the 2 roots m1 and m2 from AR(2) stationary: Condition of stationarity for complex conjugates a+ib: AR(2) infinite MA representation:

32 32 Mean Autocovariance function For k=0: For k>0: Yule-Walker equations

33 33 Autocorrelation function Solutions A. Solve the Yule-Walker equations recursively B. General solution Obtain it through the roots m 1 & m 2 associated with the polynomial

34 34 Case I: m 1, m 2 distinct real roots c 1, c 2 constants: can be obtained from ρ (0),ρ(1) stationarity: ACF form: mixture of 2 exponentially decay terms e.g. AR(2) model It can be seen as an adjusted AR(1) model for which a single exponential decay expression as in the AR(1) is not enough to describe the pattern in the ACF and thus, an additional decay expression is added by introducing the second lag term y t-2

35 35 Case II: m 1, m 2 complex conjugates in the form c 1, c 2 : particular constants ACF form: damp sinusoid; damping factor R; frequency ; period

36 36 Case III: one real root m 0 ; m 1 = m 2 =m 0 ACF form: exponential decay pattern

37 37 AR(2) process :y t =4+0.4y t y t-2 +e t Roots of the polynomial: real ACF form: mixture of 2 exponential decay terms

38 38 AR(2) process: y t =4+0.8y t y t-2 +e t Roots of the polynomial: complex conjugates ACF form: damped sinusoid behavior

39 39 General Autoregressive Process, AR(p) Consider a pth order AR model or

40 40 AR(P) stationary If the roots of the polynomial are less than 1 in absolute value AR(P) absolute summable infinite MA representation Under the previous condition

41 41 Weights of the random shocks as

42 42 For stationary AR(p)

43 43 ACF p th order linear difference equations AR(p) : -satisfies the Yule-Walker equations -ACF can be found from the p roots of the associated polynomial e.g. distinct & real roots : - In general the roots will not be real ACF : mixture of exponential decay and damped sinusoid

44 44 ACF -MA(q) process: useful tool for identifying order of process cuts off after lag k -AR(p) process: mixture of exponential decay & damped sinusoid expressions Fails to provide information about the order of AR

45 45 Partial Autocorrelation Function Consider : -three random variables X, Y, Z & -Simple regression of X on Z & Y on Z The errors are obtained from

46 46 Partial correlation between X & Y after adjusting for Z: The correlation between X* & Y* Partial correlation can be seen as the correlation between two variables after being adjusted for a common factor that affects them

47 47 Partial autocorrelation function (PACF) between y t & y t-k The autocorrelation between y t & y t-k after adjusting for y t-1, y t-2, …y t-k AR(p) process: PACF between y t & y t-k for k>p should equal zero Consider -a stationary time series y t ; not necessarily an AR process -For any fixed value k, the Yule-Walker equations for the ACF of an AR(p) process

48 48 Matrix notation Solutions For any given k, k =1,2,… the last coefficient is called the partial autocorrelation coefficient of the process at lag k AR(p) process: Identify the order of an AR process by using the PACF

49 49 Cuts off after 1 st lag Decay pattern AR(2) MA(1)MA(2) Decay pattern AR(1) AR(2) Cuts off after 2 nd lag

50 50 Invertibility of MA models Invertible moving average process: The MA(q) process is invertible if it has an absolute summable infinite AR representation It can be shown: The infinite AR representation for MA(q)

51 51 Obtain We need Condition of invertibility The roots of the associated polynomial be less than 1 in absolute value An invertible MA(q) process can then be written as an infinite AR process

52 52 PACF of a MA(q) process is a mixture of exponential decay & damp sinusoid expressions In model identification, use both sample ACF & sample PACF PACF possibly never cuts off

53 53 Mixed Autoregressive –Moving Average (ARMA) Process ARMA (p,q) model Adjust the exponential decay pattern by adding a few terms

54 54 Stationarity of ARMA (p,q) process Related to the AR component ARMA(p,q) stationary if the roots of the polynomial less than one in absolute value ARMA(p,q) has an infinite MA representation

55 55 Invertibility of ARMA(p,q) process Invertibility of ARMA process related to the MA component Check through the roots of the polynomial If the roots less than 1 in absolute value then ARMA(p,q) is invertible & has an infinite representation Coefficients:

56 56 ARMA(1,1) Sample ACF & PACF: exponential decay behavior

57 57

58 58

59 59

60 60 Non Stationary Process Not constant level, exhibit homogeneous behavior over time y t is homogeneous, non stationary if -It is not stationary -Its first difference, w t =y t -y t-1 =(1-B)y t or higher order differences w t =(1-B) d y t produce a stationary time series Y t autoregressive intergrated moving average of order p, d,q –ARIMA(p,d,q) If the d difference, w t =(1-B) d y t produces a stationary ARMA(p,q) process ARIMA(p,d,q)

61 61 The random walk process ARIMA(0,1,0) Simplest non-stationary model First differencing eliminates serial dependence & yields a white noise process

62 62 y t =20+y t-1 +e t Evidence of non-stationary process -Sample ACF : dies out slowly -Sample PACF: significant at the first lag -Sample PACF value at lag 1 close to 1 First difference -Time series plot of w t : stationary -Sample ACF& PACF: do not show any significant value -Use ARIMA(0,1,0)

63 63 The random walk process ARIMA(0,1,1) Infinite AR representation, derived from: ARIMA(0,1,1)= (IMA(1,1)): expressed as an exponential weighted moving average (EWMA) of all past values

64 64 ARIMA(0,1,1) -The mean of the process is moving upwards in time -Sample ACF: dies relatively slow -Sample PACF: 2 significant values at lags 1& 2 -First difference looks stationary -Sample ACF & PACF: an MA(1) model would be appropriate for the first difference, its ACF cuts off after the first lag & PACF decay pattern Possible model :AR(2) Check the roots

65 65

66 66

Download ppt "Autoregressive Integrated Moving Average (ARIMA) models 1."

Similar presentations

Ads by Google