1. Introduction to stochastic processes
Mark Scheuerell
FISH 507 – Applied Time Series Analysis
13 January 2015

2. Topics for today Quick review of Correlograms White noise Random walks
Linear stationary models
Autoregressive (AR)
Moving average (MA)
Autoregressive moving average (ARMA)
Using ACF & PACF for model ID

3. The correlogram

4. White noise (WN)
A time series {wt : t = 1,2,3,…,n} is discrete white noise if the variables w1, w2, w3, …, wn are
independent, and
identically distributed with a mean of zero
Gaussian WN has the following 2nd-order properties:

5. White noise

6. Random walk (RW)
A time series {xt : t = 1,2,3,…,n} is a random walk if
xt = xt-1 + wt, and
wt is white noise
RW has the following 2nd-order properties:
Random walks are NOT stationary!

7. Random walk (RW)

8. Iterative approach to model building
Postulate general class of models
Identify candidate model
Estimate parameters
Diagnostics:
is model adequate?
Use model for forecasting or control No
Yes

9. Linear stationary models
MAR(1) Workshop - ESA 2007, San Jose, CA
Linear stationary models
5 Aug 2007
We saw last week that linear filters are a useful way of modeling time series
Here we extend those ideas to a general class of models call autoregressive moving average (ARMA)

10. Autoregressive (AR) models
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007
An autoregressive model of order p, or AR(p), is defined as
where we assume
wt is WN, and
fp ≠ 0 for order-p process
Note: RW model is special case of AR(1) with f1 = 1

11. Stationary & nonstationary AR models
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007
We can write out an AR(p) model using the backward shift notation, such that
If we treat B as a number, we can out write the characteristic equation as
In order to be stationary, all roots of char eqn must exceed 1 in absolute value

12. Stationary & nonstationary AR models
MAR(1) Workshop - ESA 2007, San Jose, CA
Stationary & nonstationary AR models
5 Aug 2007
For example, a RW model is not stationary because f = 1 – B, and hence, B = 1
However, the AR(1) model xt = 0.5xt-1 + wt is because f = 1 – 0.5B, and hence, B = 2 > 1

13. Examples of AR(1) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

14. Partial autocorrelation function
MAR(1) Workshop - ESA 2007, San Jose, CA
Partial autocorrelation function
5 Aug 2007
The partial autocorrelation function (PACF) measures the linear correlation of a series xt and xt+k with the linear dependence of {xt-1,xt-2,…,xt-(k-1)} removed
It is defined as

15. ACF & PACF for AR(3) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

16. PACF for AR(p) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

17. Using ACF & PACF for model ID
MAR(1) Workshop - ESA 2007, San Jose, CA
Using ACF & PACF for model ID
5 Aug 2007
ACF
PACF
AR(p)
Tails off slowly
Cuts off after lag-p

18. Moving average (MA) models
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007
A moving average model of order q, or MA(q), is defined as
where wt is WN (with 0 mean)
It is simply the current error term plus a weighted sum of the q most recent error terms
Because MA processes are finite sums of stationary WN processes, they are themselves stationary

19. MAR(1) Workshop - ESA 2007, San Jose, CA
Invertible MA models
5 Aug 2007
We can write out an MA(q) model using the backward shift notation, such that
An MA process is invertible if it can be expressed as a stationary autoregressive process of infinite order without an error term
For example, an MA(1) process with q < |1|

20. Examples of MA(q) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

21. Using ACF & PACF for model ID
MAR(1) Workshop - ESA 2007, San Jose, CA
Using ACF & PACF for model ID
5 Aug 2007
ACF
PACF
AR(p)
Tails off slowly
Cuts off after lag-p
MA(q)
Cuts off after lag-q

22. Autoregressive moving average models
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007
A time series is autoregressive moving average, or ARMA(p,q), if it is stationary and
We can write out an ARMA(p,q) model using the backward shift notation, such that
ARMA models are stationary if all roots of f(B) > 1
ARMA models are invertible if all roots of q(B) > 1

23. Examples of ARMA(p,q) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

24. ACF for ARMA(p,q) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

25. PACF for ARMA(p,q) processes
MAR(1) Workshop - ESA 2007, San Jose, CA
5 Aug 2007

26. Using ACF & PACF for model ID
MAR(1) Workshop - ESA 2007, San Jose, CA
Using ACF & PACF for model ID
5 Aug 2007
ACF
PACF
AR(p)
Tails off slowly
Cuts off after lag-p
MA(q)
Cuts off after lag-q
ARMA(p,q)
Tails off (after lag [q-p])
Tails off (after lag [p-q])

27. Nonstationary time series models
MAR(1) Workshop - ESA 2007, San Jose, CA
Nonstationary time series models
5 Aug 2007
If the data appear stationary, we can try various forms of ARMA models
If not, differencing can often make them stationary
This leads to the class of autoregressive integrated moving average (ARIMA) models
ARIMA models are indexed with orders (p,d,q); the d indicates the order of differencing
For d > 0, {xt} is an ARIMA(p,d,q) process if
(1-B)d xt is a causal ARMA(p,q) process

28. Example of an ARIMA model
MAR(1) Workshop - ESA 2007, San Jose, CA
Example of an ARIMA model
5 Aug 2007
Data do not appear stationary!
Differenced data look much better

29. Example of an ARIMA model
MAR(1) Workshop - ESA 2007, San Jose, CA
Example of an ARIMA model
5 Aug 2007
xt is ARIMA(1,1,0)
∇xt is AR(1)

30. Topics for today Quick review of Correlograms White noise Random walks
Linear stationary models
Autoregressive (AR)
Moving average (MA)
Autoregressive moving average (ARMA)
Using ACF & PACF for model ID