1. Chapter 8. Model Identification
Given X1, ..., Xn -- a realization from an ARMA (p,q)
model, we usually need to estimate p and q
- “estimating” p and q is called model identification
First Step: Test for white noise
NOTE: Some slides have blank sections. They are based on a teaching style in which the corresponding blank (derivations, theorem proofs, examples, …) are worked out in class on the board or overhead projector.

2. White Noise
True Autocorrelations

3. Tests for white noise:
k > 0

4. Notes: 1. Test the hypotheses:
5% applies separately for each k and so approximately 5% of the sample autocorrelations may be outside the limits when data are white noise
if these occurrences are for small lags, continue modeling

5. White Noise
True Autocorrelations

6. Other Tests for White Noise
- Portmanteau tests (discussed in Chapter 9)
- Cumulative spectrum
- Tests for randomness

7. Another Example

8. Lesson
Always plot the data!

9. tswge demo Figure 3.18a AR(4) fit to data in Figure 3.18a
data(fig3.18a)
demo=est.ar.wge(fig3.18a,p=4,type='mle')
plotts.sample.wge(demo$res,lag.max=40,arlimits=TRUE)

10. Akaike’s Information Criterion
AIC
Akaike’s Information Criterion
General criterion for statistical model identification
Applies to a wide range of applications (including identifying p and q in an ARMA model)
Maximum Likelihood
applies to a model or class of distributions with a fixed set of parameters, say a1, … , as 

11. (i.e. 1 unknown parameter)
Likelihood function:
(i.e. 2 unknown parameters)
Likelihood function:
Suppose we are trying to decide between models with:
(a) m unknown and s =1
(b) m and s both unknown
Consider the strategy of selecting the model type for which we get the larger maximized likelihood.

12. Note: AIC – imposes a penalty for adding terms
ML does not apply to situation in which you are adding parameters to an existing model
adding parameters to an existing model will never cause the maximized likelihood to decrease
AIC – imposes a penalty for adding terms
Pick the model for which
AIC = -2[log(maximized likelihood)] + 2[# of free parameters]
is a minimum

13. In ARMA(p,q) modeling, the AIC criterion becomes:
Pick the model that minimizes
In ARMA Setting:
And in Practice:
AIC is approximated by

14. Other Criteria:
AICC
BIC
Note:

15. tswge AIC demo data(fig3.16a) plotts.sample.wge(fig3.16a)
aic.wge(fig3.16a,p=0:5,q=0:2, type='aic')
# estimating parameters in the model given by aic d=est.ar.wge(fig3.16a,p=3, type='mle')
plotts.sample.wge(d$res,lag.max=40,arlimits=TRUE)
# to check the 5 “best” models
aic5.wge(fig3.16a,p=0:5,q=0:2,type='aic')

16. tswge AIC demo - continued
data(fig3.24a)
plotts.sample.wge(fig3.24a)
d=aic.wge(fig3.24a,p=0:8,q=0:4, type='aic')
# aic5.wge(fig3.24a,p=0:8,q=0:4, type='aic') #
d1=est.arma.wge(fig3.24a,p=6,q=1)
factor.wge(d1$phi)
aic5.wge(fig3.24a,p=0:8,q=0:4, type='bic')
d21=est.arma.wge(fig3.24a,p=2,q=1)
plotts.sample.wge(d21$res,lag.max=40,arlimits=TRUE)

17. Model ID for ARUMA(p, d, q) Models  

18. Recall Theorem 5.1 (Findley-Quinn Theorem)
order is based on roots on unit circle of highest multiplicity
can cause parameter estimation problems
stationary components are often hidden by the nonstationary components
Tables YW estimates only “saw” the root near the unit circle
Tables YW estimates only “saw” one of the repeated roots

19. Model ID in Nonstationary Case - General Comments
Burg and ML estimates are better than YW in “nonstationary” case and for repeated roots
- see Tables
YW and Burg estimates always produce stationary AR models
Decision whether to fit stationary or nonstationary model depends on the physical problem being considered

20. Steps for obtaining an ARUMA(p,d,q) Model
Indications of Nonstationarity
• Nonstationary data tendencies
- wandering
- strong cyclic behavior
• Slowly damping sample autocorrelations
Steps for obtaining an ARUMA(p,d,q) Model
• Identify the nonstationary component(s) of the model
• Transform the data using the nonstationary factor
• Estimate the parameters of the “stationarized data”

21. Classical Box-Jenkins Procedure for Including a Unit Root in the Model
If data are wandering and sample autocorrelations damp slowly
-- difference the data
if differenced data still show evidence of a unit root
-- difference the data again, etc.

22. Sample Autocorrelations of Differenced Data
Original Data
Sample Autocorrelations
Sample Autocorrelations of Differenced Data
Differenced Data

23. A Model with 2 Unit Roots

24. Testing for Unit Roots Formal tests are available for
H0: model has a root of +1
- Dickey-Fuller tests (Section )
- popular in Econometrics

25. How Can You Find the Nonstationary Component(s) of a Model in General?
Tiao and Tsay -- Annals of Statistics (1983), pages
Theorem 8.1 (Tiao-Tsay Theorem)

26. Note: Overfitting using the tswge package: est.ar.wge function
Tiao/Tsay result says that roots on the unit circle can be found by successively overfitting data with an AR model
Gray and Woodward -- JASA (1986), pages
demonstrated application of the Tiao/Tsay result for identifying nonstationarities
used Burg estimates
Overfitting using the tswge package:
est.ar.wge function
finds MLE (or Burg or YW) estimates of an AR(p) model fit to the data
shows Factor Table of fitted model
for overfitting we use Burg estimates(always stationary)

27. Overfit Procedure Example 8.4 (1)
Figure 8.3 (Figure 5.3c)
Original Data
Sample Autocorrelations
Overfit Procedure
Examine high order AR fits to the data

28. Overfit Tables using tswge – see Table 8.1
data(fig5.3c)
f6=est.ar.wge(fig5.3c,p=6, type='burg') #
f8=est.ar.wge(fig5.3c,p=8, type='burg')
f10=est.ar.wge(fig5.3c,p=10, type='burg')
Each table has a factor associated with a positive real root substantially closer to the unit circle than roots associated with other factors
This suggests the need to difference the data
d1=artrans.wge(fig5.3c,phi.tr=1) #
f8=est.ar.wge(d1,p=8, type='burg')
f10=est.ar.wge(d1,p=10, type='burg')

29. A Model with 2 Unit Roots

30. tswge demo Differenced data do not seem stationary difference again
data(fig8.6a)
plotts.sample.wge(fig8.6a) #
f8=est.ar.wge(fig8.6a,p=8, type='burg')
f10=est.ar.wge(fig8.6a,p=10, type='burg')
d1=artrans.wge(fig8.6a,phi.tr=1)
f8=est.ar.wge(d1,p=8, type='burg')
f10=est.ar.wge(d1,p=10, type='burg')
Differenced data do not seem stationary
difference again
d2=artrans.wge(d1,phi.tr=1) #
f8=est.ar.wge(d2,p=8, type='burg')
f10=est.ar.wge(d2,p=10, type='burg')

31. Example 8.4 (1) Use tswge Figure 8.4 data(fig8.4a)
Original Data
Sample Autocorrelations
Use tswge
data(fig8.4a)
plotts.sample.wge(fig8.4a) #
f8=est.ar.wge(fig8.4a,p=8, type='burg')

32. Overfit Tables – see Table 8.2
Each table has a factor associated with a pair of complex conjugate roots with frequency close to f0=0.1 that are substantially closer to the unit circle than roots associated with other factors
This suggests the need to transform the data
Should the transformation be based on a nonstationary factor?
depends on the situation

33. Notes:
1. In Example 8.4(a) overfitting and the Box-Jenkins procedures identified the need to difference
2. In Example 8.4(b) the overfitting procedure shows to be applicable for other types of nonstationarities besides a positive unit root

34. Final Models: Example 8.5 (1)
We previously decided to difference the data.
data(fig5.3c)
plotts.sample.wge(fig5.3c)
dif=artrans.wge(fig5.3c,phi.tr=1)
plotts.sample.wge(dif) …
Final Model

35. Final Models: Example 8.5 (2) Model for transformed data: Final Model
data(fig8.4a)
plotts.sample.wge(fig8.4a)
d12=est.ar.wge(fig8.4a,p=12,type='burg')
# what transformation?
tr.dat=artrans.wge(fig8.4a,
plotts.sample.wge(tr.dat) …
Final Model

36. Seasonal Models Note: s 2s 3s 4s • • • • • 1 .5
• • • • •
••••••••• •••••••• ••••••••• ••••••••• •••••• 1 .5
-.5 -1
s s s s

37. Factor Tables (Tables 5.4 and 5.5)
Abs Recip f
Root(s)
1-B 1
1+B2
.25
 i
1+B .5 -1
factor.wge(phi=c(0,0,0,1))
Factor
Abs Recip f
Root(s)
1 – B 1
.083 i
1 - B + B2
.167 i
1+B2
.25
+ i
1 + B + B2
.333 i
.417 i
1 + B .5 -1
factor.wge(phi=c(0,0,0,0,0,0,0,0,0,0,0,1))

38. Procedure for Including 1 – B s in the Model
Compare factors in overfit factor table with those for – Bs under consideration.
If “nonstationary” factors match with factors of 1 – Bs fairly well and if 1 – Bs makes sense in the model, transform data by 1 – Bs .

39. Example 8.7 Simulated Seasonal Data
Figure 8.8 (a) and (b)
use tswge to perform overfit procedure
data(fig8.8a)
plotts.sample.wge(fig8.8a)
d15=est.ar.wge(fig8.8a,p=15, type='burg')

40. Based on overfit tables found using est.ar.wge
Final Model:

41. Example 8.8 Airline Data
Again - use overfit procedure using est.ar.wge

42. Based on overfit tables found using est.ar.wge
Final Model:

43. Example 8.9 Pennsylvania Monthly Temperature Data
Again! - use overfit procedure

44. Based on overfit tables found using est.ar.wge
Final Model:

45. Note: - or just because you have monthly data
DO NOT include 1 – B12, for example, in the model simply because there is a 12th order nonstationarity or because the data shows a period of 12
- or just because you have monthly data