1. Linear Filters

2. Let
denote a bivariate time series with zero mean.

3. Suppose that the time series {yt : t  T} is constructed as follows:
The time series {yt : t  T} is said to be constructed from {xt : t  T} by means of a Linear Filter.

5. The autocovariance function of the filtered series

6. Thus the spectral density of the time series
{yt : t  T} is:

7. Comment A:
is called the Transfer function of the linear filter.
is called the Gain of the filter while
is called the Phase Shift of the filter.

8. Also

9. Thus cross spectrum of the bivariate time series
is:

10. Definition:
= Squared Coherency function
Note:

11. Comment B:
= Squared Coherency function.
if {yt : t T} is constructed from {xt : t T} by means of a linear filter

12. Linear Filters with additive noise at the output

13. Let
denote a bivariate time series with zero mean.
Suppose that the time series {yt : t T} is constructed as follows:
t = ..., -2, -1, 0, 1, 2, ...
The noise {vt : t T} is independent of the series {xt : t T} (may be white)

14. nt

15. The autocovariance function of the filtered series with added noise

16. continuing
Thus the spectral density of the time series
{yt : t T} is:

17. Also

18. Thus cross spectrum of the bivariate time series
is:

19. Thus
= Squared Coherency function.
Noise to Signal Ratio

20. Box-Jenkins Parametric Modelling of a Linear Filter

21. Consider the Linear Filter of the time series {Xt : t T}:
where
and
= the Transfer function of the filter.

22. {at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then :
for t T Xt at
Linear
Filter

23. Also Note:

24. Hence {DYt} and {DXt} are related by the same Linear Filter.
Definition The Linear Filter
is said to be stable if :
converges for all |B| ≤1.

25. Discrete Dynamic Models:

26. Many physical systems whose output is represented by Y(t) are modeled by the following differential equation:
Where X(t) is the forcing function.

27. If X and Y are measured at discrete times this equation can be replaced by:
where D = I-B denotes the differencing operator.

28. This equation can in turn be represented with the operator B.
where

29. This equation can also be written in the form as a Linear filter as
Stability: It can easily be shown that this filter is stable if the roots of d(x) = 0 lie outside the unit circle.

30. Linear filter of a time series
Suppose that the time series {yt : t  T} is constructed as follows:
The time series {yt : t  T} is said to be constructed from {xt : t  T} by means of a Linear Filter.

31. The spectral density of the filtered time series
{yt : t  T} is:
is called the Transfer function
is called the Gain of the filter while
is called the Phase Shift of the filter.

32. Thus cross spectrum of the bivariate time series
Squared Coherency function
if {yt : t T} is constructed from {xt : t T} by means of a linear filter

33. Linear filter of a time series plus noise
Let
denote a bivariate time series with zero mean.
Suppose that the time series {yt : t T} is constructed as follows:
t = ..., -2, -1, 0, 1, 2, ...
The noise {vt : t T} is independent of the series {xt : t T} (may be white)

34. Then

35. The Box-Jenkins model of a Linear filter as
Stability: d(x) and p(x) are polynomials. a(x) is a power series. It can easily be shown that this filter is stable if the roots of d(x) = 0 lie outside the unit circle.

36. The sequence {at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then :
for t T Xt at
Linear
Filter

37. Determining the Impulse Response function from the Parameters of the Filter:

38. Now or
Hence

39. Equating coefficients results in the following conclusions:
aj = 0 for j < b.
aj - d1aj-1 - d2aj dr aj-r= wj
or aj = d1aj-1 + d2aj dr aj-r+ wj
for b ≤ j ≤ b+s.
and aj - d1aj-1 - d2aj dr aj-r= 0
or aj = d1aj-1 + d2aj dr aj-r for j > b+s.

40. Thus the coefficients of the transfer function,
a0, a1, a2,... ,
satisfy the following properties
1) b zeroes a0, a1, a2,..., ab-1
2) No pattern for the next s-r+1 values
ab, ab+1, ab+2,..., ab+s-r
3) The remaining values
ab+s-r+1, ab+s-r+2, ab+s-r+3,...
follow the pattern of an rth order difference equation
aj = d1aj-1 + d2aj dr aj-r

41. Example r =1, s=2, b=3, d1 = d
a0 = a1 = a2 = 0
a3 = da2 + w0 = w0
a4 = da3 + w1 = dw0 + w1
a5 = da4 + w2
= d[dw0 + w1] + w2
= d2w0 + dw1 + w2
aj = daj-1 for j ≥ 6.

42. Transfer function {at}

43. Identification of the Box-Jenkins Transfer Model with r=2

44. Recall the solution to the second order difference equation
aj = d1aj-1 + d2aj-2
follows the following patterns:
Mixture of exponentials if the roots of
1 - d1x - d2x2 = 0 are real.
2) Damped Cosine wave if the roots to
1 - d1x - d2x2 = 0 are complex.
These are the patterns of the Impulse Response function one looks for when identifying b,r and s.

45. Estimation of the Impulse Response function, aj
(without pre-whitening).

46. Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:
Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with
{Xt : t T}. Then

47. Suppose that for s > M, as = 0. Then a0, a1,
Suppose that for s > M, as = 0. Then a0, a1, ... ,aM can be found solving the following equations:

48. If the Cross autocovariance function, sXY(h), and the Autocovariance function, sXX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

49. In matrix notation this set of linear equations can be written:

50. If the Cross autocovariance function, sXY(h), and the Autocovariance function, sXX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

51. Estimation of the Impulse Response function, aj
(with pre-whitening).

52. Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:
Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with
{Xt : t T}.

53. In addition assume that {Xt : t T}, the weakly stationary time series has been identified as an ARMA(p,q) series, estimated and found to satisfy the following equation:
b(B)Xt = a(B)ut
where {ut : t T} is a white noise time series. Then
[a(B)]-1b(B)Xt = ut
transforms the Time series {Xt : t T} into the white noise time series{ut : t T}.

54. This process is called Pre-whitening the Input series.
Applying this transformation to the Output series {Yt : t T} yeilds:

55. or
where
and

56. In this case the equations for the impulse response function - a0, a1,
In this case the equations for the impulse response function - a0, a1, ... ,aM - become (assuming that for s > M, as = 0):

57. Identification and Estimation of Box-Jenkins transfer model
Summary
Identification and Estimation of
Box-Jenkins transfer model

58. To identify the series we need to determine
b, r and s.
The first step is to compute
the ACF’s and the cross CF’s
Cxx(h) and Cxy(h)
Estimate the impulse response function using

59. The Impulse response function {at}
Determine the value of b, r and s from the pattern of the impulse response function
The Impulse response function {at}
Pattern of an rth order difference equation b
s- r + 1

60. aj = d1aj-1 + d2aj-2+...+ dr aj-r
Determine preliminary estimates of the Box-Jenkins transfer function parameters using:
for j > b+s. .
aj = d1aj-1 + d2aj dr aj-r
for b ≤ j ≤ b+s
aj = d1aj-1 + d2aj dr aj-r+ wj
Determine preliminary estimates of the ARMA parameters of the input time series {xt}

61. Determine preliminary estimates of the ARIMA parameters of the noise time series {nt}

62. Maximum Likelihood estimation of the parameters of the Box-Jenkins Transfer function model

63. The Box- Jenkins model is written
The parameters of the model are:
In addition
the ARMA parameters of the input series {xt}
The ARIMA parameters of the noise series {nt}

64. The model for the noise {nt}series can be written

65. Given starting values for {yt}, {xt}, and
and the parameters of the transfer function model and the noise model
We can calculate successively:
The maximum likelihood estimates are the values
that minimize:

66. Fitting a transfer function model
Example: Monthly Sales (Y) and Monthly Advertising expenditures

67. The Data

68. Available in the Arts computer lab
Using SAS
Available in the Arts computer lab

69. The Start up window for SAS

70. To import data - Choose File -> Import data

71. The following window appears

72. Browse for the file to be imported

73. Identify the file in SAS

74. The next screen (not important) click Finish

75. The finishing screen

76. To fit a transfer function model we need to identify the model
You can now run analysis by typing code into the Edit window or selecting the analysis form the menu
To fit a transfer function model we need to identify the model
Determine the order of differencing to achieve Stationarity
Determine the value of b, r and s.

77. To determine the degree of differencing we look at ACF’s and PACF’s for various order of differencing

78. To produce the ACF, PACF – type the following commands into the Editor window- Press Run button

79. To identify the transfer function model we need to estimate the impulse response function using:
For this we need the ACF of the input series and the cross ACF of the input with the output

80. To produce the Cross correlation function – type the following commands into the Editor window

81. the impulse response function using can be determined using some other package (i.e. Excel)
r,s = 1
b = 4

82. To Estimate the transfer function model – type the following commands into the Editor window

83. To estimate the following model
Use
input=( b $ (w -lags ) / (d -lags) x)
In SAS

84. The Output

85. The Output

86. Using R to fit a Box- Jenkins transfer function model
One has to use the TSA package and the arimax function To load the TSA package use the command > library(TSA)

87. To estimate the following model
Use the command
> arimax(Sales,order=c(1,0,0),
fixed=c(0,NA,NA,NA,0,0,0,0,NA,NA,NA),
xtransf=Adver,transfer=list(c(2,6)),method="ML")
Coefficients:
ar1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4
s.e
T1-MA5 T1-MA6
s.e

88. Comparison with SAS Coefficients:
ar1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4
s.e
T1-MA5 T1-MA6
s.e

89. Examination of the residuals of this model
indicated that nt is not white noise but an MA(1) series
To fit this model we use the command:
> arimax(Sales,order=c(0,0,1),
fixed=c(NA,NA,NA,NA,0,0,0,0,NA,NA,NA),
xtransf=Adver,transfer=list(c(2,6)),method="ML")
To yield the estimates:
Coefficients:
ma1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3
s.e
T1-MA4 T1-MA5 T1-MA6
s.e

90. The fitted transfer function model
The constant term is not significant. To fit a model with constant term 0, we use:
> arimax(x = Sales, order = c(0, 0, 1), include.mean = FALSE, fixed = c(NA,NA, NA,0,0,0,0,NA,NA,NA), method="ML",xtransf= Adver,transfer =list(c(2,6)))

91. The estimated parameters
Coefficients:
ma1 T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4 T1-MA5
s.e
T1-MA6
3.0240
s.e
The fitted transfer function model