Linear Filters

Let denote a bivariate time series with zero mean.

Suppose that the time series {yt : t  T} is constructed as follows: The time series {yt : t  T} is said to be constructed from {xt : t  T} by means of a Linear Filter.

The autocovariance function of the filtered series

Thus the spectral density of the time series {yt : t  T} is:

Comment A: is called the Transfer function of the linear filter. is called the Gain of the filter while is called the Phase Shift of the filter.

Also

Thus cross spectrum of the bivariate time series is:

Definition: = Squared Coherency function Note:

Comment B: = Squared Coherency function. if {yt : t T} is constructed from {xt : t T} by means of a linear filter

Linear Filters with additive noise at the output

Let denote a bivariate time series with zero mean. Suppose that the time series {yt : t T} is constructed as follows: t = ..., -2, -1, 0, 1, 2, ... The noise {vt : t T} is independent of the series {xt : t T} (may be white)

nt

The autocovariance function of the filtered series with added noise

continuing Thus the spectral density of the time series {yt : t T} is:

Also

Thus cross spectrum of the bivariate time series is:

Thus = Squared Coherency function. Noise to Signal Ratio

Box-Jenkins Parametric Modelling of a Linear Filter

Consider the Linear Filter of the time series {Xt : t T}: where and = the Transfer function of the filter.

{at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then : for t T Xt at Linear Filter

Also Note:

Hence {DYt} and {DXt} are related by the same Linear Filter. Definition The Linear Filter is said to be stable if : converges for all |B| ≤1.

Discrete Dynamic Models:

Many physical systems whose output is represented by Y(t) are modeled by the following differential equation: Where X(t) is the forcing function.

If X and Y are measured at discrete times this equation can be replaced by: where D = I-B denotes the differencing operator.

This equation can in turn be represented with the operator B. where

This equation can also be written in the form as a Linear filter as Stability: It can easily be shown that this filter is stable if the roots of d(x) = 0 lie outside the unit circle.

Linear filter of a time series Suppose that the time series {yt : t  T} is constructed as follows: The time series {yt : t  T} is said to be constructed from {xt : t  T} by means of a Linear Filter.

The spectral density of the filtered time series {yt : t  T} is: is called the Transfer function is called the Gain of the filter while is called the Phase Shift of the filter.

Thus cross spectrum of the bivariate time series Squared Coherency function if {yt : t T} is constructed from {xt : t T} by means of a linear filter

Linear filter of a time series plus noise Let denote a bivariate time series with zero mean. Suppose that the time series {yt : t T} is constructed as follows: t = ..., -2, -1, 0, 1, 2, ... The noise {vt : t T} is independent of the series {xt : t T} (may be white)

Then

The Box-Jenkins model of a Linear filter as Stability: d(x) and p(x) are polynomials. a(x) is a power series. It can easily be shown that this filter is stable if the roots of d(x) = 0 lie outside the unit circle.

The sequence {at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then : for t T Xt at Linear Filter

Determining the Impulse Response function from the Parameters of the Filter:

Now or Hence

Equating coefficients results in the following conclusions: aj = 0 for j < b. aj - d1aj-1 - d2aj-2-...- dr aj-r= wj or aj = d1aj-1 + d2aj-2+...+ dr aj-r+ wj for b ≤ j ≤ b+s. and aj - d1aj-1 - d2aj-2-...- dr aj-r= 0 or aj = d1aj-1 + d2aj-2+...+ dr aj-r for j > b+s.

Thus the coefficients of the transfer function, a0, a1, a2,... , satisfy the following properties 1) b zeroes a0, a1, a2,..., ab-1 2) No pattern for the next s-r+1 values ab, ab+1, ab+2,..., ab+s-r 3) The remaining values ab+s-r+1, ab+s-r+2, ab+s-r+3,... follow the pattern of an rth order difference equation aj = d1aj-1 + d2aj-2+...+ dr aj-r

Example r =1, s=2, b=3, d1 = d a0 = a1 = a2 = 0 a3 = da2 + w0 = w0 a4 = da3 + w1 = dw0 + w1 a5 = da4 + w2 = d[dw0 + w1] + w2 = d2w0 + dw1 + w2 aj = daj-1 for j ≥ 6.

Transfer function {at}

Identification of the Box-Jenkins Transfer Model with r=2

Recall the solution to the second order difference equation aj = d1aj-1 + d2aj-2 follows the following patterns: Mixture of exponentials if the roots of 1 - d1x - d2x2 = 0 are real. 2) Damped Cosine wave if the roots to 1 - d1x - d2x2 = 0 are complex. These are the patterns of the Impulse Response function one looks for when identifying b,r and s.

Estimation of the Impulse Response function, aj (without pre-whitening).

Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation: Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with {Xt : t T}. Then

Suppose that for s > M, as = 0. Then a0, a1, Suppose that for s > M, as = 0. Then a0, a1, ... ,aM can be found solving the following equations:

If the Cross autocovariance function, sXY(h), and the Autocovariance function, sXX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

In matrix notation this set of linear equations can be written:

If the Cross autocovariance function, sXY(h), and the Autocovariance function, sXX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

Estimation of the Impulse Response function, aj (with pre-whitening).

Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation: Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with {Xt : t T}.

In addition assume that {Xt : t T}, the weakly stationary time series has been identified as an ARMA(p,q) series, estimated and found to satisfy the following equation: b(B)Xt = a(B)ut where {ut : t T} is a white noise time series. Then [a(B)]-1b(B)Xt = ut transforms the Time series {Xt : t T} into the white noise time series{ut : t T}.

This process is called Pre-whitening the Input series. Applying this transformation to the Output series {Yt : t T} yeilds:

or where and

In this case the equations for the impulse response function - a0, a1, In this case the equations for the impulse response function - a0, a1, ... ,aM - become (assuming that for s > M, as = 0):

Identification and Estimation of Box-Jenkins transfer model Summary Identification and Estimation of Box-Jenkins transfer model

To identify the series we need to determine b, r and s. The first step is to compute the ACF’s and the cross CF’s Cxx(h) and Cxy(h) Estimate the impulse response function using

The Impulse response function {at} Determine the value of b, r and s from the pattern of the impulse response function The Impulse response function {at} Pattern of an rth order difference equation b s- r + 1

aj = d1aj-1 + d2aj-2+...+ dr aj-r Determine preliminary estimates of the Box-Jenkins transfer function parameters using: for j > b+s. . aj = d1aj-1 + d2aj-2+...+ dr aj-r for b ≤ j ≤ b+s aj = d1aj-1 + d2aj-2+...+ dr aj-r+ wj Determine preliminary estimates of the ARMA parameters of the input time series {xt}

Determine preliminary estimates of the ARIMA parameters of the noise time series {nt}

Maximum Likelihood estimation of the parameters of the Box-Jenkins Transfer function model

The Box- Jenkins model is written The parameters of the model are: In addition the ARMA parameters of the input series {xt} The ARIMA parameters of the noise series {nt}

The model for the noise {nt}series can be written

Given starting values for {yt}, {xt}, and and the parameters of the transfer function model and the noise model We can calculate successively: The maximum likelihood estimates are the values that minimize:

Fitting a transfer function model Example: Monthly Sales (Y) and Monthly Advertising expenditures

The Data

Available in the Arts computer lab Using SAS Available in the Arts computer lab

The Start up window for SAS

To import data - Choose File -> Import data

The following window appears

Browse for the file to be imported

Identify the file in SAS

The next screen (not important) click Finish

The finishing screen

To fit a transfer function model we need to identify the model You can now run analysis by typing code into the Edit window or selecting the analysis form the menu To fit a transfer function model we need to identify the model Determine the order of differencing to achieve Stationarity Determine the value of b, r and s.

To determine the degree of differencing we look at ACF’s and PACF’s for various order of differencing

To produce the ACF, PACF – type the following commands into the Editor window- Press Run button

To identify the transfer function model we need to estimate the impulse response function using: For this we need the ACF of the input series and the cross ACF of the input with the output

To produce the Cross correlation function – type the following commands into the Editor window

the impulse response function using can be determined using some other package (i.e. Excel) r,s = 1 b = 4

To Estimate the transfer function model – type the following commands into the Editor window

To estimate the following model Use input=( b $ (w -lags ) / (d -lags) x) In SAS

The Output

The Output

Using R to fit a Box- Jenkins transfer function model One has to use the TSA package and the arimax function To load the TSA package use the command > library(TSA)

To estimate the following model Use the command > arimax(Sales,order=c(1,0,0), fixed=c(0,NA,NA,NA,0,0,0,0,NA,NA,NA), xtransf=Adver,transfer=list(c(2,6)),method="ML") Coefficients: ar1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4 0 1.3449 -0.0921 0.0446 0 0 0 0 10.0726 s.e. 0 9.6455 0.1879 0.0757 0 0 0 0 0.1757 T1-MA5 T1-MA6 5.9908 2.7742 s.e. 1.8903 0.3929

Comparison with SAS Coefficients: ar1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4 0 1.3449 -0.0921 0.0446 0 0 0 0 10.0726 s.e. 0 9.6455 0.1879 0.0757 0 0 0 0 0.1757 T1-MA5 T1-MA6 5.9908 2.7742 s.e. 1.8903 0.3929

Examination of the residuals of this model indicated that nt is not white noise but an MA(1) series To fit this model we use the command: > arimax(Sales,order=c(0,0,1), fixed=c(NA,NA,NA,NA,0,0,0,0,NA,NA,NA), xtransf=Adver,transfer=list(c(2,6)),method="ML") To yield the estimates: Coefficients: ma1 intercept T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 -0.7714 -0.2819 -0.7087 0.2617 0 0 0 0 s.e. 0.0477 1.9804 0.0180 0.0181 0 0 0 0 T1-MA4 T1-MA5 T1-MA6 10.0652 12.9035 3.0862 s.e. 0.1188 0.1371 0.1978

The fitted transfer function model The constant term is not significant. To fit a model with constant term 0, we use: > arimax(x = Sales, order = c(0, 0, 1), include.mean = FALSE, fixed = c(NA,NA, NA,0,0,0,0,NA,NA,NA), method="ML",xtransf= Adver,transfer =list(c(2,6)))

The estimated parameters Coefficients: ma1 T1-AR1 T1-AR2 T1-MA0 T1-MA1 T1-MA2 T1-MA3 T1-MA4 T1-MA5 -0.6753 -0.3976 0.1530 0 0 0 0 10.1337 9.2484 s.e. 0.0701 0.0316 0.0324 0 0 0 0 0.1362 0.2645 T1-MA6 3.0240 s.e. 0.3188 The fitted transfer function model