Presentation on theme: "ELG5377 Adaptive Signal Processing"— Presentation transcript:
1 ELG5377 Adaptive Signal Processing Lecture 7: Stochastic Models: Moving Average (MA), Autoregressive (AR) and ARMA
2 Stochastic ModelsThe term model is used for any hypothesis that may be applied to describe the hidden laws that govern the generation of physical data.A time series u(k) consisting of highly correlated observations may be generated by applying a series of statistically independent “shocks” to a linear filter.The shocks are drawn from a fixed distribution (usually Gaussian) with zero mean and constant variance.This time series of shocks is v(k).E[v(k)v*(k-i)]=sv2 for i = 0 and 0 for i ≠ 0.Discrete-timeLinear Filterv(k)u(k)
3 Types of Models u(k) = Sbi*v(k-i) Moving Average Model FIR filter implementationSai*u(k-i) = v(k) (usually a0 = 1)u(k) = v(k) – Sai*u(k-i) (i ≠ 0)Autoregressive ModelIIR filter implementationARMASai*u(k-i) = Sbi*v(k-i)Cascade of FIR and IIR filters
4 Autoregressive Models The time series u(k), u(k-1), …, u(k-M) represents the realization of an autoregressive model of order M if it satisfies the difference equationu(k) + a1*u(k-1)+…+aM*u(k-M) = v(k).Oru(k) = w1*u(k-1)+…+wM*u(k-M) = v(k).Where wi = -ai.It is called an autoregressive model since u(k) is regressed on previous values of itself.
5 Correlation function of an asymptotically stationary AR process Starting with Sai*u(k-i) = v(k)Multiply both sides by u*(k-l) and take the expectation.E[Sai*u(k-i)u*(k-l)] = E[v(k)u*(k-l)]The right side of the equation is 0 for l > 0.The left side is Sai*r(l-i)Therefore for l > 0, Sai*r(l-i) = 0 (for i = 0, 1, … M) (a0 = 1)Or we can write this asr(l) = Swi*r(l-i) = 0 (for i = 1,2, … M)Want to find w1, w2, … wM.
6 Coefficients of AR model Recall that r(-x) = r*(x). If we take the complex conjugateof both sides, we getor r = Rw
7 Coefficients of AR model 2 r = [r*(1) r*(2) … r*(M)]T and w=[w1 w2 … wM]T.R is the M×M correlation matrix of u(k).Therefore w = R-1r.a0 = 1 and ai = -wi.Next, let l = 0.E[v(k)u*(k)] = E[v(k)(Swi*u(k-i)+v(k))*] = E[v(k)v*(k)] = sv2.The right side of the equation becomes Sair(i).Therefore:
8 AR model exampleFind a third order AR model that produces a process with the following correlation functionr(i) = sinc(i/2)SolutionM = 3.r(0) = 1, r(1) = 0.637, r(2) = 0, r(3) =
9 AR model example continued w= R-1r.w = [1.552, , 0.703]T.a0 = 1, a1 = , a2 = and a3 =sv2 = Sair(i) = 0.16v(k)u(k)++w1+w2w3
10 Applying autoregressive models to nonstationary systems Let us consider a first order autoregressive model.w(k) = b1w(k-1)+v(k)We need r(0) and r(1).b1 = r(1)/r(0).sv2 = r(0)-r2(1)/r(0).Next let us consider the relationship between x(k) and d(k) in a stationary system.d(k) = yo(k)+eo(k) = woHx(k)+eo(k).Suppose that wo is time varying. Then the cross correlation between d(k) and x(k) would also be time-varying.Nonstationary system.Can represent as d(k) = woH(k)x(k)+eo(k).where wo(k)=awo(k-1)+w(k)