1. AGC DSP AGC DSP Professor A G Constantinides©1 Modern Spectral Estimation Modern Spectral Estimation is based on a priori assumptions on the manner, the observed process has been generated Validity of these assumptions is taken to hold over all possible realisations and to be of infinite temporal extent. Thus limitations of FFT-based methods are circumvented These assumptions may be entirely statistical or deterministic model-based or both.

2. AGC DSP AGC DSP Professor A G Constantinides©2 Modern Spectral Estimation Statistical methods make assumptions on the probabilities pertaining to data generation. Wiener-Hopf, and Bayesian methods are typical examples Model-based deterministic methods assume a linear or a non-linear equation for the input/output process driven by a stochastic or a deterministic signal. Linear Predictive Least SquaresTechniques are typical of this class

3. AGC DSP AGC DSP Professor A G Constantinides©3 Modern Spectral Estimation Main directions are: Least Squares Maximum Entropy

4. AGC DSP AGC DSP Professor A G Constantinides©4 Modern Spectral Estimation An optimisation problem: Measurements: Problem: Find the best FIR model to filter to yield a given signal We need a) order of FIR system b) decide on how to measure “best fit”

5. AGC DSP AGC DSP Professor A G Constantinides©5 Modern Spectral Estimation “Order Estimation” is an area by itself Goodness of fit is another large area Usually: we have some idea beforehand on the order we select an “error criterion” which reasonably reflects reality and is analytically tractable

6. AGC DSP AGC DSP Professor A G Constantinides©6 Modern Spectral Estimation Formulation: Assume FIR order be and unknown filter weights Output of FIR filter is Instantaneous error is

7. AGC DSP AGC DSP Professor A G Constantinides©7 Modern Spectral Estimation The best solution would be when all such errors are zero. However, this may not possible because of many reasons e.g. the order is not correct, the actual model is not FIR, or is not linear, the noise present in the data, etc Hence need to be selected to minimise some measure of the error.

8. AGC DSP AGC DSP Professor A G Constantinides©8 Modern Spectral Estimation Error measure can take many forms We draw a distinction between stochastic and deterministic measures For example (a) Stochastic (b) Deterministic

9. AGC DSP AGC DSP Professor A G Constantinides©9 Modern Spectral Estimation With Problem (a) is known as the Wiener filtering problem Problem(b) is known as the Least Squares problem These problems are also analytically easily tractable

10. AGC DSP AGC DSP Professor A G Constantinides©10 Modern Spectral Estimation Extensive work has been done in these problems in their various forms. The absolute value squared error is Or

11. AGC DSP AGC DSP Professor A G Constantinides©11 Modern Spectral Estimation Where for the stochastic case While for the deterministic case we have the same expressions but Expectations are replaced by Summations.

12. AGC DSP AGC DSP Professor A G Constantinides©12 Modern Spectral Estimation In both cases we have is the crosscorrelation between the measurements (data) and the desired signal is the autocorrelation matrix of the data

13. AGC DSP AGC DSP Professor A G Constantinides©13 Modern Spectral Estimation The autocorrelation matrix for real signals is symmetric, positive definite This is seen, for the stochastic case, from Expanding

14. AGC DSP AGC DSP Professor A G Constantinides©14 Modern Spectral Estimation Differentiating with respect to and setting the result to zero we obtain Or Differentiating again yields the autocorrelation matrix, which is positive definite and hence we have a minimum

15. AGC DSP AGC DSP Professor A G Constantinides©15 Modern Spectral Estimation Differentiating with respect to and setting the result to zero we obtain However,

16. AGC DSP AGC DSP Professor A G Constantinides©16 Modern Spectral Estimation On taking expectations we obtain This is known as the orthogonality condition “At the optimum the error vector is orthogonal to the data”

17. AGC DSP AGC DSP Professor A G Constantinides©17 Modern Spectral Estimation For the stochastic case this solution is known as the Wiener–Hopf solution. For the deterministic case the solution is known as the Yule-Walker solution. The framework of modelling has been FIR or Moving Average (MA). It can be extended to include more involved linear models such as Autoregressive (AR), and ARMA

18. AGC DSP AGC DSP Professor A G Constantinides©18 AR Spectral Estimation This is also known as the Maximum Entropy Method and the Burg Method. Burg solved the problem of extrapolating a given finite set of autocorrelations to an infinite set while keeping the autocorrelation matrix positive semidefinite. In view of the infinite possibile solutions he postulated selecting that which produces the flattest PSD. Equivalently it maximises uncertainty (entropy) or randomness.

19. AGC DSP AGC DSP Professor A G Constantinides©19 Modern Spectral Estimation Thus the problem becomes the constrained optimisation problem Subject to

20. AGC DSP AGC DSP Professor A G Constantinides©20 Modern Spectral Estimation Thus if the PSD of the observations is taken to be that of the output of an AR system driven by a white Gaussian process the problem reduces to finding the parameters of the following model

21. AGC DSP AGC DSP Professor A G Constantinides©21 Modern Spectral Estimation Where N is the number or poles. are obtainable in the autocorrelation method from (N+1)X(N+1)

22. AGC DSP AGC DSP Professor A G Constantinides©22 Modern Spectral Estimation Where the autocorrelation sequence is estimated as The signal above is extended by padding with zeros whever the argument demands more samples.

23. AGC DSP AGC DSP Professor A G Constantinides©23 Modern Spectral Estimation If we take only the entral part of the autocorrelation matrix containing no zero padding then we have the Covariance Method. The signal vector in both cases may be windowed prior to the computations.

24. AGC DSP AGC DSP Professor A G Constantinides©24 Modern Spectral Estimation While the Burg method is a decided improvement over the non-parametric methods, it has several disadvantages 1) Exhibits Spectral Line Splitting particularly at high SNR 2) For high order systems introduces spurious spectral peaks 3) In estimating sinusoids in noise it shows a bias dependent on the initial sinusoid phases

25. AGC DSP AGC DSP Professor A G Constantinides©25 Linear Prediction Assume From the measurements in conjunction with the assumed model we can write

26. AGC DSP AGC DSP Professor A G Constantinides©26 Linear Prediction

27. AGC DSP AGC DSP Professor A G Constantinides©27 Linear Prediction The above can be seen as solving an underlying AR prediction problem In a compact form The solution to this can be cast as an optimisation problem

28. AGC DSP AGC DSP Professor A G Constantinides©28 Modern Spectral Estimation Form the error function to be minimised as the difference between the two sides of the equation Then seek solution as The solution is (normal equations)

29. AGC DSP AGC DSP Professor A G Constantinides©29 Modern Spectral Estimation The autocorrelation matrix is computed directly from the given signal. Hence we obtain Again in the covariance method, only a subset of the total possible rows used in the autocorrelation method, is taken.