Adv DSP Spring-2015 Lecture#9 Optimum Filters (Ch:7) Wiener Filters

Introduction  Estimation of one signal from another is one of the most important problems in signal processing  In many applications, the desired signal is not available or observed directly. Speech, Radar, EEG etc  Desired signal may be noisy and highly distorted  In very simple and idealized environments, it may be possible to design classical filters such as LP,HP or BP, to restore the desired signal from the measured data.

Introduction  These classical filters shall rarely be optimal in the sense of producing the “best” estimate of the signal.  Class of filters called OPTIMUM DIGITAL FILTERS  Two important Types Digital Wiener Filter Discrete Kalman Filter

Wiener Filter  In the 1940’s, Norbert Wiener pioneered research in the problem of designing a filter that would produce the optimum estimate of a signal from a noisy measurement or observation.

Wiener Filter  The Wiener filtering problem, is to design a filter to recover a signal d[n] from noisy measurement  Assuming that both d[n] and v[n] are wide-sense stationary random process, Wiener considered the problem of designing the filter that would produces the minimum mean square estimate of d[n] (by using x[n])

Wiener Filter  Thus the error terms are (Mean Square Error)  Problem is to find the filter (filter coefficients, FIR or IIR) that minimizes ξ (minimum mean square error).

Wiener Filter  Depending on how the signals x[n] and d[n] are related to each other, a number of different and important problems may be cast into Wiener filtering framework.  These problems are  Filtering  Smoothing  Prediction  De-convolution

Wiener Filter

The FIR Wiener Filter  Design of an FIR Wiener Filter That will produce the minimum mean-square estimate of a given process d[n] by filtering a set of observations of statistically related process x[n]  It is assumed that x[n] and d[n] are jointly wide-sense stationary with known autocorrelations r x [k] and r d [k], and known cross-correlation r dx [k]

The FIR Wiener Filter  Denoting the unit sample response of the Wiener filter by w[n], and assuming (p-1)st order filter, the system function W(z) is  With x[n] as input to this filter, the output is (using DT convolution)

The FIR Wiener Filter  Wiener filter design problem requires that we find the filter coefficients w[k], that minimize the mean-square error:  Using optimization steps, for k=0,1,…,p-1

Error Taking Partial Derivative for k th value Orthogonality Principle After putting e[n] back

The FIR Wiener Filter  Since x[n] and d[n] are jointly WSS  Set of ‘p’ linear equations in the ‘p’ unknowns w[k] for k=0,1,….,p-1

The FIR Wiener Filter  In matrix form using the fact that autocorrelation sequence is conjugate symmetric r x [k]=r* x [-k] Wiener-Hopf Equations

The FIR Wiener Filter  Wiener-Hopf Equations

The FIR Wiener Filter  The minimum mean square error in the estimate of d[n] is Equal to zero bz of Orthognality

The FIR Wiener Filter  After taking expected values In Vector Notation

The FIR Wiener Filter

Filtering  In the filtering problem, a signal d[n] is to be estimated from a noise (v[n]) corrupted observation x[n]  Assuming that noise has a zero mean and it is uncorrelated with d[n]

Filtering  The cross-correlation between d[n] and x[n] becomes

Filtering  With v[n] and d[n] uncorrelated, it follows  To simplify these equations, specific information about the statistic of the signal and noise are required Example:7.2.1

Linear Prediction  With noise-free observations, linear prediction is concerned with the estimation (prediction) of x[n+1] in terms of linear combination of the current and previous values of x[n]

Linear Prediction  An FIR linear predictor of order ‘p-1’ has the form where w[k] for k=0,1,…,p-1 are the coefficients of the prediction filter.  Linear predictor may be cast into Wiener filtering problem by setting d[n]=x[n+1]

Linear Prediction  Setting up the Wiener-Hopf equations Ex:7.2.2

Linear Prediction in noise  With noise present, a more realistic model for linear prediction is the one in which the signal to be predicted is measured in the presence of noise.

Linear Prediction in noise  Input to Wiener filter is given by  Goal is to design a filter that will estimate x[n+1] in terms of linear combination of ‘p’ previous values of y[n]

Linear Prediction in noise  The Wiener-Hopf equations are  If the noise is uncorrelated with signal x[n], then R y, the autocorrelation matrix for y[n] is

Linear Prediction in noise  The only difference between linear prediction with and without noise is in the autocorrelation matrix for the input signal.  In the case of noise that is uncorrelated with x[n],

Multi-Step Prediction  In one-step linear prediction, x[n+1] is predicted in terms of linear combination of the current and previous values of x[n]  In multi-step prediction, x[n+δ] is predicted in terms of linear combination of the ‘p’ values x[n],x[n-1],…,x[n-p+1]

Multi-Step Prediction  In multi-step prediction  In multi-step prediction, since Positive Integer

Multi-Step Prediction  Wiener-Hopf equations are

Noise Cancellation  The goal of noise canceller is to estimate a signal d[n] from a noise corrupted observation, that is recorded by primary sensor.  Unlike the filtering problem, which requires that the autocorrelation of the noise be known, with noise canceller this information is obtained from a secondary sensor that is placed within the noise field.

Noise Cancellation Primary sensor Secondary sensor

Noise Cancellation  Although the noise measured by secondary sensor, v 2 [n], will be correlated with the noise in the primary sensor v 1 [n], the two will not be same.  Reasons for being not same: Difference in sensor characteristics Difference in the propagation path from noise source to the two sensors.  Since v 1 [n]≠v 2 [n], it is not possible to estimate d[n] by simply subtracting v 2 [n] from x[n]

Noise Cancellation  Noise canceller consists of Wiener filter that is designed to estimate the noise v 1 [n] from the signal received by the secondary sensor  This estimate is then subtracted from the primary signal x[n] to form an estimate of d[n] which is given by

Noise Cancellation  With v 2 [n] as the input to Wiener filter, that is used to estimate the noise v 1 [n], the Wiener-Hopf equations are  R v 2 is the autocorrelation matrix of v 2 [n]  r v 1,v2 is the vector of cross-correlations between desired signal v 1 [n] and Wiener filter input v 2 [n]

Noise Cancellation  The cross-correlation between v 1 [n] and v 2 [n] is  If we assume v 2 [n] is uncorrelated with d[n], then second term is zero, hence

Example:7.2.6