Instructor :Dr. Aamer Iqbal Bhatti ADAPTIVE FILTERS I Instructor :Dr. Aamer Iqbal Bhatti Searching the Performance Surface CHAPTER # 4

Introduction Performance surface for the Adaptive Linear Combiner is quadratic for stationary signals In most of the applications parameters for the error surface are not known and we have to estimate them Problem is to devise algorithms that search the estimated performance surface.

Methods of Searching the Performance Surface Two descent algorithms for search of the optimal weight solution shall be followed a) Newton’s Method b) Method of Steepest Descent Newton’s method is fundamental root finding procedure in mathematics It is comparatively difficult to implement Weights are varied in each iteration

Methods of Searching the Performance Surface Changes in weight are always in the direction of the minimum of the performance surface Method of Steepest Decent is easy to implement In each iteration, weights are varied towards negative gradient of performance surface

Basic Ideas of Gradient Search Method We consider the simplest case where there is only one weight to be adjusted One weight surface is a parabola

Basic Ideas of Gradient Search Method Performance surface for single weight is represented by Problem is to find weight adjustment that causes the mean square error to be minimized

Methods of Searching the Performance Surface Iterative method to find the minimum value for the weight variable Start with an initial guess Measure the slope of the performance surface at this point Choose a new value equal to the initial guess plus an increment proportional to negative of the slope

Methods of Searching the Performance Surface Another new value is obtained and the above procedure is repeated until the minimum is achieved The values obtained by measuring the slope at discrete time points is called the “Gradient Estimate” Use of the negative of the gradient is necessary to proceed “Down Hill”

Gradient Search Algorithm and Solution For single variable the repetitive gradient search procedure can be represented as

Gradient Search Algorithm and Solution Gradient for the single weight is given by Transients and rate of convergence can be analyzed through substitution of the gradient

Gradient Search Algorithm and Solution Above equation is a constant coefficient linear difference equation It can be solved through induction from the first few iterations

Gradient Search Algorithm and Solution Generalized results for the kth iteration The result give the weight variable at any point in the search procedure and thus a solution to the gradient search algorithm

Stability and Rate of Convergence In the solution for the gradient search algorithm equation geometric ratio is given by Search equation is stable if and only if If the condition is met than the algorithm is seen to converge to the optimum solution

Stability and Rate of Convergence The figure depicts the gradient search for different values of r

Stability and Rate of Convergence The effect of the choice of u on r and on the weight iterative procedure are summarized in the following table

The Learning Curve Effect of variations in the adjustment of the weight on mean square error can be observed from For the continuous update of the weights the mean square error becomes

The Learning Curve Mean square error under goes a geometric progression The geometric ratio of the progression is given by This ratio can never be negative , the mean square progression shall never be oscillatory

The learning Curve For the single weight system the figure shows the relaxation of the mean square error from its initial value towards the optimal value For the learning curve for the value of r=0.5

The Learning Curve