1. Instructor :Dr. Aamer Iqbal Bhatti
ADAPTIVE FILTERS I
Instructor :Dr. Aamer Iqbal Bhatti
Searching the Performance Surface
CHAPTER # 4

2. Introduction
Performance surface for the Adaptive Linear Combiner is quadratic for stationary signals
In most of the applications parameters for the error surface are not known and we have to estimate them
Problem is to devise algorithms that search the estimated performance surface.

3. Methods of Searching the Performance Surface
Two descent algorithms for search of the optimal weight solution shall be followed
a) Newton’s Method
b) Method of Steepest Descent
Newton’s method is fundamental root finding procedure in mathematics
It is comparatively difficult to implement
Weights are varied in each iteration

4. Methods of Searching the Performance Surface
Changes in weight are always in the direction of the minimum of the performance surface
Method of Steepest Decent is easy to implement
In each iteration, weights are varied towards negative gradient of performance surface

5. Basic Ideas of Gradient Search Method
We consider the simplest case where there is only one weight to be adjusted
One weight surface is a parabola

6. Basic Ideas of Gradient Search Method
Performance surface for single weight is represented by
Problem is to find weight adjustment that causes the mean square error to be minimized

7. Methods of Searching the Performance Surface
Iterative method to find the minimum value for the weight variable
Start with an initial guess
Measure the slope of the performance surface at this point
Choose a new value equal to the initial guess plus an increment proportional to negative of the slope

8. Methods of Searching the Performance Surface
Another new value is obtained and the above procedure is repeated until the minimum is achieved
The values obtained by measuring the slope at discrete time points is called the “Gradient Estimate”
Use of the negative of the gradient is necessary to proceed “Down Hill”

9. Gradient Search Algorithm and Solution
For single variable the repetitive gradient search procedure can be represented as

10. Gradient Search Algorithm and Solution
Gradient for the single weight is given by
Transients and rate of convergence can be analyzed through substitution of the gradient

11. Gradient Search Algorithm and Solution
Above equation is a constant coefficient linear difference equation
It can be solved through induction from the first few iterations

12. Gradient Search Algorithm and Solution
Generalized results for the kth iteration
The result give the weight variable at any point in the search procedure and thus a solution to the gradient search algorithm

13. Stability and Rate of Convergence
In the solution for the gradient search algorithm equation geometric ratio is given by
Search equation is stable if and only if
If the condition is met than the algorithm is seen to converge to the optimum solution

14. Stability and Rate of Convergence
The figure depicts the gradient search for different values of r

15. Stability and Rate of Convergence
The effect of the choice of u on r and on the weight iterative procedure are summarized in the following table

16. The Learning Curve
Effect of variations in the adjustment of the weight on mean square error can be observed from
For the continuous update of the weights the mean square error becomes

17. The Learning Curve
Mean square error under goes a geometric progression
The geometric ratio of the progression is given by
This ratio can never be negative , the mean square progression shall never be oscillatory

18. The learning Curve
For the single weight system the figure shows the relaxation of the mean square error from its initial value towards the optimal value
For the learning curve for the value of r=0.5

19. The Learning Curve