1. Properties Of the Quadratic Performance Surface
Lecture Three

2. The Quadratic Performance Surface
From derivation of Weiner Hopf; Quadratic Performance Surface is expressed by:
 =  min + (W – W*)TR (W – W*)
Quadratic Performance Surface is a function of autocorrelation matrix (R) of input vector
The Eigenanalysis of the autocorrelation matrix gives information about the characteristics of the Performance Surface

3. Introduction
From the standpoint of Engineering applications eigenvalue problem are among the most important problems in connection with matrices.
For a linear discrete time and continuous linear systems matrix A completely determine system stability.
The eigenvectors of A forms a very convenient choice for the eigenvectors .
Eigenvectors may be used to uncouple the state equations and determine convenient way for the system analysis

4. Definition
Let the domain and range of a linear transformation be D(A) and R(A) within the vector space X.
Those vectors xi and scalars λi which satisfy the condition A(xi)= λi xi are called as eigenvectors and eigenvectors respectively.
Case for xi is excluded.

5. Eigenvalues For Linear transformation this becomes
Necessary condition for the existence of solution to set of n homogeneous equations is that rank(A- Iλi )<n
Which results

6. If there are p<n distinct roots
The integer mi is called the algebraic multiplicity of λi
Equation has to be solved for the roots to get the Eigenvalues

7. Eigenvectors and Eigenvalues Problems
Membrane Stretch problem
An elastic membrane is stretched such that a point P goes over into point Q
Find the principle directions of stretch

9. Governing equations for the stretch is
Eigenvalues for transformation matrix are 2 and 8

11. Original and Stretched Membrane

12. Thus the eigenvectors specify dimensions along which the output is directed for the specific value of the input vector.
Output is just the integral multiple of input at these values
The constant factor is called as the eigenvalue
The output is eigenvector corresponding to that eigenvalue

13. The Quadratic Performance Surface
Using Eigen analysis we can get the idea of basis of the performance surfaces on which it is defined
Hence analysis of the performance surface is simplified

14. Normal Form of the Input Correlation Matrix
The Eigenvalues of the input autocorrelation matrix R is defined as
R Qn = n Qn
Where Qn is the nth eigenvector corresponding to n nth eigenvalue
The eigenvalues are computed from the following characteristic equation
det[R - I] = 0

15. Normal Form of the Input Correlation Matrix
Eigenvector form the basis vectors for the input autocorrelation matrix R.
And Eigenvalues are the weights of the vectors.
We can write

16. Normal Form of the Input Correlation Matrix
Therefore we can also write
RQ = Q or R = QQ-1
This is the normal form of R
Where Q = [Q0 Q1 … QL], is the eigenvector matrix
And  is a diagonal matrix with eigenvalue as the diagonal entries and is called ‘eigenvalue matrix’

17. Properties of the Eigenvalues and Eigenvectors
As R is a symmetric matrix, the eigenvectors corresponding to distinct eigenvalues are mutually orthogonal.
Since R is real, all eigenvalues must be real and greater than or equal to zero
The eigenvector matrix Q can be normalized such that QQT = I

18. Geometrical Significance of the Eigenvectors and Eigenvalues
The eigenvectors and the eigenvalues are related to certain properties of the error surface.
We know that the error performance surface form a hyperparabolic surface in a space of N dimensions for N-1 weights.

19. Geometrical Significance of the Eigenvectors and Eigenvalues
MSE w1 w0
The hyperparabolic surface of three dimensions for 2 weights

20. Geometrical Significance of the Eigenvectors and Eigenvalues
If we cut the paraboloid with planes parallel to the w0w1-plane, we obtain concentric ellipses corresponding to different values of mean square error.

21. Geometrical Significance of the Eigenvectors and Eigenvaluesw1 w0
Ellipses with different color shades corresponding to different values of
mean square error

22. Geometrical Significance of the Eigenvectors and Eigenvalues
From mean square error expression, equation for ellipses it can be written as
WTRW – 2PTW = constant
Using the Alternate expression for gradient we can also write it as
VTRV = constant
The general expression for the ellipses in function form can be written as
F(V) = VTRV

23. Geometrical Significance of the Eigenvectors and Eigenvalues
A vector normal to the ellipses can be obtained by taking gradient of F
The principal axis of the ellipse pass through the origin and therefore is of form V

24. Geometrical Significance of the Eigenvectors and Eigenvalues
Also the principal axis is normal to the ellipses F(V),therefore
2RV’ = V’
[R – (/2)I] V’ = 0
Thus V’ is the principle axis and also the eigenvector of the matrix R.

25. Geometrical Significance of the Eigenvectors and Eigenvalues
The eigenvectors of the input correlation matrix define the principle axes of the error surface

26. Geometrical Significance of the Eigenvectors and Eigenvalues
Take the expression for the mean square error
 =  min + VTR V
Where V = (W – W*)
Replace R by its normal form
R = QQ-1
We have
 =  min + V’T V’

27. Geometrical Significance of the Eigenvectors and Eigenvalues
The gradient of above expression would be
 = 2 V’
= 2[0v’0 1v’1 … Lv’L]
To summarize
V = (W – W*) can be considered as a translation to a new axis
V’ = QTV is the transformation to the principal coordinate system

28. Geometrical Significance of the Eigenvectors and Eigenvalues
The gradient of  along any principal axis is given as
Thus the eigenvalues of the input correlation matrix R give the second derivative of the error surface, , with respect to the principal axes of 

29. Assignment All exercise Questions from Chapter 2
MATLAB questions due in next lab Tuesday