Properties Of the Quadratic Performance Surface Lecture Three

The Quadratic Performance Surface From derivation of Weiner Hopf; Quadratic Performance Surface is expressed by:  =  min + (W – W*)TR (W – W*) Quadratic Performance Surface is a function of autocorrelation matrix (R) of input vector The Eigenanalysis of the autocorrelation matrix gives information about the characteristics of the Performance Surface

Introduction From the standpoint of Engineering applications eigenvalue problem are among the most important problems in connection with matrices. For a linear discrete time and continuous linear systems matrix A completely determine system stability. The eigenvectors of A forms a very convenient choice for the eigenvectors . Eigenvectors may be used to uncouple the state equations and determine convenient way for the system analysis

Definition Let the domain and range of a linear transformation be D(A) and R(A) within the vector space X. Those vectors xi and scalars λi which satisfy the condition A(xi)= λi xi are called as eigenvectors and eigenvectors respectively. Case for xi is excluded.

Eigenvalues For Linear transformation this becomes Necessary condition for the existence of solution to set of n homogeneous equations is that rank(A- Iλi )<n Which results

If there are p<n distinct roots The integer mi is called the algebraic multiplicity of λi Equation has to be solved for the roots to get the Eigenvalues

Eigenvectors and Eigenvalues Problems Membrane Stretch problem An elastic membrane is stretched such that a point P goes over into point Q Find the principle directions of stretch

Governing equations for the stretch is Eigenvalues for transformation matrix are 2 and 8

Original and Stretched Membrane

Thus the eigenvectors specify dimensions along which the output is directed for the specific value of the input vector. Output is just the integral multiple of input at these values The constant factor is called as the eigenvalue The output is eigenvector corresponding to that eigenvalue

The Quadratic Performance Surface Using Eigen analysis we can get the idea of basis of the performance surfaces on which it is defined Hence analysis of the performance surface is simplified

Normal Form of the Input Correlation Matrix The Eigenvalues of the input autocorrelation matrix R is defined as R Qn = n Qn Where Qn is the nth eigenvector corresponding to n nth eigenvalue The eigenvalues are computed from the following characteristic equation det[R - I] = 0

Normal Form of the Input Correlation Matrix Eigenvector form the basis vectors for the input autocorrelation matrix R. And Eigenvalues are the weights of the vectors. We can write

Normal Form of the Input Correlation Matrix Therefore we can also write RQ = Q or R = QQ-1 This is the normal form of R Where Q = [Q0 Q1 … QL], is the eigenvector matrix And  is a diagonal matrix with eigenvalue as the diagonal entries and is called ‘eigenvalue matrix’

Properties of the Eigenvalues and Eigenvectors As R is a symmetric matrix, the eigenvectors corresponding to distinct eigenvalues are mutually orthogonal. Since R is real, all eigenvalues must be real and greater than or equal to zero The eigenvector matrix Q can be normalized such that QQT = I

Geometrical Significance of the Eigenvectors and Eigenvalues The eigenvectors and the eigenvalues are related to certain properties of the error surface. We know that the error performance surface form a hyperparabolic surface in a space of N dimensions for N-1 weights.

Geometrical Significance of the Eigenvectors and Eigenvalues MSE w1 w0 The hyperparabolic surface of three dimensions for 2 weights

Geometrical Significance of the Eigenvectors and Eigenvalues If we cut the paraboloid with planes parallel to the w0w1-plane, we obtain concentric ellipses corresponding to different values of mean square error.

Geometrical Significance of the Eigenvectors and Eigenvalues w1 w0 Ellipses with different color shades corresponding to different values of mean square error

Geometrical Significance of the Eigenvectors and Eigenvalues From mean square error expression, equation for ellipses it can be written as WTRW – 2PTW = constant Using the Alternate expression for gradient we can also write it as VTRV = constant The general expression for the ellipses in function form can be written as F(V) = VTRV

Geometrical Significance of the Eigenvectors and Eigenvalues A vector normal to the ellipses can be obtained by taking gradient of F The principal axis of the ellipse pass through the origin and therefore is of form V

Geometrical Significance of the Eigenvectors and Eigenvalues Also the principal axis is normal to the ellipses F(V),therefore 2RV’ = V’ [R – (/2)I] V’ = 0 Thus V’ is the principle axis and also the eigenvector of the matrix R.

Geometrical Significance of the Eigenvectors and Eigenvalues The eigenvectors of the input correlation matrix define the principle axes of the error surface

Geometrical Significance of the Eigenvectors and Eigenvalues Take the expression for the mean square error  =  min + VTR V Where V = (W – W*) Replace R by its normal form R = QQ-1 We have  =  min + V’T V’

Geometrical Significance of the Eigenvectors and Eigenvalues The gradient of above expression would be  = 2 V’ = 2[0v’0 1v’1 … Lv’L] To summarize V = (W – W*) can be considered as a translation to a new axis V’ = QTV is the transformation to the principal coordinate system

Geometrical Significance of the Eigenvectors and Eigenvalues The gradient of  along any principal axis is given as Thus the eigenvalues of the input correlation matrix R give the second derivative of the error surface, , with respect to the principal axes of 

Assignment All exercise Questions from Chapter 2 MATLAB questions due in next lab Tuesday