Lecture 13 - Eigen-analysis CVEN 302 July 1, 2002

Lecture’s Goals –Shift Method –Inverse Power Method –Accelerated Power Method –QR Factorization –Householder –Hessenberg Method

Shift method It is possible to obtain another eigenvalue from the set equations by using a technique known as shifting the matrix. Subtract the a vector from each side, thereby changing the maximum eigenvalue

Shift method The eigenvalue, s, is the maximum value of the matrix A. The matrix is rewritten in a form. Use the Power method to obtain the largest eigenvalue of [B].

Example of Shift Method Consider the follow matrix A Assume an arbitrary vector x 0 = { 1 1 1} T

Example of Shift Method Multiply the matrix by the matrix [A] by {x} Normalize the result of the product

Example of Shift Method Continue with the iteration and the final value is = -5. However, to get the true you need to shift back by:

Inverse Power Method The inverse method is similar to the power method, except that it finds the smallest eigenvalue. Using the following technique.

Inverse Power Method The algorithm is the same as the Power method and the “eigenvector” is not the eigenvector for the smallest eigenvalue. To obtain the smallest eigenvalue from the power method.

Inverse Power Method The inverse algorithm use the technique avoids calculating the inverse matrix and uses a LU decomposition to find the {x} vector.

Example The matrix is defined as:

Matlab Program There are set of programs Power and InversePower. The InversePower(A, x 0,iter,tol) does the inverse method.

Accelerated Power Method The Power method can be accelerated by using the Rayleigh Quotient instead of the largest w k value. The Rayleigh Quotient is defined as:

Accelerated Power Method The values of the next z term is defined as: The Power method is adapted to use the new value.

Example of Accelerated Power Method Consider the follow matrix A Assume an arbitrary vector x 0 = { 1 1 1} T

Example of Accelerated Power Method Multiply the matrix by the matrix [A] by {x}

Example of Accelerated Power Method Multiply the matrix by the matrix [A] by {x}

Example of Accelerated Power Method

And so on...

QR Factorization The technique can be used to find the eigenvalue using a successive iteration using Householder transformation to find an equivalent matrix to [A] having an eigenvalues on the diagonal

QR Factorization Another form of factorization A = Q*R Produces an orthogonal matrix (“Q”) and a right upper triangular matrix (“R”) Orthogonal matrix - inverse is transpose

Why do we care? We can use Q and R to find eigenvalues 1. Get Q and R (A = Q*R) 2. Let A = R*Q 3. Diagonal elements of A are eigenvalue approximations 4. Iterate until converged QR Factorization Note: QR eigenvalue method gives all eigenvalues simultaneously, not just the dominant

In practice, QR factorization on any given matrix requires a number of steps First transform A into Hessenberg form Hessenberg matrix - upper triangular plus first sub-diagonal Special properties of Hessenberg matrix make it easier to find Q, R, and eigenvalues QR Eigenvalue Method

QR Factorization Construction of QR Factorization

QR Factorization Use Householder reflections and given rotations to reduce certain elements of a vector to zero. Use QR factorization that preserve the eigenvalues. The eigenvalues of the transformed matrix are much easier to obtain.

Jordan Canonical Form Any square matrix is orthogonally similar to a triangular matrix with the eigenvalues on the diagonal

Similarity Transformation Transformation of the matrix A of the form H -1 AH is known as similarity transformation. A real matrix Q is orthogonal if Q T Q = I. If Q is orthogonal, then A and Q -1 AQ are said to be orthogonally similar The eigenvalues are preserved under the similarity transformation.

Upper Triangular Matrix The diagonal elements R ii of the upper triangular matrix R are the eigenvalues

Householder Reflector Householder reflector is a matrix of the form It is straightforward to verify that Q is symmetric and orthogonal

Householder Matrix Householder matrix reduces z k+1,…,z n to zero To achieve the above operation, v must be a linear combination of x and e k

Householder Transformation

Householder matrix Corollary (k th Householder matrix): Let A be an nxn matrix and x any vector. If k is an integer with 1< k<n-1 we can construct a vector w (k) and matrix H (k) = I - 2w (k) w’ (k) so that

Householder matrix Define the value  so that The vector w is found by Choose  = sign(x k )g to reduce round-off error

Householder Matrices

Example: Householder Matrix

Basic QR Factorization [A] = [Q] [R] [Q] is orthogonal, Q T Q = I [R] is upper triangular QR factorization using Householder matrices Q = H (1) H (2) ….H (n-1)

Example: QR Factorization

Similarity transformation B = Q T AQ preserve the eigenvalues QR Factorization QR = A

Finding Eigenvalues Using QR Factorization Generate a sequence A (m) that are orthogonally similar to A Use Householder transformation H -1 AH the iterates converge to an upper triangular matrix with the eigenvalues on the diagonal Find all eigenvalues simultaneously!

QR Eigenvalue Method QR factorization: A = QR Similarity transformation: A (new) = RQ

Example: QR Eigenvalue

» A=[1 2 -1; ; ] A = » [Q,R]=QR_factor(A) Q = R = » e=QR_eig(A,6); A = A = A = A = A = A = e = MATLAB Example QR factorization eigenvalue

Improved QR Method Using similarity transformation to form an upper Hessenberg Matrix (upper triangular matrix & one nonzero band below diagonal). More efficient to form Hessenberg matrix without explicitly forming the Householder matrices (not given in textbook). function A = Hessenberg(A) [n,nn] = size(A); for k = 1:n-2 H = Householder(A(:,k),k+1); A = H*A*H; end

» A=[1 2 -1; ; ] A = » [Q,R]=QR_factor_g(A) Q = R = » e=QR_eig_g(A,6); A = A = Improved QR Method A = A = A = A = e = » eig(A) ans = Hessenberg matrix MATLAB function eigenvalue

Summary Single value eigen-analysis –Power Method –Shifting technique –Inverse Power Method QR Factorization –Householder matrix –Hessenberg matrix

Homework Check the Homework webpage