1. Lecture 17 Introduction to Eigenvalue Problems
Shang-Hua Teng

2. Eigenvalue Problems
Eigenvalue problems occur in many areas of science and engineering
E.g., Structure analysis
It is important for analyzing numerical and linear algebra algorithms
Impact of roundoff errors and precision requirement
It is widely used in information management and web-search
It is the key ingredient for the analysis of Markov process, sampling algorithms, and various approximation algorithms in computer science

3. Eigenvalues and Eigenvectors
Standard Eigenvalue Problem: Given an n by n matrix A, find a scalar l and nonzero vector x such that
A x = l x
l is eigenvalue, and x is corresponding eigenvector

4. Spectrum of Matrices Spectrum(A) = l(A) = set of all eigenvalues of A
Spectral radius (A) = r (A) = max {|l|: l in l(A)}
Spectral analysis
Spectral methods

5. Geometric Interpretation
Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor
Expansion of contraction factor given by corresponding eigenvalue l
Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions

6. Examples: Eigenvalues and Eigenvectors
Note: x1 and x2 are perpendicular to each other

7. Examples: Eigenvalues and Eigenvectors
Note: x1 and x2 are not perpendicular to each other

8. Examples: Eigenvalues and Eigenvectors
Note: x1 and x2 are perpendicular to each other

9. Examples: Eigenvalues and Eigenvectors
Note: x1 and x2 are perpendicular to each other

10. Examples: Eigenvalues and Eigenvectors
Note: x1 and x2 are not perpendicular to each other
: eigenvalues or eigenvectors may not be real!!!

11. Simple Facts of Eigenvalue Problem
If (l,x) is a eigenvalue-eigenvector pair of A, then for any k, (lk, x) is a eigenvalue-eigenvector pair of Ak.
If (l,x) is a eigenvalue-eigenvector pair of A, then for any c, (cl, x) is a eigenvalue-eigenvector pair of cA.

12. Algebraic Interpretation: Equation for the Eigenvalues
A x = l x
(A - l I ) x = 0
The eigenvectors make up the nullspace of (A – l I ) if we know l.

13. Eigenvalue First If (A - l I ) x = 0 has a nonzero solution, then
A - l I is not invertible
The determinant of A - l I must be zero.

14. Characteristic Equation for Eigenvalues
The number l is an eigenvalue of A if and only if (A - l I ) is singular:
det( A - l I ) = 0

15. Characteristic Polynomial for Eigenvalues
det (A - l I ) = 0 is a polynomial in l of degree at most n.
The spectrum of A is the set of roots of this characteristic polynomial:
Fundamental Theorem of Algebra implies that n by n matrix A always has n eigenvalues, but they need be neither distinct nor real

16. Examples: Characteristic Polynomial
So Spectrum(A) = {1,2}

17. Examples: Characteristic Polynomial
So Spectrum(A) = {2, 4}

18. Examples: Characteristic Polynomial
So Spectrum(A) = {i, -i}

19. A Possible Methods for Solving the Eigenvalue Problems
Compute the characteristic polynomial of A in l by expanding det(A – l I) = 0
Find the roots of the characteristic polynomial
For each eigenvalue l, solve (A – l I) x=0 to find an eigenvector x.

20. Practical Difficulties
Computing eigenvalues using characteristic polynimial is not recommended or used because
Roots of polynomial of degree > 4 cannot always be computed in finite number of steps
A lot of work is needed in computing coefficients of the characteristic polynomial
Computer has round-off errors

21. Examples: Characteristic Polynomial
So Spectrum(A) = {1+e, 1-e}
But in machine, e2 < emachine is equal to 0
So, the algorithm will return
Spectrum(A) = {1,1}

22. Theory and Practice
Characteristic polynomial is a powerful theoretical tool but usually is not useful computationally.

23. Special Matrix What is Spectrum( I )?
Multiplicity is the number of times root appears when polynomial written as product of linear factors
det(I – l I ) = (1-l)n
What is Spectrum( upper or lower triangular matrix )?

24. Bad News
Elimination does not preserve the l’s.

25. Diagonalizing A Matrix
Suppose the n by n matrix A has n linearly independent eigenvectors x1, x2,…, xn.
Eigenvector matrix S: x1, x2,…, xn are columns of S.
Then
L is the eigenvalue matrix

26. Matrix Power Ak S-1AS = L implies A = S LS-1
implies A2 = S LS-1 S LS-1 = S L2S-1
implies Ak = S LkS-1

27. Random walks
How long does it take to get completely lost?

28. Random walks Transition Matrix1 2 3 4 5 6

29. Matrix Powers As if A is a diagonal matrix!!!!
If A is diagonalizable as A = S LS-1 then for any vector u, we can compute Aku efficiently
Solve S c = u
Aku = S LkS-1 S c = S Lk c
As if A is a diagonal matrix!!!!

30. Independent Eigenvectors from Different Eigenvalues
Eigenvectors x1, x2,…, xk that correspond to distinct (all different) eigenvalues are linear independent.
An n by n matrix that has n different eigenvalues (no repeated l’s) must be diagonalizable
Proof: Show that
implies all ci = 0