1. MA2213 Lecture 8 Eigenvectors

2. Application of Eigenvectors Vufoil 18, lecture 7 : The Fibonacci sequence satisfies

3. Fibonacci Ratio Sequence

5. Another Biomathematics Application Leonardo da Pisa, better known as Fibonacci, invented his famous sequence to compute the reproductive success of rabbits* Similar sequences describe frequencies in males, females of a sex-linked gene. For genes (2 alleles) carried in the X chromosome** *page i, ** pages 10-12 in The Theory of Evolution and Dynamical Systems,J. Hofbauer and K. Sigmund, 1984. The solution has the form where

6. Eigenvector Problem (pages 333-351) Recall that if is a square matrix then a nonzero vector if is an eigenvector corresponding to the Eigenvectors and eigenvalues arise in biomathematics where they describe growth and population genetics eigenvalue They arise in physical problems, especially those that involve vibrations in which eigenvalues are related to vibration frequencies They arise in numerical solution of linear equations because they determine convergence properties

7. Example 7.2.1 pages 333-334 For the eigenvalue-eigenvector pairs are We observe that every (column) vector and where

8. Example 7.2.1 pages 333-334 Therefore, since x  Ax is a linear transformation and since We can repeat this process to obtain are eigenvectors Question What happens as ?

9. Example 7.2.1 pages 333-334 General Principle : If a vector v can be expressed as a linear combination of eigenvectors of a matrix A, then it is very easy to compute Av It is possible to express every vector as a linear combination of eigenvectors of an n by n matrix A iff either of the following equivalent conditions is satisfied : (i) there exists a basis consisting of eigenvectors of A (ii) the sum of dimensions of eigenspaces of A = n Question Does this condition hold for? Question What special form does this matrix have ?

10. Example 7.2.1 pages 333-334 The characteristic polynomial of is 2 is the (only) eigenvalue, it has algebraic multiplicity 2 so the eigenspace for eigenvalue 5 has dimension 1 the eigenvalue 5 is said to have geometric multiplicity 1 Question What are alg.&geom. mult. in Example 7.2.7 ?

11. Characteristic Polynomials pp. 335-337 Example 7.22 (p. 335) The eigenvalue-eigenvector pairs of the matrix in Example 7.2.1 are corresponding eigenvectors Question What is the equation for?

12. Eigenvalues of Symmetric Matrices The following real symmetric matrices that we studied have real eigenvalues and eigenvectors corresponding to distinct eigenvectors are orthogonal. Question What are the eigenvalues of these matrices ? Question What are the corresponding eigenvectors ? Question Compute their scalar products

13. Eigenvalues of Symmetric Matrices Theorem 1. All eigenvalues of real symmetric matrices Proof For a matrix are real valued. with complex (or real) entries let denote the matrix whose entries are the complex conjugates of the entries of Question Proveis real (all entries are real) iff Question Prove Assume that and observe that therefore and

14. Eigenvalues of Symmetric Matrices Theorem 2. Eigenvectors of a real symmetric matrix that Proof Assume that correspond to distinct eigenvalues are orthogonal. Then compute and observe that

15. Orthogonal Matrices so Definition A matrixis orthogonal if Ifis orthogonal then therefore eitheror is nonsingular and has an inversehence soExamples

16. Permutation Matrices Definition A matrix matrix if there exists a function (called a permutation) is called a permutation that is 1-to-1 (and therefore onto) such that Examples Question Why is every permutation matrix orthogonal ?

17. Eigenvalues of Symmetric Matrices Theorem 7.2.4 pages 337-338 If of is symmetric then there exists a set eigenvalue-eigenvector pairs Proof Uses Theorems 1 and 2 and a little linear algebra. Choose eigenvectors so that construct matrices and observe that

18. MATLAB EIG Command >> help eig EIG Eigenvalues and eigenvectors. E = EIG(X) is a vector containing the eigenvalues of a square matrix X. [V,D] = EIG(X) produces a diagonal matrix D of eigenvalues and a full matrix V whose columns are the corresponding eigenvectors so that X*V = V*D. [V,D] = EIG(X,'nobalance') performs the computation with balancing disabled, which sometimes gives more accurate results for certain problems with unusual scaling. If X is symmetric, EIG(X,'nobalance') is ignored since X is already balanced. E = EIG(A,B) is a vector containing the generalized eigenvalues of square matrices A and B. [V,D] = EIG(A,B) produces a diagonal matrix D of generalized eigenvalues and a full matrix V whose columns are the corresponding eigenvectors so that A*V = B*V*D. EIG(A,B,'chol') is the same as EIG(A,B) for symmetric A and symmetric positive definite B. It computes the generalized eigenvalues of A and B using the Cholesky factorization of B. EIG(A,B,'qz') ignores the symmetry of A and B and uses the QZ algorithm. In general, the two algorithms return the same result, however using the QZ algorithm may be more stable for certain problems. The flag is ignored when A and B are not symmetric. See also CONDEIG, EIGS.

19. MATLAB EIG Command >> A = [-7 13 -16;13 -10 13;-16 13 -7] A = -7 13 -16 13 -10 13 -16 13 -7 >> [U,D] = eig(A); >> U U = -0.5774 0.4082 0.7071 0.5774 0.8165 -0.0000 -0.5774 0.4082 -0.7071 >> D D = -36.0000 0 0 0 3.0000 0 0 0 9.0000 Example 7.2.3 page 336 >> A*U ans = 20.7846 1.2247 6.3640 -20.7846 2.4495 -0.0000 20.7846 1.2247 -6.3640 >> U*D ans = 20.7846 1.2247 6.3640 -20.7846 2.4495 -0.0000 20.7846 1.2247 -6.3640

20. Positive Definite Symmetric Matrices Theorem 4 A symmetric matrix is [lec4,slide24] (semi) positive definite iff all of its eigenvalues Proof Letbe the orthogonal, diagonal matrices on the previous page that satisfy Then for every whereSinceis nonsingular therefore is (semi) positive definite iff Clearly this condition holds iff

21. Singular Value Decomposition Theorem 3 If exist orthogogonal matrices wheresuch that and Singular Values = sqrt eig has the form then there Proof Outline Choose so and are diagonal, then satisfies try to finish

22. MATLAB SVD Command >> help svd SVD Singular value decomposition. [U,S,V] = SVD(X) produces a diagonal matrix S, of the same dimension as X and with nonnegative diagonal elements in decreasing order, and unitary matrices U and V so that X =U*S*V'. S = SVD(X) returns a vector containing the singular values. [U,S,V] = SVD(X,0) produces the "economy size“ decomposition. If X is m-by-n with m > n, then only the first n columns of U are computed and S is n-by-n. See also SVDS, GSVD.

23. MATLAB SVD Command >> M = [ 0 1; 0.5 0.5 ] M = 0 1.0000 0.5000 0.5000 >> [U,S,V] = svd(M) U = -0.8507 -0.5257 -0.5257 0.8507 S = 1.1441 0 0 0.4370 V = -0.2298 0.9732 -0.9732 -0.2298 >> U*S*V' ans = 0.0000 1.0000 0.5000 0.5000

24. SVD Algebra

25. SVD Geometry

27. Square Roots Theorem 5 A symmetric positive definite matrix has a symmetric positive definite ‘square root’. Proof Letbe the orthogonal, diagonal matrices on the previous page that satisfy Then construct the matrices and observe that is symmetric positive definite and satisfies

28. Polar Decomposition Theorem 6 Every nonsingular matrix can be factored as Proof Construct where and positive definite and is symmetric and positive definite. Let is symmetric is orthogonal. and observe that be symmetric positive definite and satisfy and construct Then and clearly

29. Löwdin Orthonormalization http://www.quantum-chemistry-history.com/Lowdin1.htm Proof Start with (1) Per-Olov Löwdin, On the Non-Orthogonality Problem Connected with the use of Atomic Wave Functions in the Theory of Molecules and Crystals, J. Chem. Phys. 18, 367-370 (1950). in an inner product space (assumed to be linearly independent), compute the Gramm matrix Sinceis symmetric and positive definite, Theorem 5 gives (and provides a method to compute) a matrix that is symmetric and positive definite and Then are orthonormal.

30. The Power Method pages 340-345 Finds the eigenvalue with largest absolute value of a matrix whose eigenvalues satisfy Step 1 Compute a vector with random entries Step 2 Compute Step 3 Compute ( recall that) Step 4 Compute and Thenwith and Repeat

31. The Inverse Power Method Result If is an eigevector of corresponding to eigenvalue and then is an eigenvector of corresponding to eigenvalueFurthermore, ifthen is an eigenvector of eigenvalue corresponding to Definition The inverse power method is the power method applied to the matrix It can find the eigenvalue-eigenvector pair if there is one eigenvalue that has smallest absolute value.

32. Inverse Power Method With Shifts Computes eigenvalue Step 1 Apply 1 or more interations of the power method Step 2 Compute using the matrix and iterate. Then to estimate an eigenvalue - eigenvector pair closest toof and a corresponding eigenvector - better estimate of Step 3 Apply 1 or more interations of the power method using the matrix to estimate an eigenvalue - eigenvector pair with cubic rate of convergence !

33. Unitary and Hermitian Matrices Definition The adjoint of a matrix is the matrix Example Definition A matrix is unitary if Definition A matrixis hermitian if Super Theorem : All previous theorems true for complex matrices if orthogonal is replaced by unitary, symmetric by hermitian, and old with new (semi) positive definite. Definition A matrix is (semi) positive definite if (or self-adjoint)

34. Homework Due Tutorial 5 (Week 11, 29 Oct – 2 Nov) 1. Do Problem 1 on page 348. 2. Read Convergence of the Power Method (pages 342-346) and do Problem 16 on page 350. 3. Do problem 19 on pages 350-351. 4. Estimate eigenvalue-eigenvector pairs of the matrix M using the power and inverse power methods – use 4 iterations and compute errors 5. Compute the eigenvalue-eigenvector pairs of the orthogonal matrix O 6. Prove that the vectorsdefined at the bottom of slide 29 are orthonormal by computing their inner products

35. Extra Fun and Adventure We have discussed several matrix decompositions : LUEigenvectorPolarSingular Value Find out about other matrix decompositions. How are they derived / computed ? What are their applications ?