1. 1 Chapter 8 – Symmetric Matrices and Quadratic Forms Outline 8.1 Symmetric Matrices 8.2Quardratic Forms 8.3Singular ValuesSymmetric MatricesQuardratic FormsSingular Values

2. 2 8.1 Symmetric Matrices (Spectral theorem) A matrix A is orthogonally diagonalizable (i.e., there is an orthogonal S such that S -1 AS=S T AS is diagonal) if and only if A is symmetric (i.e., A T =A). Consider a symmetric matrix A. If and are eigenvectors of A with distinct eigenvalues λ 1 and λ 2, then ; that is, is orthogonal to. A symmetric n×n matrix A has n real eigenvalues if they are counted with their algebraic multiplicities.

3. 3 Example 1 If A is orthogonally diagonalizable, what is the relationship between A T and A? (sol) –We have S -1 AS=D or A=SDS -1 =SDS T, for an orthogonal S and a diagonal D. Then A T =(SDS T ) T =SD T S T =SDS T =A. –We find that A is symmetric: A T =A.

4. 4 Example 2 For the symmetric matrix find an orthogonal S such that S -1 AS is diagonal. (sol) –We will first find an eigenbasis. The eigenvalues of A are 3 and 8, with corresponding eigenvectors respectively.

5. 5 Example 2 (II) –Note that the two eigenspaces, E 3 and E 8, are perpendicular. Therefore, we can find an orthonormal eigenbasis simply by dividing the given eigenvectors by their lengths: –If we define the orthogonal matrix –then S -1 AS will be diagonal, namely,

6. 6 Example 3 For the symmetric matrix find an orthogonal S such that S -1 AS is diagonal. (sol) –The eigenvalues are 0 and 3, with –Note that the two eigenspaces are indeed perpendicular to one another. –We can construct an orthonormal eigenbasis for A by picking an orthonormal basis of each eigenspace.

7. 7 Example 3 (II) –In Figure 3, the vectors form an orthonormal basis of E 0, and is a unit vector in E 3. Then, is an orthonormal eigenbasis for A. We can let to diagonalize A orthogonally. –If we apply the Gram–Schmidt process to the vectors –spanning E 0, we find –The computations are left as an exercise. For E 3, we get –Therefore, the orthogonal matrix –diagonalizes the matrix A:

8. 8 Orthogonal Diagonalization of a Symmetric Matrix A Find the eigenvalues of A, and find a basis of each eigenspace. Using the Gram–Schmidt process, find an orthonormal basis of each eigenspace. Form an orthonormal eigenbasis for A by combining the vectors you found in part (b), and let S is orthogonal, and S -1 AS will be diagonal.

9. 9 Example 4 Consider an invertible symmetric 2×2 matrix A. Show that the linear transformation T(x)=Ax maps the unit circle into an ellipse, and find the lengths of the semimajor and the semiminor axes of this ellipse in terms of the eigenvalues of A. Compare this with Exercise 2.2.50. (sol) –The spectral theorem tells us that there is an orthonormal eigenbasis for T, with associated real eigenvalues λ 1 and λ 2. Suppose that |λ 1 | ≥ |λ 2 |. These eigenvalues will be nonzero, since A is invertible. The unit circle in R 2 consists of all vectors of the form

10. 10 Example 4 (II) –The image of the unit circle consists of the vectors –an ellipse whose semimajor axis has the length, while the length of the semiminor axis is. –In the example illustrated in Figure 4, the eigenvalue λ 1 is positive, and λ 2 is negative.

11. 11 8.2 Quardratic Forms A function q(x 1, x 2,..., x n ) from R n to R is called a quadratic form if it is a linear combination of functions of the form x i x j (where i and j may be equal). A quadratic form can be written as for a symmetric n × n matrix A. Consider a quadratic form from R n to R. Let B be an orthonormal eigenbasis for A, with associated eigenvalues λ 1,..., λ n. Then where the c i are the coordinates of with respect to B.

12. 12 Example 1 Consider the function q(x 1, x 2 ) = 8x 1 2 -4x 1 x 2 +5x 2 2 from R 2 to R. Determine whether q(0,0)=0 is the global maximum, the global minimum, or neither. Recall that q(0,0) is called the global (or absolute) minimum if q(0,0)≤q(x 1, x 2 ) for all real numbers x 1, x 2 ; the global maximum is defined analogously. (sol) – There are a number of ways to do this problem, some of which you may have seen in a previous course. Here we present an approach based on matrix techniques. We will first develop some theory, and then do the example.

13. 13 Example 1 (II) –Note that we can write –More succinctly, we can write or –The matrix A is symmetric by construction. We find with associated eigenvalues λ 1 = 9 and λ 2 = 4. –If we write, we can express the value of the function as follows: (Recall that since is an orthonormal basis of R 2.)

14. 14 Example 1 (III) –The formula shows that for all nonzero, because at least one of the terms is positive. –Thus q(0,0)= 0 is the global minimum of the function. The preceding work shows that the c 1 –c 2 coordinate system defined by an orthonormal eigenbasis for A is “well adjusted” to the function q. The formula is easier to work with than the original formula because no term involves c 1 c 2 :

15. 15 Example 2 Consider the quadratic form Find a symmetric matrix A such that for all in R 3. (sol) –We let –Therefore,

16. 16 Positive Definite Quadratic Forms Consider a quadratic form, where A is a symmetric n×n matrix. We say that A is positive definite if is positive for all nonzero x in R n, and we call A positive semidefinite if, for all x in R n. Negative definite and negative semidefinite symmetric matrices are defined analogously. Finally, we call A indefinite if q takes positive as well as negative values. A symmetric matrix A is positive definite if (and only if) all of its eigenvalues are positive. The matrix A is positive semidefinite if (and only if) all of its eigenvalues are positive or zero. Consider a symmetric n×n matrix A. For m=1,...,n, let A (m) be the m×m matrix obtained by omitting all rows and columns of A past the mth. These matrices A (m) are called the principal submatrices of A. The matrix A is positive definite if (and only if) det(A (m) )>0, for all m=1,...,n.

17. 17 Example 3 Consider an m×n matrix A. Show that the function is a quadratic form, find its matrix, and determine its definiteness. (sol) –We can write This shows that q is a quadratic form, with matrix A T A. This quadratic form is positive semidefinite, because for all vectors in R n. –Note that if and only if x is in the kernel of A. Therefore, the quadratic form is positive definite if and only if

18. 18 Example 4 Sketch the curve (sol) –We found that we can write this equation as where c 1, c 2 are the coordinates of with respect to the orthonormal eigenbasis for We sketch this ellipse in Figure 4. –The c 1 - and the c 2 -axes are called the principal axes of the quadratic form –Note that these are the eigenspaces of the matrix of the quadratic form.

19. 19 Principal Axes Consider a quadratic form, where A is a symmetric n×n matrix with n distinct eigenvalues. Then the eigenspaces of A are called the principal axes of q. (Note that these eigenspaces will be one-dimensional.) Consider the curve C in R 2 defined by Let λ 1 and λ 2 be the eigenvalues of the matrix of q. both λ 1 and λ 2 are positive, then C is an ellipse. If there is a positive and a negative eigenvalue, then C is a hyperbola.

20. 20 8.3 Singular Values Show that if is a linear transformation from R 2 to R 2, then there are two orthogonal unit vectors in R 2 such that vectors are orthogonal as well (although not necessarily unit vectors). (sol) –Following the hint, we first note that matrix A T A is symmetric, since (A T A) T =A T (A T ) T =A T A. The spectral theorem tells us that there is an orthonormal eigenbasis for A T A, with associated eigenvalues λ 1, λ 2. We can verify that vectors and are orthogonal, as claimed:

21. 21 Example 2 Consider the linear transformation where –Find an orthonormal basis of R 2 such that vectors are orthogonal. –Show that the image of the unit circle under transformation L is an ellipse. Find the lengths of the two semiaxes of this ellipse, in terms of the eigenvalues of matrix A T A. (sol) –We will find an orthonormal eigenbasis for matrix A T A: –The characteristic polynomial of A T A is λ 2 -125λ+2500=(λ-100)(λ-25), so that the eigenvalues of A T A are λ 1 =100 and λ 2 =25.

22. 22 Example 2 (II) –To find an orthonormal basis, we need to multiply these vectors by the reciprocals of their lengths: –The unit circle consists of the vectors of the form, and the image of the unit circle consists of the vectors. This image is the ellipse whose semimajor and semiminor axes are. What are the lengths of these axes? Likewise, Thus, –We can compute the lengths of vectors directly, of course, but the way we did it before is more informative. For example, so that

23. 23 Example 2 (III)

24. 24 Singular Values The singular values of an m×n matrix A are the square roots of the eigenvalues of the symmetric n× n matrix A T A, listed with their algebraic multiplicities. It is customary to denote the singular values by σ 1, σ 2,..., σ n, and to list them in σ 1 ≥σ 2 ≥···≥σ n. Let be an invertible linear transformation from R 2 to R 2. The image of the unit circle under L is an ellipse E. The lengths of the semimajor and the semiminor axes of E are the singular values σ 1 and σ 2 of A, respectively. Let be a linear transformation from R n to R m. Then there is an orthonormal basis of R n such that –Vectors are orthogonal, and –The lengths of vectors are the singular values σ 1, σ 2,..., σ n of matrix A. –To construct, find an orthonormal eigenbasis for matrix A T A. Make sure that the corresponding eigenvalues λ 1, λ 2,..., λ n appear in descending order: λ 1 ≥λ 2 ≥···≥λ n.

25. 25 Example 3 Consider the linear transformation –Find the singular values of A. –Find orthonormal vectors in R 3 such that are orthogonal. –Sketch and describe the image of the unit sphere under the transformation L. (sol) – –The eigenvalues are λ 1 =3, λ 2 =1, λ 3 =0. The singular values of A are

26. 26 Example 3 (II) –Find an orthonormal eigenbasis v 1, v 2, v 3 for A T A: –We compute and check orthogonality: –We can also check that the length of is σ i :

27. 27 Example 3 (III) –The unit sphere in R 3 consists of all vectors of the form –The image of the unit sphere consists of the vectors

28. 28 Singular-Value Decomposition (SVD) If A is an m×n matrix of rank r, then the singular values σ 1,..., σ r are nonzero, while σ r+1,..., σ n are zero. Any m×n matrix A can be written as A=UΣV T, where U is an orthogonal m×m matrix; V is an orthogonal n×n matrix; and Σ is an m×n matrix whose first r diagonal entries are the nonzero singular values σ 1,..., σ r of A, and all other entries are zero (where r = rank(A)). Alternatively, this singular value decomposition can be written as where the and the are the columns of U and V, respectively.

29. 29 Singular-Value Decomposition (SVD) (II)

30. 30 Example 4 Find an SVD for (sol) –We find –The columns of U are defined by –Finally, –You can check that A=UΣV T.

31. 31 Example 5 Find an SVD for (sol) –We find that –Check that A = UΣV T.