Outline Numerical Stability Singular Value Decomposition Properties on the SVD

Numerical Stability, Condition Number Smaller Perturbation of Coefficients May Cause Significant Solution Variation Original Function Ax = b Perturbation of A and b Perturbation of Vector b b  b + ∆b, thus solution x  x + ∆x A(x + ∆x) = b + ∆b, ∆x = A-1∆b Upper bound: ||∆x||2 ≤ ||A-1||2 ||∆b||2

Numerical Stability, Condition Number Perturbation of Matrix A A  A+∆A, thus solution x  x+∆x (A+∆A)(x+∆x) = b, Ax = b and thus ||∆x||2 ≤ ||A-1||2||∆A||2||x + ∆x||2 ||∆x||2/||x + ∆x||2 ≤ ||A||2||A-1||2(||∆A||2/||A||2) Condition Number: ||A||2||A-1||2 Critically Depends on the Norm ||A-1||2

Numerical Stability, Condition Number Condition Number of a 2*2 Matrix Proportional to det(A-1) A-1 = adj(A) * det(A-1) Question: Numerical Stability Could Be Improved via Least Square Method? Condition Number for the LS solution cond(AHA) could be very large

Singular Value Decomposition Further Characterize ||A||2||A-1||2 A = UΣVH, U and V unitary matrices, Σ diagonal matrix with non-negative elements Equivalent form for the SVD A = UΣVH  AV = UΣ A = UΣVH  UHA = ΣVH Eigenvalue decomposition for AAH AAH = UΣΣHUH = UΣ2UH

SVD Properties A = UΣVH, U and V unitary matrices, m*n matrix A, rank(A)=r the first r rows/columns of Σ are not zero First r columns of U, ortho-normal basis for the column space of A First r rows of V, ortho-normal basis for the row space of A Last m-r columns of V, ortho-normal basis for the zero column space of A Last n-r columns of U, ortho-normal basis for the zero row space of A

SVD Properties First r columns of U, ortho-normal basis for the column space of A First r rows of VH, ortho-normal basis for the row space of A Last m-r columns of U, ortho-normal basis for the zero column space of A Last n-r rows of VH, ortho-normal basis for the zero row space of A

SVD Properties MP inverse based on the SVD A = UΣVH, A+ = VΣ+UH Σ+: Pseudo Inverse of Σ

SVD Properties Spectral of Matrix A ||A||spec = maximum singular value of A F-Norm of A: ||A||F = ||Σ||F Orthonormal matrix does not change 2-Norm Determinant of Matrix A: |det(A)| = |det(U)||det(Σ)||det(VH)| = |det(Σ)| cond(A) = maximum singular value / minimum singular value

SVD Properties cond(A) = maximum singular value / minimum singular value ||A||2 = maximum singular value ||A-1||2 = inverse of minimum singular value Minimum singular value min (xHAHAx), s.t. xHx = 1 Maximum singular value max (xHAHAx), s.t. xHx = 1

SVD Properties Low Rank Matrix approximation Norm and spectrum Signal and data processing Minimum Spectral and F-Norm under Rank-k Matrix

SVD Properties Inequalities on the Singular Values Rank the Singular Values of Matrices A and B Consider the Singular Values of Matrix A+B

SVD Properties Ky-Fan Norm: the largest k singular values of matrix Special Cases of Ky-Fan Norm k = 1, the largest singular value k = N, the trace norm Ky-Fan Norm Inequalities