Download presentation

1
**Singular Value Decomposition**

Jonathan P. Bernick Department of Computer Science Coastal Carolina University Picture of Metabolic Pathway 1

2
**Outline Derivation Properties of the SVD Applications**

Research Directions

3
**Matrix Decompositions**

Definition: The factorization of a matrix M into two or more matrices M1, M2,…, Mn, such that M = M1M2…Mn. Many decompositions exist… QR Decomposition LU Decomposition LDU Decomposition Etc. One is special…

4
Theorem One [Will] For an m by n matrix A:nm and any orthonormal basis {a1,...,an} of n, define (1) si = ||Aai|| (2) Then…

5
**Theorem One (continued)**

Proof:

6
Theorem Two [Will] For an m by n matrix A, there is an orthonormal basis {a1,...,an} of n such that for all i j, Aai Aaj = 0 Proof: Since ATA is symmetric, the existence of {a1,...,an} is guaranteed by the Spectral Theorem. Put Theorems One and Two together, and we obtain…

7
**Singular Value Decomposition**

[Strang]: Any m by n matrix A may be factored such that A = UVT U: m by m, orthogonal, columns are the eigenvectors of AAT V: n by n, orthogonal, columns are the eigenvectors of ATA : m by n, diagonal, r singular values are the square roots of the eigenvalues of both AAT and ATA

8
SVD Example From [Strang]:

9
**SVD Properties U, V give us orthonormal bases for the subspaces of A:**

1st r columns of U: Column space of A Last m - r columns of U: Left nullspace of A 1st r columns of V: Row space of A 1st n - r columns of V: Nullspace of A IMPLICATION: Rank(A) = r

10
**Application: Pseudoinverse**

Given y = Ax, x = A+y For square A, A+ = A-1 For any A… A+ = V-1UT A+ is called the pseudoinverse of A. x = A+y is the least-squares solution of y = Ax.

11
**Rank One Decomposition**

Given an m by n matrix A:nm with singular values {s1,...,sr} and SVD A = UVT, define U = {u1| u2| ... |um} V = {v1| v2| ... |vn}T Then… A may be expressed as the sum of r rank one matrices

12
**Matrix Approximation Let A be an m by n matrix such that Rank(A) = r**

If s1 s2 ... sr are the singular values of A, then B, rank q approximation of A that minimizes ||A - B||F, is Proof: S. J. Leon, Linear Algebra with Applications, 5th Edition, p. 414 [Will]

13
**Application: Image Compression**

Uncompressed m by n pixel image: m×n numbers Rank q approximation of image: q singular values The first q columns of U (m-vectors) The first q columns of V (n-vectors) Total: q × (m + n + 1) numbers

14
**Example: Yogi (Uncompressed)**

Source: [Will] Yogi: Rock photographed by Sojourner Mars mission. 256 × 264 grayscale bitmap 256 × 264 matrix M Pixel values [0,1] ~ numbers

15
**Example: Yogi (Compressed)**

M has 256 singular values Rank 81 approximation of M: 81 × ( ) = ~ numbers

16
Example: Yogi (Both)

17
**Application: Noise Filtering**

Data compression: Image degraded to reduce size Noise Filtering: Lower-rank approximation used to improve data. Noise effects primarily manifest in terms corresponding to smaller singular values. Setting these singular values to zero removes noise effects.

18
**Example: Microarrays Source: [Holter]**

Expression profiles for yeast cell cycle data from characteristic nodes (singular values). 14 characteristic nodes Left to right: Microarrays for 1, 2, 3, 4, 5, all characteristic nodes, respectively.

19
**Research Directions Latent Semantic Indexing [Berry] Pseudoinverse**

SVD used to approximate document retrieval matrices. Pseudoinverse Applications to bioinformatics via Support Vector Machines and microarrays.

20
References [Berry]: Michael W. Berry, et. al., “Using Linear Algebra for Intelligent Information Retrieval,” CS , Department of Computer Science, University of Tennessee, Submitted to SIAM Review. [Holter]: Neal S. Holter, et. al., “Fundamental patterns underlying gene expression profiles: Simplicity from complexity,” Proc. Natl. Acad. Sci. USA, /pnas , 2000 (preprint). Available online at

21
**References (continued)**

[Strang]: Gilbert Strang, Linear Algebra and Its Applications, 3rd edition, Academic Press, Inc., New York, 1988. [Will]: Todd Will, “Introduction to the Singular Value Decomposition,” Davidson College,

22
**Full Presentation Text**

Comments Welcome!

Similar presentations

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google