Download presentation

Presentation is loading. Please wait.

Published byAbbey Mattingley Modified over 2 years ago

1
CS 395/495-26: Spring 2004 IBMR: Singular Value Decomposition (SVD Review) Jack Tumblin jet@cs.northwestern.edu

2
Matrix Multiply: A Change-of-Axes Matrix Multiply: Ax = bMatrix Multiply: Ax = b –x and b are column vectors –A has m rows, n columns x1x1……xnxnx1x1……xnxn… a 11 … a 1n …… a m1 … a mn = b1b1……bmbmb1b1……bmbm… x1x1x1x1 x2x2x2x2 b3b3b3b3 b2b2b2b2 b1b1b1b1 Ax=b x b Input (N dim) (2D) Output (M-dim) (3D)

3
Matrix Multiply: A Change-of-Axes Matrix Multiply: Ax = bMatrix Multiply: Ax = b –x and b are column vectors –A has m rows, n columns Matrix multiply is just a set of dot-products; it ‘changes coordinates’ for vector x, makes b.Matrix multiply is just a set of dot-products; it ‘changes coordinates’ for vector x, makes b. –Rows of A = A 1,A 2,A 3,... = new coordinate axes –Ax = a dot product for each new axis x1x1……xnxnx1x1……xnxn… a 11 … a 1n …… a m1 … a mn = b1b1……bmbmb1b1……bmbm… x1x1x1x1 x2x2x2x2 A2A2A2A2 A1A1A1A1 A3A3A3A3 b3b3b3b3 b2b2b2b2 b1b1b1b1 Ax=b x b Input (N dim) (2D) Output (M-dim) (3D)

4
Sphere of all unit-length x ellipsoid of bSphere of all unit-length x ellipsoid of b Output ellipsoid’s axes are always perpendicular (true for any ellipsoid)Output ellipsoid’s axes are always perpendicular (true for any ellipsoid) **THUS** we can make ┴, unit-length axes… How Does Matrix ‘stretch space’? x1x1x1x1 x2x2x2x2 b3b3b3b3 b2b2b2b2 b1b1b1b1 Ax=b 1 Input (N dim.) Output (M dim.) u1u1u1u1 u2u2u2u2 x 1 ellipsoid (disk) b

5
Sphere of all unit-length x ellipsoid of bSphere of all unit-length x ellipsoid of b Output ellipsoid’s axes form orthonormal basis vectors U i :Output ellipsoid’s axes form orthonormal basis vectors U i : Basis vectors U i are columns of U matrixBasis vectors U i are columns of U matrix SVD(A) = USV T columns of U = OUTPUT basis vectors SVD finds: ‘Output’ Vectors U, and… x1x1x1x1 x2x2x2x2 b3b3b3b3 b2b2b2b2 b1b1b1b1 Ax=b 1 Input (N dim.) Output (M dim.) u1u1u1u1 u2u2u2u2 x 1 ellipsoid (disk) b

6
SVD finds: …‘Input’ Vectors V, and… For each U i, make a matching V i that:For each U i, make a matching V i that: –Transforms to U i (with scaling s i ) : s i (A V i ) = U i –Forms an orthonormal basis of input space SVD(A) = USV T columns of V = INPUT basis vectors x1x1x1x1 x2x2x2x2 Ax=b x Input (N dim.) b3b3b3b3 b2b2b2b2 b1b1b1b1 Output (M dim.) u1u1u1u1 u2u2u2u2 v2v2v2v2 v1v1v1v1 b

7
SVD finds: …‘Scale’ factors S. We have Unit-length Output U i, vectors,We have Unit-length Output U i, vectors, Each from a Unit-length Input V i vector, soEach from a Unit-length Input V i vector, so So we need ‘scale factor’ (singular values) s i to link input to output: s i (A V i ) = U iSo we need ‘scale factor’ (singular values) s i to link input to output: s i (A V i ) = U i SVD(A) = USV T x1x1x1x1 x2x2x2x2 Ax=b x Input (N dim.) b3b3b3b3 b2b2b2b2 b1b1b1b1 Output (M dim.) u1u1u1u1 u2u2u2u2 v2v2v2v2 v1v1v1v1 s2s2s2s2 s1s1s1s1 b

8
SVD Review: What is it? Finish: SVD(A) = USV TFinish: SVD(A) = USV T –add ‘missing’ U i or V i, define these s i =0. –Singular matrix S: diagonals are s i : ‘v-to-u scale factors’ –Matrix U, Matrix V have columns U i and V i x1x1x1x1 x2x2x2x2 Ax=b x Input (N dim.) b3b3b3b3 b2b2b2b2 b1b1b1b1 Output (M dim.) u1u1u1u1 u3u3u3u3 u2u2u2u2 v2v2v2v2 v1v1v1v1 A = USV T ‘Find Input & Output Axes, Linked by Scale’ b

9
‘Let SVDs Explain it All for You’ –cool! U and V are Orthonormal! U -1 = U T, V -1 = V T –Rank(A)? == # of non-zero singular values s i –‘ill conditioned’? Some s i are nearly zero –‘Invert’ a non-square matrix A ? Amazing! pseudo-inverse:A=USV T ; A VS -1 U T = I; so A -1 = V S -1 U T x1x1x1x1 x2x2x2x2 Ax=b x Input (N dim.) b3b3b3b3 b2b2b2b2 b1b1b1b1 Output (M dim.) u1u1u1u1 u3u3u3u3 u2u2u2u2 v2v2v2v2 v1v1v1v1 A = USV T ‘Find Input & Output Axes, Linked by Scale’

10
‘Solve for the Null Space’ Ax=0 –Easy! x is the V i axis that doesn’t affect the output (s i = 0) –are all s i nonzero? then the only answer is x=0. –Null ‘space’ because V i is a DIMENSION--- x = Vi * a –More than 1 zero-valued s i ? null space >1 dimension... x = a V i1 + b V i2 + … x1x1x1x1 x2x2x2x2 Ax=b x Input (N dim.) b3b3b3b3 b2b2b2b2 b1b1b1b1 Output (M dim.) u1u1u1u1 u3u3u3u3 u2u2u2u2 v2v2v2v2 v1v1v1v1 A = USV T ‘Find Input & Output Axes, Linked by Scale’

11
More on SVDs: Handout Zisserman Appendix 3, pg 556 (cryptic)Zisserman Appendix 3, pg 556 (cryptic) Linear Algebra booksLinear Algebra books Good online introduction: (your handout) Leach, S. “Singular Value Decomposition – A primer,” Unpublished Manuscript, Department of Computer Science, Brown University. http://www.cs.brown.edu/research/ai/dynamics/tutorial/Postscript/Good online introduction: (your handout) Leach, S. “Singular Value Decomposition – A primer,” Unpublished Manuscript, Department of Computer Science, Brown University. http://www.cs.brown.edu/research/ai/dynamics/tutorial/Postscript/ http://www.cs.brown.edu/research/ai/dynamics/tutorial/Postscript/ Google on Singular Value Decomposition...Google on Singular Value Decomposition...

12
END

Similar presentations

OK

Domain Range definition: T is a linear transformation, EIGENVECTOR EIGENVALUE.

Domain Range definition: T is a linear transformation, EIGENVECTOR EIGENVALUE.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Download ppt on life process for class 10 cbse all 10 Ppt on eye oswego Project report ppt on job satisfaction Ppt on bluetooth network security Free ppt on motivation download Ppt on credit policy of development Ppt on new technology in computer science Ppt on public provident fund Ppt on books are my best friends Ppt on construction for class 10 cbse