Download presentation

Presentation is loading. Please wait.

1
Lecture 11 Fundamental Theorems of Linear Algebra Orthogonalily and Projection Shang-Hua Teng

2
The Whole Picture Rank(A) = m = n Ax=b has unique solution Rank(A) = m < n Ax=b has n-m dimensional solution Rank(A) = n < m Ax=b has 0 or 1 solution Rank(A) < n, Rank(A) < m Ax=b has 0 or n-rank(A) dimensions

3
Basis and Dimension of a Vector Space A basis for a vector space is a sequence of vectors that –The vectors are linearly independent –The vectors span the space: every vector in the vector can be expressed as a linear combination of these vectors

4
Basis for 2D and n-D (1,0), (0,1) (1 1), (-1 –2) The vectors v 1,v 2,…v n are basis for R n if and only if they are columns of an n by n invertible matrix

5
Column and Row Subspace C(A): the space spanned by columns of A –Subspace in m dimensions –The pivot columns of A are a basis for its column space Row space: the space spanned by rows of A –Subspace in n dimensions –The row space of A is the same as the column space of A T, C(A T ) –The pivot rows of A are a basis for its row space –The pivot rows of its Echolon matrix R are a basis for its row space

6
Important Property I: Uniqueness of Combination The vectors v 1,v 2,…v n are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v 1,v 2,…v n. v = a 1 v 1 + a 2 v 2 +…+ a n v n v = b 1 v 1 + b 2 v 2 +…+ b n v n So: 0=(a 1 - b 1 ) v 1 + (a 2 -b 2 )v 2 +…+ (a n -b n )v n

7
Important Property II: Dimension and Size of Basis If a vector space V has two set of bases –v 1,v 2,…v m. V = [v 1,v 2,…v m ] –w 1,w 2,…w n. W= [w 1,w 2,…w n ]. then m = n –Proof: assume n > m, write W = VA –A is m by n, so Ax = 0 has a non-zero solution –So VAx = 0 and Wx = 0 The dimension of a vector space is the number of vectors in every basis –Dimension of a vector space is well defined

8
Dimensions of the Four Subspaces Fundamental Theorem of Linear Algebra, Part I Row space: C(A T ) – dimension = rank(A) Column space: C(A)– dimension = rank(A) Nullspace: N(A) – dimension = n-rank(A) Left Nullspace: N(A T ) – dimension = m –rank(A)

9
Orthogonality and Orthogonal Subspaces Two vectors v and w are orthogonal if Two vector subspaces V and W are orthogonal if

10
Example: Orthogonal Subspace in 5 Dimensions The union of these two subspaces is R 5

11
Orthogonal Complement Suppose V is a vector subspace a vector space W The orthogonal complement of V is Orthogonal complement is itself a vector subspace

12
Dimensions of the Four Subspaces Fundamental Theorem of Linear Algebra, Part I Row space: C(A T ) – dimension = rank(A) Column space: C(A)– dimension = rank(A) Nullspace: N(A) – dimension = n-rank(A) Left Nullspace: N(A T ) – dimension = m –rank(A)

13
Orthogonality of the Four Subspaces Fundamental Theorem of Linear Algebra, Part II The nullspace is the orthogonal complement of the row space in R n The left Nullspace is the orthogonal complement of the column space in R m

14
Proof The nullspace is the orthogonal complement of the row space in R n

15
The Whole Picture C(A T ) N(A) RnRn RmRm C(A) N(A T ) xnxn A x n = 0 xrxr b A x r = b A x= b dim r dim n- r dim m- r

16
Uniqueness of The Typical Solution Every vector in the column space comes from one and only one vector x r from the row space Proof: suppose there are two x r, y r from the row space such that Ax r =A y r =b, then Ax r -A y r = A(x r -y r ) = 0 (x r -y r ) is in row space and nullspace hence must be 0 The matching of dim in row and column spaces

17
Deep Secret of Linear Algebra Pseudo-inverse Throw away the two null spaces, there is an r by r invertible matrix hiding insider A. In some sense, from the row space to the column space, A is invertible It maps an r-space in n space to an r-space in m-space

18
Invertible Matrices Any n linearly independent vector in R n must span R n. They are basis. So Ax = b is always uniquely solvable A is invertible

19
Projection Projection onto an axis (a,b) x axis is a vector subspace

20
Projection onto an Arbitrary Line Passing through 0 (a,b)

21
Projection on to a Plane

22
Projection onto a Subspace Input: 1. Given a vector subspace V in R m 2.A vector b in R m … Desirable Output: –A vector in x in V that is closest to b –The projection x of b in V –A vector x in V such that (b-x) is orthogonal to V

23
How to Describe a Vector Subspace V in R m If dim(V) = n, then V has n basis vectors –a 1, a 2, …, a n –They are independent V = C(A) where A = [a 1, a 2, …, a n ]

24
Projection onto a Subspace Input: 1. Given n independent vectors a 1, a 2, …, a n in R m 2.A vector b in R m … Desirable Output: –A vector in x in C([a 1, a 2, …, a n ]) that is closest to b –The projection x of b in C([a 1, a 2, …, a n ]) –A vector x in V such that (b-x) is orthogonal to C([a 1, a 2, …, a n ])

25
Think about this Picture C(A T ) N(A) RnRn RmRm C(A) N(A T ) xnxn A x n = 0 xrxr b A x r = b A x= b dim r dim n- r dim m- r

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google