1. 3 The Vector Space R n 3.2 Vector space Properties of R n 3.3 Examples of Subspaces 3.4 Bases for Subspaces 3.5 Dimension 3.6 Orthogonal Bases for Subspaces Core Sections

2. In mathematics and the physical sciences, the term vector is applied to a wide variety of objects. Perhaps the most familiar application of the term is to quantities, such as force and velocity, that have both magnitude and direction. Such vectors can be represented in two space or in three space as directed line segments or arrows. As we will see in chapter 5,the term vector may also be used to describe objects such as matrices, polynomials, and continuous real- valued functions. 3.1 Introduction

3. In this section we demonstrate that Rn, the set of n-dimensional vectors, provides a natural bridge between the intuitive and natural concept of a geometric vector and that of an abstract vector in a general vector space.

4. 3.2 VECTOR SPACE PROPERTIES OF R n

5. The Definition of Subspaces of R n A subset W of R n is a subspace of R n if and only if the following conditions are met: (s 1 ) * The zero vector, θ, is in W. (s 2 )X+Y is in W whenever X and Y are in W. (s 3 )aX is in W whenever X is in W and a is any scalar.

6. Example 1: Let W be the subset of R 3 defined by Verify that W is a subspace of R 3 and give a geometric interpretation of W. Solution:

7. Step 1. An algebraic specification for the subset W is given, and this specification serves as a test for determining whether a vector is in R n or is not in W. Step 2.Test the zero vector, θ, of R n to see whether it satisfies the algebraic specification required to be in W. (This sows that W is nonempty.) Verifying that W is a subspace of R n

8. Step 3.Choose two arbitrary vectors X and Y from W. Thus X and Y are in R n, and both vectors satisfy the algebraic specification of W. Step 4. Test the sum X+Y to see whether it meets the specification of W. Step 5. For an arbitrary scalar, a, test the scalar multiple aX to see whether it meets the specification of W.

9. Example 3: Let W be the subset of R 3 defined by Show that W is not a subspace of R 3. Example 2: Let W be the subset of R3 defined by Verify that W is a subspace of R 3 and give a geometric interpretation of W.

10. Example 4:Let W be the subset of R 2 defined by Demonstrate that W is not a subspace of R 2. Example 5:Let W be the subset of R 2 defined by Demonstrate that W is not a subspace of R 2. Exercise P 175 18 32

11. 3.3 EXAMPLES OF SUBSPACES In this section we introduce several important and particularly useful examples of subspaces of R n.

12. The span of a subset Theorem 3:If v 1, …v r are vectors in R n, then the set W consisting of all linear combinations of v 1, …,v r is a subspace of R n. If S={v 1, …,v r } is a subset of R n, then the subspace W consisting of all linear combinations of v 1, …,v r is called the subspace spanned by S and will be denoted by Sp(S) or Sp{v 1, …,v r }.

13. For example: (1)For a single vector v in R n, Sp{v} is the subspace Sp{v}={av:a is any real number}. (2)If u and v are noncollinear geometric vectors, then Sp{u,v}={au+bv:a,b any real numbers} (3) If u, v, w are vectors in R 3,and are not on the same space,then Sp{u,v,w}={au+bv+cw : a,b,c any real numbers}

14. Example 1: Let u and v be the three-dimensional vectors Determine W=Sp{u,v} and give a geometric interpretation of W.

15. The null space of a matrix We now introduce two subspaces that have particular relevance to the linear system of equations Ax=b, where A is an (m×n) matrix. The first of these subspaces is called the null space of A (or the kernel of A) and consists of all solutions of Ax=θ. Definition 1:Let A be an (m × n) matrix. The null space of A [denoted N(A)] is the set of vectors in R n defined by N(A)={x:Ax= θ, x in R n }. Theorem 4:If A is an (m × n) matrix, then N(A) is a subspace of R n.

16. Example 2:Describe N(A), where A is the (3 × 4) matrix Solution: N(A) is determined by solving the homogeneous system Ax= θ. This is accomplished by reducing the augmented matrix [A| θ] to echelon form. It is easy to verify that [A| θ] is row equivalent to

17. Solving the corresponding reduced system yields x 1 =-2x 3 -3x 4 x 2 =-x 3 +2x 4 Where x 3 and x 4 are arbitrary; that is,

18. Example 5:Let S={v 1,v 2,v 3,v 4 } be a subset of R 3, where Show that there exists a set T={w 1,w 2 } consisting of two vectors in R 3 such that Sp(S)=Sp(T). Solution: let

19. Set row operation to A and reduce A to the following matrix: So, Sp(S)={av 1 +bv 2 :a,b any real number} Because Sp(T)=Sp(S), then Sp(T)={av 1 +bv 2 :a,b any real number} For example, we set

21. The solution on P184 And the row vectors of A T are precisely the vectors v 1 T,v 2 T,v 3 T, and v 4 T. It is straightforward to see that A T reduces to the matrix So, by Theorem 6, A T and B T have the same row space. Thus A and B have the same column space where

22. In particular, Sp(S)=Sp(T), where T={w1,w2},

23. Two of the most fundamental concepts of geometry are those of dimension and the use of coordinates to locate a point in space. In this section and the next, we extend these notions to an arbitrary subspace of R n by introducing the idea of a basis for a subspace. 3.4 BASES FOR SUBSPACES

24. An example from R 2 will serve to illustrate the transition from geometry to algebra. We have already seen that each vector v in R 2, can be interpreted geometrically as the point with coordinates a and b. Recall that in R 2 the vectors e 1 and e 2 are defined by

25. Clearly the vector v in (1) can be expressed uniquely as a linear combination of e 1 and e 2 : v=ae 1 +be 2 (2)

26. As we will see later, the set {e 1,e 2 } is an example of a basis for R 2 (indeed, it is called the natural basis for R 2 ). In Eq.(2), the vector v is determined by the coefficients a and b (see Fig.3.12). Thus the geometric concept of characterizing a point by its coordinates can be interpreted algebraically as determining a vector by its coefficients when the vector is expressed as a linear combination of “basis” vectors.

27. Spanning sets Let W be a subspace of Rn, and let S be a subset of W. The discussion above suggests that the first requirement for S to be a basis for W is that each vector in W be expressible as a linear combination of the vectors in S. This leads to the following definition.

28. Definition 3:Let W be a subspace of R n and let S={w 1,…,w m } be a subset of W. we say that S is a spanning set for W, or simply that S spans W, if every vector w in W can be expressed as a linear combination of vectors in S; w=a 1 w 1 +…+a m w m.

29. A restatement of Definition 3 in the notation of the previous section is that S is a spanning set of W provided that Sp(S)=W. It is evident that the set S={e 1,e 2,e 3 }, consisting of the unit vectors in R 3, is a spanning set for R 3. Specifically, if v is in R 3, Then v=ae 1 +be 2 +ce 3. The next two examples consider other subset of R 3.

30. Example 1: In R 3, let S={u 1,u 2,u 3 }, where Determine whether S is a spanning set for R 3. Solution: The augmented matrix this matrix is row equivalent to

31. Example 2: Let S={v 1,v 2,v 3 } be the subset of R 3 defined by Does S span R 3 ? Solution: and the matrix [A|v] is row equivalent to

32. So, is in R 3 but is not in Sp(S); that is, w cannot be expressed as a linear combination of v 1, v 2,and v 3.

33. The next example illustrates a procedure for constructing a spanning set for the null space, N(A), of a matrix A. Example 3:Let A be the (3×4) matrix Exhibit a spanning set for N(A), the null space of A. Solution: The first step toward obtaining a spanning set for N(A) is to obtain an algebraic specification for N(A) by solving the homogeneous system Ax=θ.

34. Let u 1 and u 2 be the vectors

35. Therefore, N(A)=Sp{u 1,u 2 }

36. Minimal spanning sets If W is a subspace of R n, W≠{θ}, then spanning sets for W abound. For example a vector v in a spanning set can always be replaced by av, where a is any nonzero scalar. It is easy to demonstrate, however, that not all spanning sets are equally describe. For example, define u in R 2 by The set S={e 1,e 2,u} is a spanning set for R 2. indeed, for an arbitrary vector v in R 2,

37. V=(a-c)e 1 +(b-c)e 2 +cu, where c is any real number whatsoever. But the subset {e 1,e 2 } already spans R 2, so the vector u is unnecessary. Recall that a set {v 1,…,v m } of vectors in R n is linearly independent if the vector equation x 1 v 1 +…+x m v m =θ (9) has only the trivial solution x 1 =…=x m =0; if Eq.(9) has a nontrivial solution, then the set is linearly dependent. The set S={e 1,e 2,u} is linearly dependent because e 1 +e 2 -u=θ.

38. Our next example illustrates that a linearly dependent set is not an efficient spanning set; that is, fewer vectors will span the same space. Example 4: Let S={v 1,v 2,v 3 } be the subset of R3, where Show that S is a linearly dependent set, and exhibit a subset T of S such that T contains only two vectors but Sp(T)=Sp(S).

39. Solution: The vector equation x 1 v 1 +x 2 v 2 +x 3 v 3 =θ (10) is equivalent to the (3 × 3) homogeneous system of equations with augmented matrix

40. Matrix is row equivalent to So v 3 =-1v 1 +2v 2

41. On the other hand, if B={v 1,…,v m } is a linearly independent spanning set for W, then no vector in B is a linear combination of the other m-1 vectors in B. The lesson to be drawn from example 4 is that a linearly dependent spanning set contains redundant information. That is, if S={w 1,…,w r } is a linearly dependent spanning set for a subspace W, then at least one vector from S is a linear combination of the other r-1 vectors and can be discarded from S to produce a smaller spanning set.

42. Hence if a vector is removed from B, this smaller set cannot be a spanning set for W (in particular, the vector removed from B is in W but cannot be expressed as a linear combination of the vectors retained). In this sense a linearly independent spanning set is a minimal spanning set and hence represents the most efficient way of characterizing the subspace. This idea leads to the following definition. Definition 4:Let W be a nonzero subspace of R n. A basis for W is a linearly independent spanning set for W.

43. Uniqueness of representation Remark Let B={v 1,v 2, …,v p } be a basis for W, where W is a subspace of R n. If x is in W, then x can be represented uniquely in terms of the basis B. That is, there are unique scalars a 1,a 2, …,a p such that x=a 1 v 1 +a 2 v 2 +…+a p v p. As we see later, these scalars are called the coordinates x with respect to the basis. Example of bases It is easy to show that the unit vectors is a basis for R 3

44. In general, the n-dimensional vectors e 1,e 2,…,e n form a basis for R n, frequently called the natural basis. Provide another basis for R 3. And the vectors

45. Example 6:Let W be the subspace of R 4 spanned by the set S={v 1,v 2,v 3,v 4,v 5 }, where Find a subset of S that is a basis for W. Solution: So {v 1,v 2,v 4 }is a basis for W.

46. The procedure demonstrated in the preceding example can be outlined as follows: 1.A spanning set S{v 1,…,v m } for a subspace W is given. 2.Solve the vector equation x 1 v 1 +…+x m v m =θ (20) 3.If Eq.(20) has only the trivial solution x 1 =…=x m =0, then S is a linearly independent set and hence is a basis for W. 4.If Eq.(20) has nontrivial solutions, then there are unconstrained variables. For each x j that is designated as an unconstrained variable, delete the vector v j from the set S. The remaining vectors constitute a basis for W.

47. Theorem 7:If the nonzero matrix A is row equivalent to the matrix B in echelon form, then the nonzero rows of B form a basis for the row space of A.

48. 3.5 DIMENSION In this section we translate the geometric concept of dimension into algebraic terms. Clearly R 2 and R 3 have dimension 2 and 3, respectively, since these vector spaces are simply algebraic interpretations of two-space and three-space. It would be natural to extrapolate from these two cases and declare that R n has dimension n for each positive integer n; indeed, we have earlier referred to elements of R n as n-dimensional vectors. But if W is a subspace of R n, how is the dimension of W to be determined? An examination of the subspace,W, of R 3 defined by

49. Suggests a possibility. Geometrically, W is the plane with equation x=y-2z, so naturally the dimension of W is 2. The techniques of the previous section show that W has a basis [v 1,v 2 ] consisting of the two vectors

50. Thus in this case the dimension of W is equal to the number of vectors in a basis for W. The definition of dimension More generally, for any subspace W of R n, we wish to define the dimension of W to be the number of vectors in a basis for W. We have seen, however, that a subspace W may have many different bases. In fact, Exercise 30 of section 3.4 shows that any set of three linearly independent vectors in R3 is a basis for R3. Therefore, for the concept of dimension to make sense, we must show that all bases for a given subspace W contain the same number of vectors. This fact will be an easy consequence of the following theorem.

51. Theorem 8: Let W be a subspace of R n, and let B={w 1,w 2,…,w p } be a spanning set for W containing p vectors. Then an set of p+1 or more vectors in W is linearly dependent. As an immediate corollary of Theorem 8, we can show that all bases for a subspace contain the same number of vectors. Corollary: Let W be a subspace of R n, and let B={w 1,w 2,…w p } be a basis for W containing p vectors. Then every basis for W contains p vectors.

52. Given that every basis for a subspace contains the same number of vectors, we can make the following definition without any possibility of ambiguity. Definition 5 : Let W be a subspace of R n. If W has a basis B={w 1,w 2,…,w p } of p vectors, then we say that W is a subspace of dimension p, and we write dim(W)=p. In exercise 30, the reader is asked to show that every nonzero subspace of R n does have a basis. Thus a value for dimension can be assigned to any subspace of R n, where for completeness we define dim(W)=0 if W is the zero subspace.

53. Since R 3 has a basis {e 1,e 2,e 3 } containing three vectors, we see that dim(R)=3. In general, R n has a basis {e 1,e 2,…,e n } that contains n vectors; so dim(R n )=n. Thus the definition of dimension– the number of vectors in a basis– agrees with the usual terminology; R 3 is three—dimensional, and in general, R n is n-dimensional. Example 1: Let W be the subspace of R 3 defined by Exhibit a basis for W and determine dim(W).

54. Solution: A vector x in W can be written in the form Therefore, the set {u} is a basis for W, where

55. Example 2 Let W be the subspace of R3, W=span{u 1,u 2,u 3,u 4 },where Find three different bases for W and give the dimension of W. Properties of a p-Dimensional subspace An important feature of dimension is that a p-dimensional subspace W has many of the same properties as R p. For example, Theorem 11 of section 1.7 shows that any set of p+1 or more vectors in R p is linearly dependent. The following theorem shows that this same property and others hold in W when dim(W)=p.

56. Theorem 9:Let W be a subspace of R n with dim(W)=p. 1.Any set of p+1 or more vectors in W is linearly dependent. 2.Any set of fewer than p vectors in W does not span W. 3.Any set of p linearly independent vectors in W is a basis for W. 4.Any set of p vectors that spans W is a basis for W.

57. Example 3: Let W be the subspace of R 3 given in Example 2, and let {v 1,v 2,v 3 } be the subset of W defined by Determine which of the subsets {v 1 } {v 2 } {v 1,v 2 } {v 1,v 3 } {v 2,v 3 },and {v 1,v 2,v 3 } is a basis for W.

58. The Rank of matrix In this subsection we use the concept of dimension to characterize nonsingular matrices and to determine precisely when a system of linear equation Ax=b is consistent. For an (m×n) matrix A, the dimension of the null space is called the nullity of A, and the dimension of the range of A is called the rank of A. Example 4: Find the rank, nullity, and dimension of the row space for the matrix A, where

59. Solution: To find the dimension of the row space of A, observe that A is row equivalent to the matrix and B is in echelon form. Since the nonzero rows of B form a basis for the row space of A, the row space of A has dimension 3.

60. It now follows that the nullity of A is 1 because the vector A is row equivalent to matrix C, where form a basis for R(A). Thus the rank of A is 3 forms a basis for N(A).

61. Note in the previous example that the row space of A is a subspace of R 4, whereas the column space (or range) of A is a subspace of R 3. Thus they are entirely different subspaces; even so, the dimensions are the same, and the next theorem states that this is always the case. Theorem 10: If A is an (m×n) matrix, then the rank of A is equal to the rank of A T. Remark: If A is an (m × n) matrix, then n=rank(A)+nullity(A).

62. Theorem 11: An (m × n) system of linear equations, Ax=b, is consistent if and only if rank(A)=rank([A|b]). Theorem 12: An (n × n) matrix A is nonsingular if and only if the rank of A is n. The following theorem uses the concept of the rank of a matrix to establish necessary and sufficient conditions for a system of equations, Ax=b, to be consistent.

63. 3.6 ORTHOGONAL BASES FOR SUBSPACES We have seen that a basis provides a very efficient way to characterize a subspace. Also, given a subspace w, we know that there are many different ways to construct a basis for w. In this section we focus on a particular type of basis called an orthogonal basis. Orthogonal Bases The idea of orthogonality is a generalization of the vector geometry concept of perpendicularity. If u and v are two vectors in R 2 or R 3, then we know that u and v are perpendicular if u T v=0. For example, consider the vectors u and v given by

64. Clearly u T v=0, and these two vectors are perpendicular when viewed as directed line segments in the plane. In general, for vectors in R n, we use the term orthogonal rather than the term perpendicular. Specially, if u and v are vectors in R n, we say that u and v are orthogonal if u T v=0 We will also find the concept of an orthogonal set of vectors to be useful.

65. Definition 6: Let S ={u 1 u 2 … u p,} be a set of vectors in R n, The set S is said to be an orthogonal set if each pair of distinct vectors form S is orthogonal; that is Example 1 verify that S is an orthogonal set of vectors, where

66. Theorem 13 : let S ={u 1 u 2 … u p,} be a set of nonzero vectors in R n,. If S is an orthogonal set of vectors, then S is a linearly independent set of vectors. Proof:

67. Definition 7: Let W be a subspace of R n, and let B={u 1, u 2 u p } be a basis for W. If B is an orthogonal set of vectors, then B is called an orthogonal basis for W. Furthermore, if Then B is said to be an orthonormal basis for W The word orthonormal suggests both orthogonal and normalized. Thus an orthonormal basis is an orthogonal basis consisting of vectors having length 1, where a vector of length 1 is a unit vector or a normalized vector. Observe that the unit vectors e1 form an orthonormal basis for R n.

68. Example 2 Verify that the set B={v 1 v 2 v 3 }, is an orthogonal basis for R 3,where Corollary: Let W be a subspace of R n, where dim(W)=p, If S is an orthogonal set of p nonzero vectors and is also a subset of W, then S is an orthogonal basis for W.

69. Orthonormal Bases If B={u 1, u 2 …,u p } is an orthogonal set, then C={a 1 u 1, a 2 u 2, a p u p } is also an orthogonal set for any scalars a 1 a 2, a p. If B contains only nonzero vectors and if we define the scalars a i by Then C is an orthonormal set. That is, we can convert an orthogonal set of nonzero vectors into an orthonormal set by dividing each vector by its length.

70. Example3: Recall that the set B in Example 2 is an orthogonal basis for R 3. Modify B so that it is an orthonormal basis. Solution:

71. Determining Coordinates Suppose that W is a p-dimensional subspace of R n, and B={w 1 w 2 …. w p } is a basis for W. if v is any vector in W, then v can be written uniquely in the form v=a 1 w 1 +a 2 w 2 +…+ a p w p (3) The scalars a 1,a 2,…,a p, in Eq.(3) are called the coordinates of v with respect to the basis B As we will see, it is fairly easy to determine the coordinates of a vector with respect to an orthogonal basis. To appreciate the savings in computation, consider how coordinates are found when the basis is not orthogonal.

72. Example4: Express the vector v in terms of the orthogonal basis B={w 1 w 2 w 3 },where

73. In general, let W be a subspace of R n, and let B={w 1 w 2 … w p } be an orthogonal basis for W. If v is any vector in W, then v can be expressed uniquely in the form v=a 1 w 1 +a 2 w 2 +…+a p w p ; (5a) Where Constructing an Orthogonal Basis The next theorem gives a procedure that can be used to generate an orthogonal basis from any given basis. This procedure, called the Gram-Schmidt process, is quite practical from a computational standpoint

74. Theorem 14 Gram-Schmidt Let W be a p-dimensional subspace of R n, and let {w 1 w 2… w p } be any basis for W. Then the set of vectors {u 1 u 2… u p } is an orthogonal basis for W, where And where, in general

75. Example 3 : let W be the subspace of R 3 defined by W=Sp{w 1,w 2 },where Use the Gram-Schmidt process to construct an orthogonal basis for W

76. Example 6 : Use the Gram-Schmidt orthogonalization process to generate an orthogonal basis for W=Sp{w 1 w 2 w 3 },where Exercise P224 15, 19