1. Chap. 5 Inner Product Spaces 5.1 Length and Dot Product in R n 5.2 Inner Product Spaces 5.3 Orthonormal Bases: Gram-Schmidt Process 5.4 Mathematical Models & Least Squares Analysis 5.5 Applications of Inner Product Spaces

2. Ming-Feng YehChapter 55-2 Vectors in the plane can be characterized as directed line segments having a certain length and direction. If v = (v 1, v 2 ), then the length, or magnitude, of v, denoted by, is defined to be The length or norm of a vector in R n is given by Unit vector: Zero vector: 5.1 Length and Dot Product

3. Ming-Feng YehChapter 55-3 Standard Unit Vector Each vector in the standard basis for R n has length 1 and is called a standard unit vector. R 2 : {i, j} = {(1, 0), (0, 1)} R 3 : {i, j, k} = {(1, 0, 0), (0, 1, 0), (0, 0, 1)} (1, 1) = (1, 0) + (0, 1) = i + j (2, 2) = 2(1, 1) = 2i + 2j Section 5-1 (1, 1) (2, 2) = 2(1, 1) i j

4. Ming-Feng YehChapter 55-4 Length of a Scalar Multiple Two nonzero vectors u and v in R n are parallel if one is a scalar multiple of the other, i.e., u = cv. If c > 0, then u and v have the same direction, and if c < 0, then u and v have the opposite direction. Theorem 5.1: Length of a Scalar Multiple Let v be a vector in R n and c be a scalar. Then where is the absolute value of c. Section 5-1

5. Ming-Feng YehChapter 55-5 Theorem 5.2 Unit Vector in the Direction of v If v is a nonzero vector in R n, then the vector has length 1 and has the same direction as v. This vector u is called the unit vector in the direction of v. pf: 1. v is nonzero  is positive and Therefore u has the same direction as v. 2. The above process is called normalizing the vector v. Section 5-1

6. Ming-Feng YehChapter 55-6 Example 2 Find the unit vector in the direction of v = (3,  1, 2), and verify that this vector has length 1. Sol: The unit vector in the direction of v is which is a unit vector because Section 5-1

7. Ming-Feng YehChapter 55-7 Distance Between 2 Vectors: R 2 The distance between two points in the plane, (u 1, u 2 ) and (v 1, v 2 ), is given by In vector terminology, this distance can be viewed as the length of u – v, i.e., Section 5-1

8. Ming-Feng YehChapter 55-8 Distance Between 2 Vectors: R n The distance between two vectors u and v in R n is Properties of distance: 1. 2. if and only if u = v. 3. Let u = (0, 2, 2) and v = (2, 0, 1). Then Section 5-1

9. Ming-Feng YehChapter 55-9 Angle Between Two Vectors: R 2 The angle between two nonzero vector u = (u 1, u 2 ) and v = (v 1, v 2 ) is The Law of Cosines Section 5-1

10. Ming-Feng YehChapter 55-10 Dot Product The dot product of u = (u 1, u 2, …, u n ) and v = (v 1, v 2, …, v n ) is the scalar quantity Theorem 5.3: Properties of Dot Product 1. 2. 3. 4. 5. if and only if v = 0. Section 5-1

11. Ming-Feng YehChapter 55-11 Example 5 Given u = (2, –2), v = (5, 8) and w = (– 4, 3). 1. 2. 3. 4. 5. Section 5-1

12. Ming-Feng YehChapter 55-12 Example 6 Given two vectors u and v in R n such that, and evaluate Sol: Section 5-1

13. Ming-Feng YehChapter 55-13 Angle Between Two Vectors: R n The angle  between two nonzero vectors u and v in R n is Two vectors u and v in R n is orthogonal if The zero vector 0 is orthogonal to every vector. Section 5-1

14. Ming-Feng YehChapter 55-14 Theorem 5.4 Cauchy-Schwarz Inequality If u and v are vectors in R n, then where denotes the absolute value of pf: If u = 0, then Hence the theorem is true if u = 0. If from and we have Section 5-1

15. Ming-Feng YehChapter 55-15 Example 7 Verify the Cauchy-Schwarz Inequality for u = (1, –1, 3) and v = (2, 0, –1). Sol: Because and, we have Therefore Section 5-1

16. Ming-Feng YehChapter 55-16 Examples 8 & 9 Ex 8: The angle between u = (–4, 0, 2, –2) and v = (2, 0, –1, 1) is given by u and v should have opposite directions, because u = –2v. Ex 9: The vectors u = (3, 2, –1, 4) and v = (1, –1, 1, 0) are orthogonal because Section 5-1

17. Ming-Feng YehChapter 55-17 Example 10 Determine all vectors in R 2 that are orthogonal to u = (4, 2). Sol: Let v = (v 1, v 2 ) be orthogonal to u. Then This implies that Therefore every vector that is orthogonal to (4, 2) is of the form Section 5-1

18. Ming-Feng YehChapter 55-18 Theorem 5.5: Triangle Inequality If u and v are vectors in R n, then pf: Section 5-1 Equality occurs if and only if the vectors u and v have the same directions.

19. Ming-Feng YehChapter 55-19 Theorem 5.6: Pythagorean Thm If u and v are vectors in R n, then u and v are orthogonal iff pf: If u and v are orthogonal, then Section 5-1

20. Ming-Feng YehChapter 55-20 Dot Product and Matrix Multiplication Represent a vector in R n as an n  1 column matrix. Let and. Then Section 5-1

21. Ming-Feng YehChapter 55-21 5.2 Inner Product Spaces = dot product (Euclidean inner product for R n ) = general inner product for vector space V. Definition: Let u, v, and w be vectors in a vector space V, and let c be any scalar. An inner product on V is a function that associates a real number with pair of vectors u and v and satisfies the following axioms. 1. 2. 3. 4. and iff v = 0.

22. Ming-Feng YehChapter 55-22 Inner Product Space A vector space V with an inner product is called an inner product space. Show that the following function defines an inner product on R 2 : where and pf: 1. 2. Let. Then Section 5-2

23. Ming-Feng YehChapter 55-23 Proof of Theorem 2 3. If c is any scalar, then 4. Because the square of a real number is nonnegative, Moreover, this expression is equal to zero iff v = 0. Section 5-2

24. Ming-Feng YehChapter 55-24 Example 3 Show that the following function is not an inner product on R 3 : where and pf: Let v = (1, 2, 1). Then Axiom 4 is not satisfied. Section 5-2

25. Ming-Feng YehChapter 55-25 Inner Product on M 2,2 & P n Let and be matrices in the vector space M 2,2. The function given by is an inner product on M 2,2. Let and be polynomials in the vector space P n. The function given by is an inner product on P n. The verification of the four inner product axioms is left to you. Section 5-2

26. Ming-Feng YehChapter 55-26 Theorem 5.7 Properties of Inner Products Let u, v, and w be vectors in an inner product space V, and let c be any real number. 1. 2. 3. Section 5-2

27. Ming-Feng YehChapter 55-27 Norm, Distance & Angle Let u and v be vectors in an inner product space V. 1. The norm (or length) of u is 2. The distance between u and v is 3. The angle between two nonzero vectors u and v is given by 4. u and v are orthogonal if If, then v is called a unit vector. If v is any nonzero vector, then the vector is called the unit vector in the direction of v. Section 5-2

28. Ming-Feng YehChapter 55-28 Example 6 Let and be polynomial in P 2, and determine the following. 1. 2. 3. 4. Section 5-2

29. Ming-Feng YehChapter 55-29 Theorem 5.8 Let u and v be vectors in an inner product space V. 1. Cauchy-Schwarz Inequality: 2. Triangle Inequality: 3. Pythagorean Theorem: u and v are orthogonal iff Section 5-2

30. Ming-Feng YehChapter 55-30 Orthogonal Projections: R 2 Let u and v be vectors in the plane. If v is nonzero, then u can be orthogonally projected onto v. This projection is denoted by proj v u. proj v u is a scalar multiple of v, i.e., proj v u = av. If a > 0, then and Therefore u v proj v u  Section 5-2

31. Ming-Feng YehChapter 55-31 Proj v u in R 2 & Example 9 If a < 0, then The orthogonal projection of u onto v is given by the same formula. Example 9: The orthogonal projection of u = (4, 2) onto v = (3, 4) is given by u v proj v u  v u Section 5-2

32. Ming-Feng YehChapter 55-32 Orthogonal Projection & Ex 10 Let u and v be vectors in an inner product space V, such that Then the orthogonal projection of u onto v is given by Example 10: The orthogonal projection of u = (6, 2, 4) onto v = (1, 2, 0) is given by Section 5-2

33. Ming-Feng YehChapter 55-33 Remark of Example 10 In Example 10, u = (6, 2, 4), v = (1, 2, 0), and proj v u = (2, 4, 0). u – proj v u = (6, 2, 4) – (2, 4, 0) = (4, –2, 4) is orthogonal to v = (1, 2, 0). If u and v are nonzero vectors in an inner product space, then u – proj v u is orthogonal to v. u v proj v u  d(u, proj v u) Section 5-2

34. Ming-Feng YehChapter 55-34 Theorem 5.9 Orthogonal Projection and Distance Let u and v be vectors in an inner product space V, such that Then u v proj v u  d(u, cv) d(u, proj v u) Section 5-2

35. Ming-Feng YehChapter 55-35 5.3 Orthogonal Bases The standard basis for R 3 : B = {(1, 0, 0), (0, 1, 0), (0, 0, 1)} 1. The three vectors are mutually orthogonal. 2. Each vector in the basis is a unit vector. Definition: Orthogonal & Orthonormal Sets A set S of vectors in an inner product space V is called orthogonal if every pair of vectors in S is orthogonal. If, in addition, each vector in the set is a unit vector, then S is called orthonomal.

36. Ming-Feng YehChapter 55-36 Orthonormal Basis For S = {v 1, v 2, …, v n }, Orthogonal Orthonormal 1. = 0, i  j 1. = 0, i  j 2. If S is a basis, then it is called an orthogonal basis or an orthonormal basis. The standard basis for R n is orthomormal, but it is not the only orthonormal basis for R n. For example, is a nonstandard orthonormal basis for R 3. Section 5-3

37. Ming-Feng YehChapter 55-37 Examples 1 & 2 Example 1: Show that the following set is an orthonormal basis for R 3. Sol: 1. 2. Therefore S is an orthonormal set. Example 2: In P 3, with inner product = a 0 b 0 + a 1 b 1 + a 2 b 2 + a 3 b 3, the standard basis B = {1, x, x 2, x 3 } is orthonormal Section 5-3

38. Ming-Feng YehChapter 55-38 Theorem 5.10 Orthogonal Sets Are Linearly Independent If S = {v 1, v 2, …, v n } is an orthogonal set of nonzero vectors in an inner product space V, then S is linearly independent. pf: Because S is orthogonal, = 0, i  j. = c 1 + c 2 + …+ c i +…+ c n = c i = 0 Hence every c i must be zero and the set must be linearly independent. Section 5-3

39. Ming-Feng YehChapter 55-39 Corollary 5.10 & Example 4 Corollary 5.10 If V is an inner product space of dimension n, then any orthogonal set of n nonzero vectors is a basis for V. Example 4: Show that the following set is a basis for R 4. Sol: Because Thus S is orthogonal. Section 5-3

40. Ming-Feng YehChapter 55-40 Theorem 5.11 Coordinates Relative to an Orthonormal Basis If B = {v 1, v 2, …, v n } is an orthonormal basis for an inner product space V, then the coordinate representation of a vector w with respect to B is w = v 1 + v 2 + … + v n pf: Because B is a basis for V, then there exists unique scalars c 1, c 2, …, c n, such that w = c 1 v 1 + c 2 v 2 + … + c n v n. Taking the inner product of the both sides of this equation, = = c 1 + c 2 + … + c n = c i Because = 1, = c i. Section 5-3

41. Ming-Feng YehChapter 55-41 Coordinate Matrix & Example 5 The coordinates representation of w relative to the orthonormal basis B = {v 1, v 2, …, v n } is w = v 1 + v 2 + … + v n The corresponding coordinate matrix of w relative to B is Example 5: Find the coordinates of w = (5, –5, 2) relative to Sol: Because B is orthonormal, Thus Section 5-3

42. Ming-Feng YehChapter 55-42 Gram-Schmidt Orthonormal Process #1 Let B = {v 1, v 2, …, v n } be a basis for an inner product space V. Let is given by Then is an orthogonal basis for V. Section 5-3

43. Ming-Feng YehChapter 55-43 Gram-Schmidt Orthonormal Process #2 Let. Then the set is an orthonormal basis for V. Moreover, span{v 1, v 2, …, v k } = span{u 1, u 2, …, u k } for k = 1,2,…,n. Let {v 1, v 2 } be a basis for R 2. {w 1, w 2 } is an orthogonal basis. is an orthonormal basis Section 5-3

44. Ming-Feng YehChapter 55-44 Example 6 Apply the Gram-Schmidt orthogonormal process to the following basis for R 2 : B = {(1, 1), (0, 1)}. Sol: Section 5-3

45. Ming-Feng YehChapter 55-45 Example 7 Apply the Gram-Schmidt orthogonormal process to the following basis for R 3 : B = {(1, 1, 0), (1, 2, 0), (0, 1, 2)}. Sol: Section 5-3

46. Ming-Feng YehChapter 55-46 Example 8 The vectors v 1 = (0, 1, 0) and v 2 = (1, 1, 1) span a plane in R 3. Find an orthonormal basis for this subspace. Sol: x y z (0, 1, 0) (1, 1, 1) Section 5-3

47. Ming-Feng YehChapter 55-47 Example 10 Find an orthonormal basis for the solution space of the following homogeneous system of linear equations Sol: Let x 3 = s and x 4 = t, Section 5-3

48. Ming-Feng YehChapter 55-48 Example 10 (cont.) One basis for solution space is Apply the Gram-Schmidt orthonormalization process process to the basis B: Section 5-3