Download presentation

Presentation is loading. Please wait.

Published byConstance Clark Modified over 4 years ago

1
CHAPTER FIVE Orthogonality Why orthogonal? Least square problem Accuracy of Numerical computation

2
Least square problem Motivation: Curve fitting Problem formulation: Given, Find such that

3
Outlines Results of Orthogonal subspace Inner product spaces Least square problems Orthonormal sets Gram-Schmidt Orthogonalization process

4
The scalar product in Def: Let,.the inner product (or scalar product) of and is defined to be The norm of is defined as

5
Theorem5.1.1 : Let. Then pf: By the law of cosines, Note: If is the angle between, then Thus Def:

6
Cor: (Cauchy-Schwartz Inequality) Let. Then Moreover, equality holds for some

7
Scalar and Vector Projections Let. Then the quantify is the scalar projection of onto and the vector is called the vector projection of onto If,then the scalar projection of onto is and the vector projection of onto is

8
Example: Find the point on the line that is closest to the point (1,4) Sol: Note that the vector is on the line Then the desired point is

9
Example: Find the equation of the plane passing through and normal to Sol:

10
Example: Find the distance form to the plane Sol: a normal vector to the plane is desired distance

11
Orthogonal Subspace Def: Let be two subspace of. We say that if, and. Example: Let but does not orthogonal to

12
Def: Let be a subspace of. Then the orthogonal complement of is defined as Example: In,

13
Lemma: (i) Two subspaces (ii) If is a subspace of,then is also a subspace of. Pf: (i) If (ii) Let and

14
Four Fundamental Subspaces Let for some It will be shown later that

15
Theorem5.1.2: Let. Then and pf: Let and _____(1) for some _____(2) ________(3) Also, if _____(4) Similarly,

16
Example: Let Clearly,

17
Theorem5.2.2: Let be a subspace of, then (i) (ii) If is a basis for and is a basis for,then is a basis for. pf: If The result follows Suppose. Let and

18
To show that is a basis for, It remains to show their independency. Let. Then Similarly, This completes the proof

19
Def: Let are two subspaces of. We say that is a direct sum of,denoted by, if each can be written uniquely as a sum, where Example: Let Then but

20
Theorem5.2.2: If is a subspace of, then pf: By Theorem5.2.2, To show uniqueness, Suppose where

21
Theorem5.2.4: If is a subspace of, then pf: Let If (Why?)

22
Remark: Let. i.e., Since and are bijections.

23
Let bijection

24
Cor5.2.5: Let and. Then either (i) or (ii) pf: or Note that

25
Example: Let. Find The basic idea is that the row space and the sol. of are invariant under row operation. Sol: (i) (Why?) (ii) (Why?) (iii) Similarly, and (iv) Clearly,

26
Example: Let (i) and (ii) The mapping is a bijection and (iv) What is the matrix representation for ?

27
Linear Product Space A tool to measure the orthogonality of two vectors in general vector space

28
Def: An inner product on a vector space is a function Satisfying the following conditions: (i) with equality iff (ii) (iii)

29
Example: (i) Let Then is an inner product of (ii) Let, Then is an inner product of (iii) Let and then is an inner product of (iv) Let, is a positive function and are distinct real numbers. Then is an inner product of

30
Def: Let be an inner product of a vector space and. we say The length or norm of is

31
Theorem5.3.1: (The Pythagorean Law) pf:

32
Example: Consider with inner product (i) (ii) (iii) (iv) (Pythagorean Law) or

33
Example: Consider with inner product It can be shown that (i) (ii) (iii) Thus are orthonormal set.

34
Example: Let and let Then not orthogonal to

35
Def: Let be two vectors in an inner product space. Then the scalar projection of onto is defined as The vector projection of onto is

36
Lemma: Let be the vector projection of onto. Then for some pf:

37
Theorem5.3.2: (Cauchy-Schwarz Inequality) Let be two vectors in an inner product space. Then Moreover, equality holds are linear dependent. pf: If If Equality holds i.e., equality holds iff are linear dependent.

38
Note: From Cauchy-Schwarz Inequality. This, we can define as the angle between the two vectors

39
Def: Let be a vector space A fun is said to be a norm if it satisfies with equality scalar

40
Theorem5.3.3: If is an inner product space, then is a norm on pf: trivial Def: The distance between is defined as

41
Example: Let. Then is a norm is a norm for any In particular, is the euclidean norm

42
Example: Let. Then

43
Example: Let Thus, However, (Why?)

44
Example: Let Then

45
Least square problem A typical example: Given Find the best line to fit the data. or or find such that is minimum Geometrical meaning :

46
Least square problem: Given then the equation may not have solutions The objective of least square problem is trying to find such that has minimum value i.e., find satisfying

47
Preview of the results: It will be shown that Moreover, If columns of are Linear independent.

48
Theorem5.4.1: H. Let be a subspace of C. (i) for all (ii) pf: where If Since the expression is unique, result (i) is then proved. (ii) follows directly from (i) by noting that

49
Question: How to find which solves Answer: Let From previous Theorem, we know that normal equation

50
Theorem5.4.2: Let and Then the normal equation. Has a unique sol. and is the unique least square sol. to pf: Clearly, is nonsingular (Why?) is the unique sol. To normal equation. is the unique sol. To the least square problem (Why?) ( has linear independent columns)

51
Note: The projection vector is the element of that is closet to in the least square sense. Thus, The matrix is called the Projection matrix (that project any vector of to )

52
Example: Suppose a spring obeys the Hook’s law and a series of data are taken (with measurement error) as How to determine ? sol: Note that is inconsistent The normal equation. is The least square sol.

53
Example: Given the data Find the best least square fit by a linear function. sol: Let the desired linear function be The problem be comes to find the least square sol. of Least square sol. The best linear least square fit is

54
Example: Find the best quadratic least square fit to the data sol: Let the desired quadratic function. be The problem becomes to find the least square sol. of least square sol. the best quadratic least square fit is

55
Orthonormal Set Simplify the least square sol. (avoid computing inverse) Numerical computational stability

56
Def: is said to be an orthogonal set in an inner product space if Moreover, if, then is said to be orthonormal

57
Example: is an orthogonal set but not orthonormal However, is orthonormal

58
Theorem5.5.1: Let be an orthogonal set of nonzero vectors in an inner product space. Then they are linear independent pf: Suppose is linear independent.

59
Example: is an orthonormal set of with inner product Note: Now you know the meaning what one says that

60
Theorem5.5.2: Let be an orthonormal basis for an inner product space. If then pf:

61
Cor: Let be an orthonormal basis for an inner product space. If and, then pf:

62
Cor: (Parseval’s Formula) If is an orthonormal basis for an innerproduct space and, then pf: direct from previous corollary

63
Example: and from an orthonormal basis for. If, then and

64
Example: Determine without computing antiderivatives. sol: and is an orthonormal set of

65
Def: is said to be an orthogonal matrix if the column vectors of form an orthonormal set in Example: The rotational matrix and the elementary reflection matrix are orthogonal matrix.

66
Properties of orthogonal matrix: at be orthogonal. Then The column vectors of form an orthonormal basis for Preserve inner product preserve norm preserve angle.

67
Note: Let the columns of form an orthonormal set of.Then and the least square sol to is This avoid computing matrix inverse.

68
Cor5.5.9: Let be a nonzero subspace of and is an orthonormal basis for. If then the projection of onto is pf:

69
Note: Let columns of be an orthonormal set The projection of onto is the sum of the projection of onto each.

70
Example: Let Find the vector in that is closet to Sol: Clearly is a basis for. Let Thus Hw: Try What is ?

71
Approximation of functions Example: Find the best least square approximation to on by a linear function. Sol: (i) Clearly, but is not orthonormal (ii) seek a function of the form By calculation is an orthonormal set of (iii) Thus the projection. is the best linear least square approximation to on

72
Approximation of trigonometric polynomials FACT: forms an orthonormal set in with respect to the inner product Problem: Given a periodic function, find a trigonometric polynomial of degree n which is a best least square approximation to.

73
Sol: It suffices to find the projection of onto the subspace The best approximation of has coefficients

74
Example: Consider with inner product of (i) Check are orthonormal (ii) Let Similarly (iii) (iv)

75
Cram-Schmidt Orthogonalization Process Question: Given a set of linear independent vectors, how to transform them into orthogonal ones while preserve spanning set.?

76
Given,Clearly Clearly Similarly, Clearly We have the next result

77
Theorem5.6.1: (The Cram-Schmidt process) H. (i) be a basis for an inner product space (ii) C. is an orthonormal basis

78
Example: Find an orthonormal basis for with inner product given by where Sol: Starting with a basis

79
QR-Decomposition Given Let _______(1) _________________(2) _______________________(3) Define Where has orthonormal columns and is upper-triangular

80
To solve with Then, the example can be solved By backsubstitution without finding Inverse (if is square)

81
Example: Solve By direct calculation, The solution can be obtained from

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google