Presentation on theme: "CHAPTER FIVE Orthogonality Why orthogonal? Least square problem Accuracy of Numerical computation."— Presentation transcript:
CHAPTER FIVE Orthogonality Why orthogonal? Least square problem Accuracy of Numerical computation
Least square problem Motivation: Curve fitting Problem formulation: Given, Find such that
Outlines Results of Orthogonal subspace Inner product spaces Least square problems Orthonormal sets Gram-Schmidt Orthogonalization process
The scalar product in Def: Let,.the inner product (or scalar product) of and is defined to be The norm of is defined as
Theorem5.1.1 : Let. Then pf: By the law of cosines, Note: If is the angle between, then Thus Def:
Cor: (Cauchy-Schwartz Inequality) Let. Then Moreover, equality holds for some
Scalar and Vector Projections Let. Then the quantify is the scalar projection of onto and the vector is called the vector projection of onto If,then the scalar projection of onto is and the vector projection of onto is
Example: Find the point on the line that is closest to the point (1,4) Sol: Note that the vector is on the line Then the desired point is
Example: Find the equation of the plane passing through and normal to Sol:
Example: Find the distance form to the plane Sol: a normal vector to the plane is desired distance
Orthogonal Subspace Def: Let be two subspace of. We say that if, and. Example: Let but does not orthogonal to
Def: Let be a subspace of. Then the orthogonal complement of is defined as Example: In,
Lemma: (i) Two subspaces (ii) If is a subspace of,then is also a subspace of. Pf: (i) If (ii) Let and
Four Fundamental Subspaces Let for some It will be shown later that
Theorem5.1.2: Let. Then and pf: Let and _____(1) for some _____(2) ________(3) Also, if _____(4) Similarly,
Cor5.2.5: Let and. Then either (i) or (ii) pf: or Note that
Example: Let. Find The basic idea is that the row space and the sol. of are invariant under row operation. Sol: (i) (Why?) (ii) (Why?) (iii) Similarly, and (iv) Clearly,
Example: Let (i) and (ii) The mapping is a bijection and (iv) What is the matrix representation for ?
Linear Product Space A tool to measure the orthogonality of two vectors in general vector space
Def: An inner product on a vector space is a function Satisfying the following conditions: (i) with equality iff (ii) (iii)
Example: (i) Let Then is an inner product of (ii) Let, Then is an inner product of (iii) Let and then is an inner product of (iv) Let, is a positive function and are distinct real numbers. Then is an inner product of
Def: Let be an inner product of a vector space and. we say The length or norm of is
Def: Let be two vectors in an inner product space. Then the scalar projection of onto is defined as The vector projection of onto is
Lemma: Let be the vector projection of onto. Then for some pf:
Theorem5.3.2: (Cauchy-Schwarz Inequality) Let be two vectors in an inner product space. Then Moreover, equality holds are linear dependent. pf: If If Equality holds i.e., equality holds iff are linear dependent.
Note: From Cauchy-Schwarz Inequality. This, we can define as the angle between the two vectors
Def: Let be a vector space A fun is said to be a norm if it satisfies with equality scalar
Theorem5.3.3: If is an inner product space, then is a norm on pf: trivial Def: The distance between is defined as
Example: Let. Then is a norm is a norm for any In particular, is the euclidean norm
Least square problem A typical example: Given Find the best line to fit the data. or or find such that is minimum Geometrical meaning :
Least square problem: Given then the equation may not have solutions The objective of least square problem is trying to find such that has minimum value i.e., find satisfying
Preview of the results: It will be shown that Moreover, If columns of are Linear independent.
Theorem5.4.1: H. Let be a subspace of C. (i) for all (ii) pf: where If Since the expression is unique, result (i) is then proved. (ii) follows directly from (i) by noting that
Question: How to find which solves Answer: Let From previous Theorem, we know that normal equation
Theorem5.4.2: Let and Then the normal equation. Has a unique sol. and is the unique least square sol. to pf: Clearly, is nonsingular (Why?) is the unique sol. To normal equation. is the unique sol. To the least square problem (Why?) ( has linear independent columns)
Note: The projection vector is the element of that is closet to in the least square sense. Thus, The matrix is called the Projection matrix (that project any vector of to )
Example: Suppose a spring obeys the Hook’s law and a series of data are taken (with measurement error) as How to determine ? sol: Note that is inconsistent The normal equation. is The least square sol.
Example: Given the data Find the best least square fit by a linear function. sol: Let the desired linear function be The problem be comes to find the least square sol. of Least square sol. The best linear least square fit is
Example: Find the best quadratic least square fit to the data sol: Let the desired quadratic function. be The problem becomes to find the least square sol. of least square sol. the best quadratic least square fit is
Orthonormal Set Simplify the least square sol. (avoid computing inverse) Numerical computational stability
Def: is said to be an orthogonal set in an inner product space if Moreover, if, then is said to be orthonormal
Example: is an orthogonal set but not orthonormal However, is orthonormal
Theorem5.5.1: Let be an orthogonal set of nonzero vectors in an inner product space. Then they are linear independent pf: Suppose is linear independent.
Example: is an orthonormal set of with inner product Note: Now you know the meaning what one says that
Theorem5.5.2: Let be an orthonormal basis for an inner product space. If then pf:
Cor: Let be an orthonormal basis for an inner product space. If and, then pf:
Cor: (Parseval’s Formula) If is an orthonormal basis for an innerproduct space and, then pf: direct from previous corollary
Example: and from an orthonormal basis for. If, then and
Example: Determine without computing antiderivatives. sol: and is an orthonormal set of
Def: is said to be an orthogonal matrix if the column vectors of form an orthonormal set in Example: The rotational matrix and the elementary reflection matrix are orthogonal matrix.
Properties of orthogonal matrix: at be orthogonal. Then The column vectors of form an orthonormal basis for Preserve inner product preserve norm preserve angle.
Note: Let the columns of form an orthonormal set of.Then and the least square sol to is This avoid computing matrix inverse.
Cor5.5.9: Let be a nonzero subspace of and is an orthonormal basis for. If then the projection of onto is pf:
Note: Let columns of be an orthonormal set The projection of onto is the sum of the projection of onto each.
Example: Let Find the vector in that is closet to Sol: Clearly is a basis for. Let Thus Hw: Try What is ?
Approximation of functions Example: Find the best least square approximation to on by a linear function. Sol: (i) Clearly, but is not orthonormal (ii) seek a function of the form By calculation is an orthonormal set of (iii) Thus the projection. is the best linear least square approximation to on
Approximation of trigonometric polynomials FACT: forms an orthonormal set in with respect to the inner product Problem: Given a periodic function, find a trigonometric polynomial of degree n which is a best least square approximation to.
Sol: It suffices to find the projection of onto the subspace The best approximation of has coefficients
Example: Consider with inner product of (i) Check are orthonormal (ii) Let Similarly (iii) (iv)
Cram-Schmidt Orthogonalization Process Question: Given a set of linear independent vectors, how to transform them into orthogonal ones while preserve spanning set.?
Given,Clearly Clearly Similarly, Clearly We have the next result
Theorem5.6.1: (The Cram-Schmidt process) H. (i) be a basis for an inner product space (ii) C. is an orthonormal basis
Example: Find an orthonormal basis for with inner product given by where Sol: Starting with a basis
QR-Decomposition Given Let _______(1) _________________(2) _______________________(3) Define Where has orthonormal columns and is upper-triangular
To solve with Then, the example can be solved By backsubstitution without finding Inverse (if is square)
Example: Solve By direct calculation, The solution can be obtained from