 # Chapter 5: The Orthogonality and Least Squares

## Presentation on theme: "Chapter 5: The Orthogonality and Least Squares"— Presentation transcript:

Chapter 5: The Orthogonality and Least Squares
Math Dept, Faculty of Applied Science, HCM University of Technology Math 415: Linear Algebra Chapter 5: The Orthogonality and Least Squares Instructor Dr. Dang Van Vinh (6/2006)

CONTENTS 3.1 – The Scalar Product in Rn 3.2 – Orthogonal Subspaces 3.3 – Orthonormal set 3.4 – The Gram - Schmidt Orthogonalization Process 3.5 – Inner Product Spaces 3.6 – Least Square Problem

3.1 The Scalar Product in Rn Definition of Inner Product in Rn Let u and v are vectors in Rn. The inner product of u and v is

3.1 The Scalar Product in Rn Example. Let Compute (u,v) and (v,u) Solution.

3.1 The Scalar Product in Rn d. (u,u) (v,u), and (u,u) = 0 if and only if u = 0. Theorem Let u, v and w be vectors in Rn, and let c be a scalar. Then a. (u,v) = (v,u) b. (u+v,w) = (u,w) + (v,w) c. (cu,v) = c(u,v)=(u,cv) The Length of a Vector The length (or norm) of vector u is the nonnegative scalar ||u|| defined by

3.1 The Scalar Product in Rn A vector whose length is 1 is called a unit vector. If we divide a nonzero vector u by its length, we obtain a unit vector. The process of creating a unit vector is called normalizing The Distance between two vectors For u and v in Rn, the distance between u and v, written as dist(u,v), is the length of the vector u – v. That is dist(u,v) = ||u – v|| Example. 1) Let v = (1,-2,2,0). Find a unit vector u in the same direction as v. 2) Compute the distance between the vectors u = (7,1) and v = (3,2).

3.2 Orthogonal Subspaces Definition of Orthogonality Two vectors u and v in Rn are orthogonal (to each other) if (u,v) = 0 The Pythagorean Theorem Two vectors u and v in Rn are orthogonal if and only if Orthogonal Complements If a vector z is orthogonal to every vector in a subspace W of Rn, then z is said to be orthogonal to W. The set of all vectors z that are orthogonal to W is called the orthogonal complement of W and is denoted by

3.2 Orthogonal Subspaces Theorem 1. A vector x is in if and only if x is orthogonal to every vector in a set that span W. is a subspace of Rn. Proof. Theorem Let A be an mxn matrix. Then the orthogonal complement of the row space of A is the nullspace of A, and the orthogonal complement of the column space of A is the nullspace of AT. (Row A)T = Null A; (Col A)T = Null AT. Proof.

3.2 Orthogonal Subspaces Example. Let be a subspace of R3. Find the basis and dimension of Solution. Dim =1; basis: {(1,-2,1)}

3.2 Orthogonal Subspaces Example. Let be a subspace of R3. Find the basis and dimension of Solution. Step 1. Find the spanning set of F. The spanning set of F is {(2,-3,1)} Step 2. Analogous the previous example.

3.3 Orthonormal Set Definition of an Orthogonal Set A set of vectors {u1, u2, ..., up} is said to be an orthogonal set if each pair of distinct vectors from the set is orthogonal. Theorem If S= {u1, u2, ..., up} is an orthogonal set of nonzero vectors in Rn, then S is linearly independent and hence is a basic for the subspace spanned by S. Proof. Example. Show that {u1, u2, u3} is an orthogonal set, where

3.3 Orthonormal Set Theorem Let E = {u1, u2, ..., up} be an orthogonal basis for a subspace W of Rn. Then each y in W has a unique representation as a linear combination of E. In fact, if y = c1u1 + c2u cpup then Proof. Example. The set E = {u1, u2, u3} is an orthogonal basis for R3, where Express the vector y = (6,1,-8) as a linear combination of the vectors in E.

3.3 Orthonormal Set Solution. Compute Remark. How easy it is to compute the weights needed to build y from an orthogonal basis. If the basis were not orthogonal, it would be necessary to solve a system of linear equations to find the weights.

3.3 Orthonormal Set Definition of an Orthonormal Set A set E = {u1, u2, ..., up} is an orthonormal set if it is an orthogonal set of unit vectors. If W is a subspace spanned by such of set, then E is an orthonormal basis for W, since the set is automatically linearly independent. Example. Show that {v1, v2, v3} is an orthonormal set, where Theorem An mxn matrix U has orthonormal columns if and only if UTU=I. Proof.

3.3 Orthonormal Set Theorem Let U be an mxn matrix with orthonormal columns, and let x and y be in Rn. Then a. b. c. Proof. Definition of an Orthogonal Matrix An orthogonal matrix is a square invertible matrix U such that U-1 = UT. Theorem A square matrix with orthonormal columns is an orthogonal matrix

3.4 The Gram-Schmidt Orthogonalization Process The Gram – Schmidt process is a simple algorithm for producing an orthogonal or orthonormal basis for any subspace of Rn. Example. Let W = Span{x1, x2}, where x1 = (3,6,0) and x2 = (1,2,2). Construct an orthogonal basis {v1, v2} for W. Solution. Let p be the projection of x2 onto x1. The component of x2 orthogonal to x1 is x2 – p, which is in W because it is formed from x2 and a multiple of x1. Let v1= x1 and Then {v1, v2} is an orthogonal set of nonzero vectors in W.

3.4 The Gram-Schmidt Orthogonalization Process Example. Let W = Span{x1=(1,1,1,1), x2=(0,1,1,1), x3 = (0,0,1,1)} Construct an orthogonal basis {v1, v2, v3} for W. Solution. Step1. Let v1 = x1 and W1 = Span{x1} = Span{v1} To simplify later computations, choose v2 = (-3,1,1,1) Step2. Step3.

3.4 The Gram-Schmidt Orthogonalization Process The Gram – Schmidt Process Given a basis {x1, x2, ..., xp} for a subspace W of Rn, define Then {v1, v2, ..., vp} is an orthogonal basis for W. In addition

3.5 Inner Product Spaces Definition of Inner Product An inner product on a vector space V is a function that, to each pair of vectors u and v in V, associates a real number (u,v) and satisfies the following axioms, for all u, v, w and all scalars c a. b. c. d. A vector space with an inner product space is called an inner product space.

3.5 Inner Product Spaces Example. For vectors x = (x1, x2) and y = (y1, y2) in R2, set Show that (x,y) is an inner product. Example. For vectors x = (x1, x2) and y = (y1, y2) in R2, set Show that (x,y) is an inner product. Example. For vectors x = (x1, x2, x3); y = (y1, y2, y3) in R3, set Show that (x,y) is an inner product.

3.5 Inner Product Spaces Example. For vectors p(x) and q(x) in P2, set a. Show that (p,q) is an inner product. b. Compute (p,q) where c. Compute the length of the vector d. Compute the distance between p(x) and q(x) where e. Compute the angle between two vector in d)

3.5 Inner Product Spaces Given a subspace Example. For vectors p(x) and q(x) in P2, set a. Find the spanning set of subspace F. c. Find the basis and dimension of b. Determine m such that is be long to d. Produce an orthogonal basis for P2 by applying the Gram-Schmidt process to the polynomials 1, x, x2.

3.6 Least Square Problem