6 6.5 © 2016 Pearson Education, Ltd. Orthogonality and Least Squares LEAST-SQUARES PROBLEMS

Slide © 2016 Pearson Education, Ltd. LEAST-SQUARES PROBLEMS

Slide LEAST-SQUARES PROBLEMS  Solution of the General Least-Squares Problem  Given A and b, apply the Best Approximation Theorem to the subspace Col A.  Let © 2016 Pearson Education, Ltd.

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Since each is a row of A T, (2)  Thus  These calculations show that each least-squares solution of satisfies the equation (3)  The matrix equation (3) represents a system of equations called the normal equations for.  A solution of (3) is often denoted by.

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Theorem 13: The set of least-squares solutions of coincides with the nonempty set of solutions of the normal equation.  Proof: The set of least-squares solutions is nonempty and each least-squares solution satisfies the normal equations.  Conversely, suppose satisfies.  Then satisfies (2), which shows that is orthogonal to the rows of A T and hence is orthogonal to the columns of A.  Since the columns of A span Col A, the vector is orthogonal to all of Col A.

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Example 1: Find a least-squares solution of the inconsistent system for  Solution: To use normal equations (3), compute:

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Then the equation becomes

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Row operations can be used to solve the system on the previous slide, but since A T A is invertible and, it is probably faster to compute and then solve as

Slide © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

Slide © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS

Slide © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS  Now that is known, we can solve.  But this is trivial, since we already know weights to place on the columns of A to produce.  It is clear from (5) that

Slide © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS

Slide © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS  The columns of Q form an orthonormal basis for Col A. (by Theorem 12)  Hence, by Theorem 10, QQ T b is the orthogonal projection of b onto Col A.  Then, which shows that is a least-squares solution of.  The uniqueness of follows from Theorem 14.