1. MAT 2401 Linear Algebra 2.5 Applications of Matrix Operations http://myhome.spu.edu/lauw

2. HW Written Homework

3. Preview We will only focus on one application – The Method of Least Squares.

4. Linear Regression Suppose that a scientist has reason to believe that 2 quantities x and y are related linearly, that is, y=mx+b. The scientist performs an experiment and collect data points (x 1,y 1 ),…,(x n,y n ).

5. Linear Regression

6. Goals Find a line y=mx+b that minimize the sum of the squares of the errors e i. Use y=mx+b to estimate the function values.

7. Linear Regression

8. Matrix Equation

10. Matrix Form of Linear Regression For the linear regression model, the coefficients of the least squares regression line are given by A= (X T X) -1 X T Y and the sum of squared error is E T E

11. Plan… Computational Example  HW Why the formula is correct?  Very Educational; Focus on the Ideas

12. Example 1 Find the least squares regression line for the points (1,1), (2,2), (3,4), and (5,6).

13. Example 1

15. Why? Give you some ideas why the formula actually work.

16. Recall Q: How to find the minimum of a function f(x)? A:

17. Recall Q: How to find the minimum of a function f(x)? A: Q: How to find the minimum of a function f(x,y)?

18. Recall: Sigma Notation A “compact” notation for sums to avoid “…”

19. Recall: Sigma Notation Final value (upper limit) Initial value (lower limit) Index

20. Recall: Linear Property 1

21. Recall: Linear Property 2

22. Why? Let g(b,m) be the function of the sum of the squared errors. We can find the critical point by solving the equations

23. Why? Let g(b,m) be the function of the sum of the squared error. We can find the critical point by solving the equations It can be shown that the critical point is a minimum (skip)

24. Why?