Least Squares Approximation: A Linear Algebra Technique By Larry Wong and James Sfregola.

Presentation on theme: "Least Squares Approximation: A Linear Algebra Technique By Larry Wong and James Sfregola."— Presentation transcript:

Least Squares Approximation: A Linear Algebra Technique By Larry Wong and James Sfregola

Soo……You have a bunch of Data. AttendenceBig 4DerbyLeague Positionper capita incomeadjusted population 39,968 1011350001582564 21,852 101036725137471 24,409 10736725371847 26,770 101943600852013 41,917 112436001351187 37,162 10636725623160 24,510 111643600724121 43,958 11336725738873 40,699101436725630043 75,595111367251194415 What can we do with this? Use Least Squares Approximation (LSE) to estimate the relationship of the dependent variable and the independent variable(s).

What is Least Squares? Approximates solutions for inconsistent over determined systems Approximates solutions for inconsistent over determined systems –What does inconsistent mean? –Over determined, what’s that? In other words LSE allows you to describe a model as being represented by the line of best fit. In other words LSE allows you to describe a model as being represented by the line of best fit. Where the “best fit” line, curve or polynomial is created from the approximations. Where the “best fit” line, curve or polynomial is created from the approximations.

Least Squares it’s Actually Pretty Useful Least Squares Approximations are highly relevant in various fields Really, how so? Still don’t believe us? Fine we’ll give examples…but not now.

So How Does it Estimate a Solution? Least Squares Approximation estimates a best fit solution for a system by Minimizing the magnitude error vector, e. Estimating the coefficients of the best fit equation

How Do We Ensure Positive Error? Least Squares – Eliminates negative terms – More susceptible to outlying data Least Absolute Difference – Difficult to work with in linear algebra

The Least Squares Theorem For an over determined inconsistent system Ax=b: This is easily solved using inverse and transpose multiplication the resulting vector x can be determined in the form x=(AT A) -1 AT b where x will correspond to the coefficients in your linear or polynomial expression.

How to Approximate the Error Term? e = b-Ax, where e is the error term e = b-Ax, where e is the error term Where e1= the distance from our data point to the best fit approximation given by the orthogonal projection Where e1= the distance from our data point to the best fit approximation given by the orthogonal projection

Application to Economics Econometrics utilizes Ordinary Least Squares Approximations (OLS) Similar to LSE except it follows 7 classical assumptions Estimated Regression Equation

Bibliography Poole, David Linear Algebra: A Modern Introduction. Canada: Thomson Brooks/Cole, 2006. Poole, David Linear Algebra: A Modern Introduction. Canada: Thomson Brooks/Cole, 2006. Studenmund, A.H. Using Econometrics: A Practical Guide. New York: Pearson Education, 2006. Studenmund, A.H. Using Econometrics: A Practical Guide. New York: Pearson Education, 2006.

Similar presentations