Download presentation

Presentation is loading. Please wait.

Published byKendall Crumb Modified over 2 years ago

1
Ordinary Least-Squares

2
Outline Linear regression Geometry of least-squares Discussion of the Gauss-Markov theorem

3
Ordinary Least-Squares One-dimensional regression

4
Ordinary Least-Squares One-dimensional regression Find a line that represent the ”best” linear relationship:

5
Ordinary Least-Squares One-dimensional regression Problem: the data does not go through a line

6
Ordinary Least-Squares One-dimensional regression Problem: the data does not go through a line Find the line that minimizes the sum:

7
Ordinary Least-Squares One-dimensional regression Problem: the data does not go through a line Find the line that minimizes the sum: We are looking for that minimizes

8
Ordinary Least-Squares Matrix notation Using the following notations and

9
Ordinary Least-Squares Matrix notation Using the following notations and we can rewrite the error function using linear algebra as:

10
Ordinary Least-Squares Matrix notation Using the following notations and we can rewrite the error function using linear algebra as:

11
Ordinary Least-Squares Multidimentional linear regression Using a model with m parameters

12
Ordinary Least-Squares Multidimentional linear regression Using a model with m parameters

13
Ordinary Least-Squares Multidimentional linear regression Using a model with m parameters

14
Ordinary Least-Squares Multidimentional linear regression Using a model with m parameters and n measurements

15
Ordinary Least-Squares Multidimentional linear regression Using a model with m parameters and n measurements

16
Ordinary Least-Squares

17

18
parameter 1

19
Ordinary Least-Squares measurement n parameter 1

20
Ordinary Least-Squares Minimizing

21
Ordinary Least-Squares Minimizing

22
Ordinary Least-Squares Minimizing is flat at

23
Ordinary Least-Squares Minimizing is flat at

24
Ordinary Least-Squares Minimizing is flat at does not go down around

25
Ordinary Least-Squares Minimizing is flat at does not go down around

26
Ordinary Least-Squares Positive semi-definite In 1-DIn 2-D

27
Ordinary Least-Squares Minimizing

28
Ordinary Least-Squares Minimizing

29
Ordinary Least-Squares Minimizing

30
Ordinary Least-Squares Minimizing Always true

31
Ordinary Least-Squares Minimizing Always true The normal equation

32
Ordinary Least-Squares Geometric interpretation

33
Ordinary Least-Squares Geometric interpretation b is a vector in R n

34
Ordinary Least-Squares Geometric interpretation b is a vector in R n The columns of A define a vector space range(A)

35
Ordinary Least-Squares Geometric interpretation b is a vector in R n The columns of A define a vector space range(A) Ax is an arbitrary vector in range(A)

36
Ordinary Least-Squares Geometric interpretation b is a vector in R n The columns of A define a vector space range(A) Ax is an arbitrary vector in range(A)

37
Ordinary Least-Squares Geometric interpretation is the orthogonal projection of b onto range(A)

38
Ordinary Least-Squares The normal equation:

39
Ordinary Least-Squares The normal equation: Existence: has always a solution

40
Ordinary Least-Squares The normal equation: Existence: has always a solution Uniqueness: the solution is unique if the columns of A are linearly independent

41
Ordinary Least-Squares The normal equation: Existence: has always a solution Uniqueness: the solution is unique if the columns of A are linearly independent

42
Ordinary Least-Squares Under-constrained problem

43
Ordinary Least-Squares Under-constrained problem

44
Ordinary Least-Squares Under-constrained problem

45
Ordinary Least-Squares Under-constrained problem Poorly selected data One or more of the parameters are redundant

46
Ordinary Least-Squares Under-constrained problem Poorly selected data One or more of the parameters are redundant Add constraints

47
Ordinary Least-Squares How good is the least-squares criteria? Optimality: the Gauss-Markov theorem

48
Ordinary Least-Squares How good is the least-squares criteria? Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define:

49
Ordinary Least-Squares How good is the least-squares criteria? Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define: If

50
Ordinary Least-Squares How good is the least-squares criteria? Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define: If Then is the best unbiased linear estimator

51
Ordinary Least-Squares a b eiei no errors in a i

52
Ordinary Least-Squares a b eiei a b eiei no errors in a i errors in a i

53
Ordinary Least-Squares a b homogeneous errors

54
Ordinary Least-Squares a b a b homogeneous errors non-homogeneous errors

55
Ordinary Least-Squares a b no outliers

56
Ordinary Least-Squares a b a b no outliers outliers

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google