Download presentation

Presentation is loading. Please wait.

1
**Ordinary Least-Squares**

2
**Outline Linear regression Geometry of least-squares**

Discussion of the Gauss-Markov theorem Ordinary Least-Squares

3
**One-dimensional regression**

Ordinary Least-Squares

4
**One-dimensional regression**

Find a line that represent the ”best” linear relationship: Ordinary Least-Squares

5
**One-dimensional regression**

Problem: the data does not go through a line Ordinary Least-Squares

6
**One-dimensional regression**

Problem: the data does not go through a line Find the line that minimizes the sum: Ordinary Least-Squares

7
**One-dimensional regression**

Problem: the data does not go through a line Find the line that minimizes the sum: We are looking for that minimizes Ordinary Least-Squares

8
**Matrix notation and Using the following notations**

Ordinary Least-Squares

9
**Matrix notation and Using the following notations**

we can rewrite the error function using linear algebra as: Ordinary Least-Squares

10
**Matrix notation and Using the following notations**

we can rewrite the error function using linear algebra as: Ordinary Least-Squares

11
**Multidimentional linear regression**

Using a model with m parameters Ordinary Least-Squares

12
**Multidimentional linear regression**

Using a model with m parameters Ordinary Least-Squares

13
**Multidimentional linear regression**

Using a model with m parameters Ordinary Least-Squares

14
**Multidimentional linear regression**

Using a model with m parameters and n measurements Ordinary Least-Squares

15
**Multidimentional linear regression**

Using a model with m parameters and n measurements Ordinary Least-Squares

16
**Ordinary Least-Squares**

17
**Ordinary Least-Squares**

18
parameter 1 Ordinary Least-Squares

19
parameter 1 measurement n Ordinary Least-Squares

20
Minimizing Ordinary Least-Squares

21
Minimizing Ordinary Least-Squares

22
Minimizing is flat at Ordinary Least-Squares

23
Minimizing is flat at Ordinary Least-Squares

24
Minimizing is flat at does not go down around Ordinary Least-Squares

25
Minimizing is flat at does not go down around Ordinary Least-Squares

26
**Positive semi-definite**

In 1-D In 2-D Ordinary Least-Squares

27
Minimizing Ordinary Least-Squares

28
Minimizing Ordinary Least-Squares

29
Minimizing Ordinary Least-Squares

30
Minimizing Always true Ordinary Least-Squares

31
Minimizing The normal equation Always true Ordinary Least-Squares

32
**Geometric interpretation**

Ordinary Least-Squares

33
**Geometric interpretation**

b is a vector in Rn Ordinary Least-Squares

34
**Geometric interpretation**

b is a vector in Rn The columns of A define a vector space range(A) Ordinary Least-Squares

35
**Geometric interpretation**

b is a vector in Rn The columns of A define a vector space range(A) Ax is an arbitrary vector in range(A) Ordinary Least-Squares

36
**Geometric interpretation**

b is a vector in Rn The columns of A define a vector space range(A) Ax is an arbitrary vector in range(A) Ordinary Least-Squares

37
**Geometric interpretation**

is the orthogonal projection of b onto range(A) Ordinary Least-Squares

38
The normal equation: Ordinary Least-Squares

39
**The normal equation: Existence: has always a solution**

Ordinary Least-Squares

40
**The normal equation: Existence: has always a solution**

Uniqueness: the solution is unique if the columns of A are linearly independent Ordinary Least-Squares

41
**The normal equation: Existence: has always a solution**

Uniqueness: the solution is unique if the columns of A are linearly independent Ordinary Least-Squares

42
**Under-constrained problem**

Ordinary Least-Squares

43
**Under-constrained problem**

Ordinary Least-Squares

44
**Under-constrained problem**

Ordinary Least-Squares

45
**Under-constrained problem**

Poorly selected data One or more of the parameters are redundant Ordinary Least-Squares

46
**Under-constrained problem**

Poorly selected data One or more of the parameters are redundant Add constraints Ordinary Least-Squares

47
**How good is the least-squares criteria?**

Optimality: the Gauss-Markov theorem Ordinary Least-Squares

48
**How good is the least-squares criteria?**

Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define: Ordinary Least-Squares

49
**How good is the least-squares criteria?**

Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define: If Ordinary Least-Squares

50
**How good is the least-squares criteria?**

Optimality: the Gauss-Markov theorem Let and be two sets of random variables and define: If Then is the best unbiased linear estimator Ordinary Least-Squares

51
b ei a no errors in ai Ordinary Least-Squares

52
b b ei ei a a no errors in ai errors in ai Ordinary Least-Squares

53
b a homogeneous errors Ordinary Least-Squares

54
**b b a a homogeneous errors non-homogeneous errors**

Ordinary Least-Squares

55
b a no outliers Ordinary Least-Squares

56
outliers b b a a no outliers outliers Ordinary Least-Squares

Similar presentations

Presentation is loading. Please wait....

OK

Sections 1.8/1.9: Linear Transformations

Sections 1.8/1.9: Linear Transformations

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google