Download presentation

Presentation is loading. Please wait.

Published byMaximillian Lonsberry Modified over 4 years ago

1
Part 3: Least Squares Algebra 3-1/27 Econometrics I Professor William Greene Stern School of Business Department of Economics

2
Part 3: Least Squares Algebra 3-2/27 Econometrics I Part 3 – Least Squares Algebra

3
Part 3: Least Squares Algebra 3-3/27 Vocabulary Some terms to be used in the discussion. Population characteristics and entities vs. sample quantities and analogs Residuals and disturbances Population regression line and sample regression Objective: Learn about the conditional mean function. Estimate and 2 First step: Mechanics of fitting a line (hyperplane) to a set of data

4
Part 3: Least Squares Algebra 3-4/27 Fitting Criteria The set of points in the sample Fitting criteria - what are they: LAD Least squares and so on Why least squares? A fundamental result: Sample moments are “good” estimators of their population counterparts We will spend the next few weeks using this principle and applying it to least squares computation.

5
Part 3: Least Squares Algebra 3-5/27 An Analogy Principle for Estimating In the population E[y | X ] = X so E[y - X |X] = 0 Continuing E[x i i ] = 0 Summing, Σ i E[x i i ] = Σ i 0 = 0 Exchange Σ i and E[] E[Σ i x i i ] = E[ X ] = 0 E[X(y - X) ]= 0 Choose b, the estimator of to mimic this population result: i.e., mimic the population mean with the sample mean Find b such that As we will see, the solution is the least squares coefficient vector.

6
Part 3: Least Squares Algebra 3-6/27 Population and Sample Moments We showed that E[ i |x i ] = 0 and Cov[x i, i ] = 0. If so, and if E[y|X] = X, then = (Var[x i ]) -1 Cov[x i,y i ]. This will provide a population analog to the statistics we compute with the data.

7
Part 3: Least Squares Algebra 3-7/27 U.S. Gasoline Market, 1960-1995

8
Part 3: Least Squares Algebra 3-8/27 Least Squares Example will be, G i on x i = [1, PG i, Y i ] Fitting criterion: Fitted equation will be y i = b 1 x i1 + b 2 x i2 +... + b K x iK. Criterion is based on residuals: e i = y i - b 1 x i1 + b 2 x i2 +... + b K x iK Make e i as small as possible. Form a criterion and minimize it.

9
Part 3: Least Squares Algebra 3-9/27 Fitting Criteria Sum of residuals: Sum of squares: Sum of absolute values of residuals: Absolute value of sum of residuals We focus on now and later

10
Part 3: Least Squares Algebra 3-10/27 Least Squares Algebra

11
Part 3: Least Squares Algebra 3-11/27 Least Squares Normal Equations

12
Part 3: Least Squares Algebra 3-12/27 Least Squares Solution

13
Part 3: Least Squares Algebra 3-13/27 Second Order Conditions

14
Part 3: Least Squares Algebra 3-14/27 Does b Minimize e’e?

15
Part 3: Least Squares Algebra 3-15/27 Sample Moments - Algebra

16
Part 3: Least Squares Algebra 3-16/27 Positive Definite Matrix

17
Part 3: Least Squares Algebra 3-17/27 Algebraic Results - 1

18
Part 3: Least Squares Algebra 3-18/27 Residuals vs. Disturbances

19
Part 3: Least Squares Algebra 3-19/27 Algebraic Results - 2 A “residual maker” M = (I - X(X’X) -1 X’) e = y - Xb= y - X(X’X) -1 X’y = My My = The residuals that result when y is regressed on X MX = 0 (This result is fundamental!) How do we interpret this result in terms of residuals? When a column of X is regressed on all of X, we get a perfect fit and zero residuals. (Therefore) My = MXb + Me = Me = e (You should be able to prove this. y = Py + My, P = X(X’X) -1 X’ = (I - M). PM = MP = 0. Py is the projection of y into the column space of X.

20
Part 3: Least Squares Algebra 3-20/27 The M Matrix M = I- X(X’X) -1 X’ is an nxn matrix M is symmetric – M = M’ M is idempotent – M*M = M (just multiply it out) M is singular; M -1 does not exist. (We will prove this later as a side result in another derivation.)

21
Part 3: Least Squares Algebra 3-21/27 Results when X Contains a Constant Term X = [1,x 2,…,x K ] The first column of X is a column of ones Since X’e = 0, x 1 ’e = 0 – the residuals sum to zero.

22
Part 3: Least Squares Algebra 3-22/27 Least Squares Algebra

23
Part 3: Least Squares Algebra 3-23/27 Least Squares

24
Part 3: Least Squares Algebra 3-24/27 Residuals

25
Part 3: Least Squares Algebra 3-25/27 Least Squares Residuals

26
Part 3: Least Squares Algebra 3-26/27 Least Squares Algebra-3 M is NxN potentially huge

27
Part 3: Least Squares Algebra 3-27/27 Least Squares Algebra-4 MX =

Similar presentations

OK

Januar MDMDFSSMDMDFSSS 3 10 17 24 31 4 11 18 25 5 12 19 26 6 13 20 27 7 14 21 28 1 8 15 22 29 2 9 16 23 30.

Januar MDMDFSSMDMDFSSS 3 10 17 24 31 4 11 18 25 5 12 19 26 6 13 20 27 7 14 21 28 1 8 15 22 29 2 9 16 23 30.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google