Download presentation

Presentation is loading. Please wait.

Published byMaximillian Lonsberry Modified over 2 years ago

1
Part 3: Least Squares Algebra 3-1/27 Econometrics I Professor William Greene Stern School of Business Department of Economics

2
Part 3: Least Squares Algebra 3-2/27 Econometrics I Part 3 – Least Squares Algebra

3
Part 3: Least Squares Algebra 3-3/27 Vocabulary Some terms to be used in the discussion. Population characteristics and entities vs. sample quantities and analogs Residuals and disturbances Population regression line and sample regression Objective: Learn about the conditional mean function. Estimate and 2 First step: Mechanics of fitting a line (hyperplane) to a set of data

4
Part 3: Least Squares Algebra 3-4/27 Fitting Criteria The set of points in the sample Fitting criteria - what are they: LAD Least squares and so on Why least squares? A fundamental result: Sample moments are “good” estimators of their population counterparts We will spend the next few weeks using this principle and applying it to least squares computation.

5
Part 3: Least Squares Algebra 3-5/27 An Analogy Principle for Estimating In the population E[y | X ] = X so E[y - X |X] = 0 Continuing E[x i i ] = 0 Summing, Σ i E[x i i ] = Σ i 0 = 0 Exchange Σ i and E[] E[Σ i x i i ] = E[ X ] = 0 E[X(y - X) ]= 0 Choose b, the estimator of to mimic this population result: i.e., mimic the population mean with the sample mean Find b such that As we will see, the solution is the least squares coefficient vector.

6
Part 3: Least Squares Algebra 3-6/27 Population and Sample Moments We showed that E[ i |x i ] = 0 and Cov[x i, i ] = 0. If so, and if E[y|X] = X, then = (Var[x i ]) -1 Cov[x i,y i ]. This will provide a population analog to the statistics we compute with the data.

7
Part 3: Least Squares Algebra 3-7/27 U.S. Gasoline Market, 1960-1995

8
Part 3: Least Squares Algebra 3-8/27 Least Squares Example will be, G i on x i = [1, PG i, Y i ] Fitting criterion: Fitted equation will be y i = b 1 x i1 + b 2 x i2 +... + b K x iK. Criterion is based on residuals: e i = y i - b 1 x i1 + b 2 x i2 +... + b K x iK Make e i as small as possible. Form a criterion and minimize it.

9
Part 3: Least Squares Algebra 3-9/27 Fitting Criteria Sum of residuals: Sum of squares: Sum of absolute values of residuals: Absolute value of sum of residuals We focus on now and later

10
Part 3: Least Squares Algebra 3-10/27 Least Squares Algebra

11
Part 3: Least Squares Algebra 3-11/27 Least Squares Normal Equations

12
Part 3: Least Squares Algebra 3-12/27 Least Squares Solution

13
Part 3: Least Squares Algebra 3-13/27 Second Order Conditions

14
Part 3: Least Squares Algebra 3-14/27 Does b Minimize e’e?

15
Part 3: Least Squares Algebra 3-15/27 Sample Moments - Algebra

16
Part 3: Least Squares Algebra 3-16/27 Positive Definite Matrix

17
Part 3: Least Squares Algebra 3-17/27 Algebraic Results - 1

18
Part 3: Least Squares Algebra 3-18/27 Residuals vs. Disturbances

19
Part 3: Least Squares Algebra 3-19/27 Algebraic Results - 2 A “residual maker” M = (I - X(X’X) -1 X’) e = y - Xb= y - X(X’X) -1 X’y = My My = The residuals that result when y is regressed on X MX = 0 (This result is fundamental!) How do we interpret this result in terms of residuals? When a column of X is regressed on all of X, we get a perfect fit and zero residuals. (Therefore) My = MXb + Me = Me = e (You should be able to prove this. y = Py + My, P = X(X’X) -1 X’ = (I - M). PM = MP = 0. Py is the projection of y into the column space of X.

20
Part 3: Least Squares Algebra 3-20/27 The M Matrix M = I- X(X’X) -1 X’ is an nxn matrix M is symmetric – M = M’ M is idempotent – M*M = M (just multiply it out) M is singular; M -1 does not exist. (We will prove this later as a side result in another derivation.)

21
Part 3: Least Squares Algebra 3-21/27 Results when X Contains a Constant Term X = [1,x 2,…,x K ] The first column of X is a column of ones Since X’e = 0, x 1 ’e = 0 – the residuals sum to zero.

22
Part 3: Least Squares Algebra 3-22/27 Least Squares Algebra

23
Part 3: Least Squares Algebra 3-23/27 Least Squares

24
Part 3: Least Squares Algebra 3-24/27 Residuals

25
Part 3: Least Squares Algebra 3-25/27 Least Squares Residuals

26
Part 3: Least Squares Algebra 3-26/27 Least Squares Algebra-3 M is NxN potentially huge

27
Part 3: Least Squares Algebra 3-27/27 Least Squares Algebra-4 MX =

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google