1. Multiple Regression Analysis: Estimation
y = b0 + b1x1 + b2x bkxk + u
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

2. Motivation for Multiple Regression
The model with two Independent Variables
Since exper/avginc is correlative with edu/expend, it is useful to explicityly put it in the equation for the unbiasedness of the estimation on b1.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

3. does not exist in 3.4 any more. Why?
How to interpret the parameters in equation 3.4? Ceteris Paribus effect
does not exist in 3.4 any more. Why?
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

4. cons
inc
consmax
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

5. The key assumption about how u is related to x1 and x2 is as equation 3.5. What it means?
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

6. The Model with K Independent Variables
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

7. Parallels with Simple Regression
b0 is still the intercept
b1 to bk all called slope parameters
u is still the error term (or disturbance)
Still need to make a zero conditional mean assumption, so now assume that
E(u|x1,x2, …,xk) = 0
Still minimizing the sum of squared residuals, so have k+1 first order conditions
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

8. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

9. Mechanics and Interpretation of Ordinary Least Squares
Obtaining the OLS Estimates.
The method of OLS chooses the estimates to minimize
the sum of squared residuals, that is to make 3.10 as
Small as possible.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

10. OLS First Order Condition
SRF
SSR
OLS First Order Condition
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

11. Interpreting the OLS Regression Equation
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

12. Example 3.1 Determinants of College GPA
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

13. Example 3.2 Hourly Wage Equation
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

14. On the Meaning of “Holding Other Factors Fixed” in Multiple Regression
The power of multiple regression analysis is that it provides this ceteris paribus interpretation even though the data have not been collected in a ceteris paribus fashion.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

15. Changing More than One Independent Variable Simutaneously
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

16. OLS Fitted Values and Residuals
Normally,
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

17. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

18. A “Partialling Out” Interpretation of Multiple Regression
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

19. Proof:
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

20. Comparison of Simple and Multiple Regression Estimates
SRF
MRF
Two cases when the parameter estimates of SRF and MRF are equal:
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

21. Example 3.3 Participation in 401 (K) Pension Plans.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

22. Goodness-of-Fit
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

23. Example 3.4 Determinants of College GPA
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

24. Example 3.5 Explaining Arrest Records
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

25. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

26. The Expected Value of the OLS Estimators
The Assumptions for Multiple Regression
Assumption MLR.1: the population model is linear in parameters as y = b0 + b1x1 + b2x2+…+ bkxk+ u
Assumption MLR.2: we can use a random sample of size n, {(xi1, xi2 ,…, xik, yi): i=1, 2, …, n}, from the population model. Thus we can write the sample model yi = b0 + b1xi1 + b2xi2+…+ bkxik+ ui
Assumption MLR.3: E(u| x1, x2 ,…, xk) = 0
Model Misspecification: cons = b0 + b1inc + b2inc2+ u
Omitting an important factor;
Measurement Error.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

27. Assumption MLR.4: No Perfect Collinearity
In the sample (and therefore in the population), none of the independent variables is constant, and there are no exact linear relationships among the independents variables.
Assumption MLR.4 does allow the independent variables to be correlated; they just cannot be perfectly correlated.
One variable is a constant multiple of another, the same variable measured in different units.
One independent variable can be expressed as an exact linear function of the two or more of the other independent variable.
The sample size, n, is too small in relation to the number of parameters being estimated.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

28. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

29. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

30. Proof:
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

31. Including Irrelevant Variables in a Regression Model
What happens if we include variables in our specification that don’t belong?
There is no effect on our parameter estimate, and OLS remains unbiased, but have undesirable effects on the variances of the OLS estimators.
What if we exclude a variable from our specification that does belong?
OLS will usually be biased
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

32. Omitted Variable Bias
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

33. Omitted Variable Bias (cont)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

34. Omitted Variable Bias (cont)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

35. Omitted Variable Bias (cont)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

36. Omitted Variable Bias Summary
Two cases where bias is equal to zero
b2 = 0, that is x2 doesn’t really belong in model
x1 and x2 are uncorrelated in the sample.
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

37. Summary of Direction of Bias
Corr(x1, x2) > 0
Corr(x1, x2) < 0
b2 > 0
Positive bias
Negative bias
b2 < 0
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

38. Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

39. The More General Case
Technically, can only sign the bias for the more general case if all of the included x’s are uncorrelated
Typically, then, we work through the bias assuming the x’s are uncorrelated, as a useful guide even if this assumption is not strictly true
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

40. Example:
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

41. Variance of the OLS Estimators
Now we know that the sampling distribution of our estimate is centered around the true parameter
Want to think about how spread out this distribution is
Much easier to think about this variance under an additional assumption, so
Assume Var(u|x1, x2,…, xk) = s2 (Homoskedasticity)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

42. Variance of OLS (cont) Let x stand for (x1, x2,…xk)
Assumption MLR.5: Var(u|x) = s2 also implies that Var(y| x) = s2
The 4 assumptions for unbiasedness, plus this homoskedasticity assumption are known as the Gauss-Markov assumptions
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

43. Variance of OLS (cont)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

44. Proof:
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

45. Components of OLS Variances
The error variance: a larger s2 implies a larger variance for the OLS estimators
The total sample variation: a larger SSTj implies a smaller variance for the estimators
Linear relationships among the independent variables: a larger Rj2 implies a larger variance for the estimators
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

46. Misspecified Models
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

47. Misspecified Models (cont)
While the variance of the estimator is smaller for the misspecified model, unless b2 = 0 the misspecified model is biased
As the sample size grows, the variance of each estimator shrinks to zero, making the variance difference less important
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

48. Estimating the Error Variance
We don’t know what the error variance, s2, is, because we don’t observe the errors, ui
What we observe are the residuals, ûi
We can use the residuals to form an estimate of the error variance
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

49. Error Variance Estimate (cont)
df = n – (k + 1), or df = n – k – 1
df (i.e. degrees of freedom) is the (number of observations) – (number of estimated parameters)
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.

50. The Gauss-Markov Theorem
Given our 5 Gauss-Markov Assumptions it can be shown that OLS is “BLUE”
Best
Linear
Unbiased
Estimator
Thus, if the assumptions hold, use OLS
Copyright © 2007 Thomson Asia Pte. Ltd. All rights reserved.