1. EC220 - Introduction to econometrics (chapter 2)
Christopher Dougherty
EC220 - Introduction to econometrics (chapter 2)
Slideshow: random components, unbiasedness of the regression coefficients
Original citation:
Dougherty, C. (2012) EC220 - Introduction to econometrics (chapter 2). [Teaching Resource]
© 2012 The Author
This version available at:
Available in LSE Learning Resources Online: May 2012
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms.

2. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
The regression coefficients are special types of random variable. We will demonstrate this using the simple regression model in which Y depends on X. The two equations show the true model and the fitted regression. 1

3. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
We will investigate the behavior of the ordinary least squares (OLS) estimator of the slope coefficient, shown above. 2

4. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Y has two components: a nonrandom component that depends on X and the parameters, and the random component u. Since b2 depends on Y, it indirectly depends on u. 3

5. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
If the values of u in the sample had been different, we would have had different values of Y, and hence a different value for b2. We can in theory decompose b2 into its nonrandom and random components. 4

6. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
We will start with the numerator, substituting for Y and its sample mean from the true model. 5

7. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
The b1 terms in the second factor cancel. We rearrange the remaining terms. 6

8. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
We expand the product. 7

9. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Substituting this for the numerator in the expression for b2, we decompose b2 into the true value b2 and an error term that depends on the values of X and u. 8

10. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
The error term depends on the value of the disturbance term in every observation in the sample, and thus it is a special type of random variable. 9

11. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
The error term is responsible for the variations of b2 around its fixed component b2. If we wish, we can express the decomposition more tidily. 10

12. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
This is the decomposition so far. 11

13. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
The next step is to make a small simplification of the numerator of the error term. First, we expand it as shown. 12

14. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
The mean value of u is a common factor of the second summation, so it can be taken outside. 13

15. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
The second term then vanishes because the sum of the deviations of X around its sample mean is automatically zero. 14

16. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
Thus we can rewrite the decomposition as shown. For convenience, the denominator has been denoted D. 15

17. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
A further small rearrangement of the expression for the error term. 16

18. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
Another one. 17

19. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
One more. 18

20. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
Thus we have shown that b2 is equal to the true value and plus a weighted linear combination of the values of the disturbance term in the sample, where the weights are functions of the values of X in the observations in the sample. 19

21. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
As you can see, every value of the disturbance term in the sample affects the sample value of b2. 20

22. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
Before moving on, it may be helpful to clarify a mathematical technicality. In the summation in the denominator of the expression for ai, the subscript has been changed to j. Why? 21

23. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
The denominator is the sum, from 1 to n, of the squared deviations of X from its sample mean. This is made explicit in the version of the expression in the box. 22

24. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
Written this way, the meaning of the denominator is clear, but the form is clumsy. Obviously, we should use S–notation to compress it. 23

25. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
For the S-notation, we need to choose an index symbol that changes as we go from the first squared deviation to the last. We can use anything we like, EXCEPT i, because we are already using i for a completely different purpose in the numerator. 24

26. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
We have used j here, but this was quite arbitrary. We could have used anything for the summation index (except i), as long as the meaning is clear. We could have used a smiley ☻ instead (please don’t). 25

27. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
The error term depends on the value of the disturbance term in every observation in the sample, and thus it is a special type of random variable. 26

28. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
We will show that the error term has expected value zero, and hence that the ordinary least squares (OLS) estimator of the slope coefficient in a simple regression model is unbiased. 27

29. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
The expected value of b2 is equal to the expected value of b2 and the expected value of the weighted sum of the values of the disturbance term. 28

30. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
b2 is fixed so it is unaffected by taking expectations. The first expectation rule (Review chapter) states that the expectation of a sum of several quantities is equal to the sum of their expectations. 29

31. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Now for each i, E(aiui) = aiE(ui). This is a really important step and we can make it only with Model A. 30

32. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Under Model A, we are assuming that the values of X in the observations are nonstochastic. It follows that each ai is nonstochastic, since it is just a combination of the values of X. 31

33. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Thus it can be treated as a constant, allowing us to take it out of the expectation using the second expected value rule (Review chapter). 32

34. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Fitted model
Under Assumption A.3, E(ui) = 0 for all i, and so the estimator is unbiased. The proof of the unbiasedness of the estimator of the intercept will be left as an exercise. 33

35. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
It is important to realize that the OLS estimators of the parameters are not the only unbiased estimators. We will give an example of another. 34

36. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model Y Yn
Yn – Y1 Y1
Xn – X1 X1 Xn X
Someone who had never heard of regression analysis, seeing a scatter diagram of a sample of observations, might estimate the slope by joining the first and the last observations, and dividing the increase in the height by the horizontal distance between them. 35

37. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model Y Y1 X1 Xn Yn X
Yn – Y1
Xn – X1
The estimator is thus (Yn–Y1) divided by (Xn–X1). We will investigate whether it is biased or unbiased. 36

38. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model Y Y1 X1 Xn Yn X
To do this, we start by substituting for the Y components in the expression. 37

39. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
The b1 terms cancel out and the rest of the expression simplifies as shown. Thus we have decomposed this naïve estimator into two components, the true value and an error term. This decomposition is parallel to that for the OLS estimator, but the error term is different. 38

40. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
We now take expectations to investigate unbiasedness. 39

41. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
The denominator of the error term can be taken outside because the values of X are nonstochastic. 40

42. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
Given Assumption A.3, the expectations of un and u1 are zero. Therefore, despite being naïve, this estimator is unbiased. 41

43. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
It is intuitively easy to see that we would not prefer the naïve estimator to OLS. Unlike OLS, which takes account of every observation, it employs only the first and the last and is wasting most of the information in the sample. 42

44. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
The naïve estimator will be sensitive to the value of the disturbance term u in those two observations, whereas the OLS estimator combines all the disturbance term values and takes greater advantage of the possibility that to some extent they cancel each other out. 43

45. RANDOM COMPONENTS, UNBIASEDNESS OF THE REGRESSION COEFFICIENTS
True model
More rigorously, it can be shown that the population variance of the naïve estimator is greater than that of the OLS estimator, and that the naïve estimator is therefore less efficient. 44

46. Copyright Christopher Dougherty 2011.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author.
The content of this slideshow comes from Section 2.3 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre
Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course
EC212 Introduction to Econometrics
or the University of London International Programmes distance learning course
20 Elements of Econometrics