1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three properties of estimators: unbiasedness, efficiency, and consistency. The first two, treated here, relate to finite sample analysis: analysis where the sample has a finite number of observations. Unbiasedness of X

2 UNBIASEDNESS AND EFFICIENCY Consistency, a property that relates to analysis when the sample size tends to infinity, is treated in a later slideshow. Unbiasedness of X

3 UNBIASEDNESS AND EFFICIENCY Suppose that you wish to estimate the population mean  X of a random variable X given a sample of observations. We will demonstrate that the sample mean is an unbiased estimator, but not the only one. Unbiasedness of X

4 UNBIASEDNESS AND EFFICIENCY We will start with the proof in the previous sequence. We use the second expected value rule to take the 1/n factor out of the expectation expression. Unbiasedness of X

5 UNBIASEDNESS AND EFFICIENCY Next we use the first expected value rule to break up the expression into the sum of the expectations of the observations. Unbiasedness of X

6 UNBIASEDNESS AND EFFICIENCY Thinking about the sample values {X 1, …, X n } at the planning stage, each expectation is equal to  X, and hence the expected value of the sample mean, before we actually generate the sample, is  X. Unbiasedness of X

7 UNBIASEDNESS AND EFFICIENCY However, the sample mean is not the only unbiased estimator of the population mean. We will demonstrate this supposing that we have a sample of two observations (to keep it simple). Unbiasedness of X Generalized estimator

8 UNBIASEDNESS AND EFFICIENCY We will define a generalized estimator Z which is the weighted sum of the two observations, 1 and 2 being the weights. Unbiasedness of X Generalized estimator

9 UNBIASEDNESS AND EFFICIENCY We will analyze the expected value of Z and determine the condition that must be satisfied by the weights for Z to be an unbiased estimator. Unbiasedness of X Generalized estimator

10 UNBIASEDNESS AND EFFICIENCY We begin by decomposing the expectation using the first expected value rule. Unbiasedness of X Generalized estimator

11 UNBIASEDNESS AND EFFICIENCY Now we use the second expected value rule to bring 1 and  2 out of the expected value expressions. Unbiasedness of X Generalized estimator

12 UNBIASEDNESS AND EFFICIENCY The expected value of X in each observation, before we generate the sample, is  X. Unbiasedness of X Generalized estimator

Thus Z is an unbiased estimator of  X if the sum of the weights is equal to one. An infinite number of combinations of 1 and 2 satisfy this condition, not just the sample mean. Unbiasedness of X if 13 UNBIASEDNESS AND EFFICIENCY Generalized estimator

How do we choose among them? The answer is to use the most efficient estimator, the one with the smallest population variance, because it will tend to be the most accurate. 14 UNBIASEDNESS AND EFFICIENCY Efficiency

In the diagram, A and B are both unbiased estimators but B is superior because it is more efficient. 15 UNBIASEDNESS AND EFFICIENCY Efficiency

16 UNBIASEDNESS AND EFFICIENCY We will analyze the variance of the generalized estimator and find out what condition the weights must satisfy in order to minimize it. Generalized estimator Efficiency of X

17 UNBIASEDNESS AND EFFICIENCY The first variance rule is used to decompose the variance. Generalized estimator Efficiency of X

18 UNBIASEDNESS AND EFFICIENCY Note that we are assuming that X 1 and X 2 are independent observations and so their covariance is zero. The second variance rule is used to bring 1 and 2 out of the variance expressions. Generalized estimator Efficiency of X

19 UNBIASEDNESS AND EFFICIENCY The variance of X 1, at the planning stage, is  X 2. The same goes for the variance of X 2. Generalized estimator Efficiency of X

20 UNBIASEDNESS AND EFFICIENCY Now we take account of the condition for unbiasedness and re-write the variance of Z, substituting for 2. if Generalized estimator Efficiency of X

if 21 UNBIASEDNESS AND EFFICIENCY Generalized estimator The quadratic is expanded. Efficiency of X

22 UNBIASEDNESS AND EFFICIENCY To minimize the variance of Z, we must choose 1 so as to minimize the final expression. Generalized estimator Efficiency of X

23 UNBIASEDNESS AND EFFICIENCY We differentiate with respect to 1 to obtain the first-order condition. Generalized estimator Efficiency of X

The expression is minimized for 1 = 0.5. It follows that 2 = 0.5 as well. So we have demonstrated that the sample mean is the most efficient unbiased estimator, at least in this example. (Note that the second differential is positive, confirming that we have a minimum.) 24 UNBIASEDNESS AND EFFICIENCY Generalized estimator Efficiency of X

Alternatively, we could find the minimum graphically. Here is a graph of the expression as a function of UNBIASEDNESS AND EFFICIENCY Efficiency of X

Again we see that the variance is minimized for 1 = 0.5 and so the sample mean is the most efficient unbiased estimator. 26 UNBIASEDNESS AND EFFICIENCY Efficiency of X

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.6 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics