EC220 - Introduction to econometrics (review chapter) Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: unbiasedness and efficiency   Original citation: Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource] © 2012 The Author This version available at: http://learningresources.lse.ac.uk/141/ Available in LSE Learning Resources Online: May 2012 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/   http://learningresources.lse.ac.uk/

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Much of the analysis in this course will be concerned with three properties of estimators: unbiasedness, efficiency, and consistency. The first two, treated here, relate to finite sample analysis: analysis where the sample has a finite number of observations. 1

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Consistency, a property that relates to analysis when the sample size tends to infinity, is treated in a later slideshow. 2

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Suppose that you wish to estimate the population mean mX of a random variable X given a sample of observations. We will demonstrate that the sample mean is an unbiased estimator, but not the only one. 3

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: We will start with the proof in the previous sequence. We use the second expected value rule to take the 1/n factor out of the expectation expression. 4

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Next we use the first expected value rule to break up the expression into the sum of the expectations of the observations. 5

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Thinking about the sample values {X1, …, Xn} at the planning stage, each expectation is equal to mX, and hence the expected value of the sample mean, before we actually generate the sample, is mX. 6

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 However, the sample mean is not the only unbiased estimator of the population mean. We will demonstrate this supposing that we have a sample of two observations (to keep it simple). 7

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 We will define a generalized estimator Z which is the weighted sum of the two observations, l1 and l2 being the weights. 8

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 We will analyze the expected value of Z and find out what condition the weights have to satisfy for Z to be an unbiased estimator. 9

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 We begin by decomposing the expectation using the first expected value rule. 10

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 Now we use the second expected value rule to bring l1 and l2 out of the expected value expressions. 11

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 The expected value of X in each observation, before we generate the sample, is mX. 12

UNBIASEDNESS AND EFFICIENCY Unbiasedness of X: Generalized estimator Z = l1X1 + l2X2 if Thus Z is an unbiased estimator of mX if the sum of the weights is equal to one. An infinite number of combinations of l1 and l2 satisfy this condition, not just the sample mean. 13

UNBIASEDNESS AND EFFICIENCY probability density function estimator B estimator A mX How do we choose among them? The answer is to use the most efficient estimator, the one with the smallest population variance, because it will tend to be the most accurate. 14

UNBIASEDNESS AND EFFICIENCY probability density function estimator B estimator A mX In the diagram, A and B are both unbiased estimators but B is superior because it is more efficient. 15

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 We will analyze the variance of the generalized estimator and find out what condition the weights must satisfy in order to minimize it. 16

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 The first variance rule is used to decompose the variance. 17

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 Note that we are assuming that X1 and X2 are independent observations and so their covariance is zero. The second variance rule is used to bring l1 and l2 out of the variance expressions. 18

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 The variance of X1, at the planning stage, is sX2. The same goes for the variance of X2. 19

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 if Now we take account of the condition for unbiasedness and re-write the variance of Z, substituting for l2. 20

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 if The quadratic is expanded. To minimize the variance of Z, we must choose l1 so as to minimize the final expression. 21

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 if We differentiate with respect to l1 to obtain the first-order condition. 22

UNBIASEDNESS AND EFFICIENCY Generalized estimator Z = l1X1 + l2X2 if The expression is minimized for l1 = 0.5. It follows that l2 = 0.5 as well. So we have demonstrated that the sample mean is the most efficient unbiased estimator, at least in this example. (Note that the second differential is positive, confirming that we have a minimum.) 23

UNBIASEDNESS AND EFFICIENCY Alternatively, we could find the minimum graphically. Here is a graph of the expression as a function of l1. 24

UNBIASEDNESS AND EFFICIENCY Again we see that the variance is minimized for l1 = 0.5 and so the sample mean is the most efficient unbiased estimator. 25

Copyright Christopher Dougherty 2011. These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.6 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre http://www.oup.com/uk/orc/bin/9780199567089/. Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx or the University of London International Programmes distance learning course 20 Elements of Econometrics www.londoninternational.ac.uk/lse. 11.07.25