Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: sampling and estimators Original citation: Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource] © 2012 The Author This version available at: Available in LSE Learning Resources Online: May 2012 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms

Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Planning (beforehand concepts) Our first step is to take a sample of n observations {X 1, …, X n }. Before we take the sample, while we are still at the planning stage, the X i are random quantities. We know that they will be generated randomly from the distribution for X, but we do not know their values in advance. So now we are thinking about random variables on two levels: the random variable X, and its random sample components. 1 SAMPLING AND ESTIMATORS

Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Planning (beforehand concepts) Our first step is to take a sample of n observations {X 1, …, X n }. Before we take the sample, while we are still at the planning stage, the X i are random quantities. We know that they will be generated randomly from the distribution for X, but we do not know their values in advance. So now we are thinking about random variables on two levels: the random variable X, and its random sample components. 2 SAMPLING AND ESTIMATORS

Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Planning (beforehand concepts) Our first step is to take a sample of n observations {X 1, …, X n }. Before we take the sample, while we are still at the planning stage, the X i are random quantities. We know that they will be generated randomly from the distribution for X, but we do not know their values in advance. So now we are thinking about random variables on two levels: the random variable X, and its random sample components. 3 SAMPLING AND ESTIMATORS

Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Realization (afterwards concepts) Once we have taken the sample we will have a set of numbers {x 1, …, x n }. This is called by statisticians a realization. The lower case is to emphasize that these are numbers, not variables. 4 SAMPLING AND ESTIMATORS

5 Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Planning (beforehand concepts) Back to the plan. Having generated a sample of n observations {X 1, …, X n }, we plan to use them with a mathematical formula to estimate the unknown population mean  X. This formula is known as an estimator. In this context, the standard (but not only) estimator is the sample mean An estimator is a random variable because it depends on the random quantities {X 1, …, X n }.

6 SAMPLING AND ESTIMATORS Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Planning (beforehand concepts) Back to the plan. Having generated a sample of n observations {X 1, …, X n }, we plan to use them with a mathematical formula to estimate the unknown population mean  X. This formula is known as an estimator. In this context, the standard (but not only) estimator is the sample mean An estimator is a random variable because it depends on the random quantities {X 1, …, X n }.

7 SAMPLING AND ESTIMATORS Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Realization (afterwards concepts) The actual number that we obtain, given the realization {x 1, …, x n }, is known as our estimate.

8 probability density function of X XX X XX X probability density function of X We will see why these distinctions are useful and important in a comparison of the distributions of X and X. We will start by showing that X has the same mean as X. SAMPLING AND ESTIMATORS

9 We start by replacing X by its definition and then using expected value rule 2 to take 1/n out of the expression as a common factor. SAMPLING AND ESTIMATORS

10 Next we use expected value rule 1 to replace the expectation of a sum with a sum of expectations. SAMPLING AND ESTIMATORS

11 Now we come to the bit that requires thought. Start with X 1. When we are still at the planning stage, X 1 is a random variable and we do not know what its value will be. SAMPLING AND ESTIMATORS

12 All we know is that it will be generated randomly from the distribution of X. The expected value of X 1, as a beforehand concept, will therefore be  X. The same is true for all the other sample components, thinking about them beforehand. Hence we write this line. SAMPLING AND ESTIMATORS

13 Thus we have shown that the mean of the distribution of X is  X. SAMPLING AND ESTIMATORS

14 We will next demonstrate that the variance of the distribution of X is smaller than that of X, as depicted in the diagram. SAMPLING AND ESTIMATORS probability density function of X XX X XX X probability density function of X

15 SAMPLING AND ESTIMATORS We start by replacing X by its definition and then using variance rule 2 to take 1/n out of the expression as a common factor.

16 SAMPLING AND ESTIMATORS Next we use variance rule 1 to replace the variance of a sum with a sum of variances. In principle there are many covariance terms as well, but they are zero if we assume that the sample values are generated independently.

17 SAMPLING AND ESTIMATORS Now we come to the bit that requires thought. Start with X 1. When we are still at the planning stage, we do not know what the value of X 1 will be.

18 SAMPLING AND ESTIMATORS All we know is that it will be generated randomly from the distribution of X. The variance of X 1, as a beforehand concept, will therefore be  X. The same is true for all the other sample components, thinking about them beforehand. Hence we write this line. 2

19 SAMPLING AND ESTIMATORS Thus we have demonstrated that the variance of the sample mean is equal to the variance of X divided by n, a result with which you will be familiar from your statistics course.

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.5 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course 20 Elements of Econometrics