1. 1 XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn Suppose we have a random variable X and we wish to estimate its unknown population mean  X. Our first step is to take a sample of n observations {X 1, …, X n }. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE

2. 2 Before we take the sample, while we are still at the planning stage, the X i are random quantities. We know that they will be generated randomly from the distribution for X, but we do not know their values in advance. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

3. 3 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE So now we are thinking about random variables on two levels: the random variable X, and the sample observations drawn randomly from its distribution. XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

4. 4 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density XX X2X2 XX XnXn Actual sample of n observations x 1, x 2,..., x n : realization Once we have taken the sample we will have a set of numbers {x 1, …, x n }. This is called by statisticians a realization. The lower case is to emphasize that these are particular numbers, not variables. x1x1 x2x2 xnxn

5. 5 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE We base our plan on the potential distributions. Having generated a sample of n observations {X 1, …, X n }, we plan to use them with a mathematical formula to estimate the unknown population mean  X. XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

6. XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn 6 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE This mathematical formula is known as an estimator. In this context, the standard (but not only) estimator is the sample mean. An estimator is a random variable because it depends on the random quantities {X 1, …, X n }. Estimator:

7. XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density XX X2X2 XX XnXn Actual sample of n observations x 1, x 2,..., x n : realization x1x1 x2x2 xnxn 7 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE Estimate: The actual number that we obtain, given the realization {x 1, …, x n }, is known as our estimate.

8. 8 probability density function of X XX X XX X probability density function of X We will see why these distinctions are useful and important in a comparison of the distributions of X and X. We will start by showing that X has the same mean as X. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE

9. 9 We start by replacing X by its definition and then using expected value rule 2 to take 1/n out of the expression as a common factor. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

10. 10 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE Next we use expected value rule 1 to replace the expectation of a sum with a sum of expectations. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

11. 11 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE Now we come to the bit that requires thought. Start with X 1. When we are still at the planning stage, before we draw a particular sample, X 1 is a random variable and we do not know what its value will be. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

12. 12 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE All we know is that it will be generated randomly from the distribution of X. The expected value of X 1, as a beforehand concept, will therefore be  X. The same is true for all the other sample components, thinking about them beforehand. Hence we write this line. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

13. 13 Thus we have shown that the mean of the distribution of X is  X. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn

14. 14 We will next demonstrate that the variance of the distribution of X is smaller than that of X, as depicted in the diagram. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE probability density function of X XX X XX X probability density function of X

15. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn 19 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE variance 15 We start by replacing X by its definition and then using variance rule 2 to take 1/n out of the expression as a common factor.

16. 16 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE Next we use variance rule 1 to replace the variance of a sum with a sum of variances. In principle there are many covariance terms as well, but they are zero if we assume that the sample values are generated independently. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn variance

17. 17 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE Now we come to the bit that requires thought. Start with X 1. When we are still at the planning stage, we do not know what the value of X 1 will be. XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn variance

18. 18 THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE All we know is that it will be generated randomly from the distribution of X. The variance of X 1, as a beforehand concept, will therefore be  X. The same is true for all the other sample components, thinking about them beforehand. Hence we write this line. 2 XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn variance

19. 19 Thus we have demonstrated that the variance of the sample mean is equal to the variance of X divided by n, a result with which you will be familiar from your statistics course. THE DOUBLE STRUCTURE OF A SAMPLED RANDOM VARIABLE XX X1X1 Sample of n observations X 1, X 2,..., X n : potential distributions XX X2X2 XX XnXn variance

20. Copyright Christopher Dougherty 2012 These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.5 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre http://www.oup.com/uk/orc/bin/9780199567089/http://www.oup.com/uk/orc/bin/9780199567089/. Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx or the University of London International Programmes distance learning course EC2020 Elements of Econometrics www.londoninternational.ac.uk/lsewww.londoninternational.ac.uk/lse. 2012.10.31