1. Introduction to Econometrics, 5th edition
Type author name/s here
Dougherty
Introduction to Econometrics, 5th edition
Chapter heading
Review: Random Variables, Sampling, Estimation, and Inference
© Christopher Dougherty, All rights reserved.

2. THE CENTRAL LIMIT THEOREM
10 million samples
If a random variable X has a normal distribution, its sample mean X will also have a normal distribution. This fact is useful for the construction of t statistics and confidence intervals if we are employing X as an estimator of the population mean. 1

3. THE CENTRAL LIMIT THEOREM
10 million samples
However, what happens if we are not able to assume that X has a normal distribution? 2

4. THE CENTRAL LIMIT THEOREM
10 million samples
The standard response is to make use of a central limit theorem. Loosely speaking, a central limit theorem states that the distribution of X will approximate a normal distribution as the sample size becomes large, even when the distribution of X itself is not normal. 3

5. THE CENTRAL LIMIT THEOREM
10 million samples
There are a number of central limit theorems, differing only in the assumptions that they make in order to obtain this result. Here we shall be content with using the simplest one, the Lindeberg–Levy central limit theorem. 4

6. THE CENTRAL LIMIT THEOREM
10 million samples
It states that, provided that the Xi in the sample are all drawn independently from the same distribution (the distribution of X), and provided that this distribution has finite population mean and variance, the distribution of X will converge on a normal distribution as n increases. 5

7. THE CENTRAL LIMIT THEOREM
10 million samples
This means that our t statistics and confidence intervals will be approximately valid after all, provided that the sample size is large enough. 6

8. THE CENTRAL LIMIT THEOREM
10 million samples
The figure shows the distribution of X for the case where the X has a uniform distribution with range 0 to 1, for 10 million samples. A uniform distribution is one in which all values over a finite range are equally likely. 7

9. THE CENTRAL LIMIT THEOREM
10 million samples
For a sample of 1, the distribution of X is the uniform distribution itself, and so it is a horizontal line. 8

10. THE CENTRAL LIMIT THEOREM
10 million samples
We now show the distribution of X for a sample of size 10, for 10 million samples. It can be seen that X has a distribution very close to a normal distribution even though the sample size is quite small. 9

11. THE CENTRAL LIMIT THEOREM
10 million samples
Here is the distribution of X for samples of size 25. It is even closer to normal. 10

12. THE CENTRAL LIMIT THEOREM
10 million samples
Here is the distribution for sample size It is indistinguishable from normal. 11

13. THE CENTRAL LIMIT THEOREM
If X had a different distribution, the sample size required for a good approximation would be different. The figure shows the case where X has a lognormal distribution. As you can see, it is heavily skewed. 12

14. THE CENTRAL LIMIT THEOREM
10 million samples
Here is the distribution of X for sample size 10, for 10 million samples. It is still heavily skewed. 13

15. THE CENTRAL LIMIT THEOREM
10 million samples
With sample size 25, the distribution is becoming less skewed. 14

16. THE CENTRAL LIMIT THEOREM
10 million samples
However, even with sample size 100, the distribution is only an approximation to a normal distribution. Notice the difference in the shapes of the tails. We need a larger value of n before we can say that the distribution is approximately normal. 15

17. THE CENTRAL LIMIT THEOREM
10 million samples
In asserting that the distribution of X tends to become normal as the sample size increases, we have glossed over an important technical point that needs to be addressed. The central limit theorem applies only in the limit, as the sample size tends to infinity. 16

18. THE CENTRAL LIMIT THEOREM
10 million samples
However, as the sample size tends to infinity, the distribution of X degenerates to a spike located at the population mean. So how can we talk about the limiting distribution being normal? 17

19. THE CENTRAL LIMIT THEOREM
10 million samples
The answer is to transform the estimator in an appropriate way so that the transformation does have a limiting distribution. Having established the limiting distribution of the transformation, we may be able to work backwards to the properties of the estimator. 18

20. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
If X has mean m and variance s2, X has mean m and variance s2/n. The mean is independent of n, but the variance tends to zero as n tends to infinity. 19

21. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
variance stable, but mean increases
We can deal with the vanishing variance problem by scaling the estimator by This multiplies its variance by n, and so the variance becomes s2, which is independent of n. We are making progress in finding the appropriate transformation. 20

22. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
variance stable, but mean increases
However, we now have a problem with the mean. This is now It increases with n, so the statistic cannot have a limiting distribution. 21

23. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
variance stable, but mean increases
mean and variance both stable
To deal with this, we consider instead the statistic This is what we need. Its mean is zero and its variance is unaffected. The mean and variance are both independent of n, and so this statistic can have a limiting distribution. 22

24. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
variance stable, but mean increases
mean and variance both stable
Application of central limit theorem
The Lindeberg–Levy central limit theorem states that, as n tends to infinity, this statistic has a normal distribution with mean zero and variance s2. 23

25. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Search for a transformation of X that has a limiting distribution
mean
variance
properties as n increases X m
mean stable, but variance → 0
variance stable, but mean increases
mean and variance both stable
Application of central limit theorem
The arrow with a d over it is mathematical shorthand that means ‘has limiting distribution as n tends to infinity’. 24

26. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Application of central limit theorem
Approximation for finite samples
This relationship is true only as n goes to infinity. However, from the limiting distribution, we can start working back tentatively to finite samples. We can say, that for large n, the relationship may hold approximately. (The symbol ~ means ‘is distributed as’.) 25

27. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Application of central limit theorem
Approximation for finite samples
Then, dividing the statistic by , we can say that, for sufficiently large n, the second equation is approximately true. 26

28. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Application of central limit theorem
Approximation for finite samples
This implies the last equation. We knew, from the beginning, that the sample mean was distributed with mean m and variance s2/n. 27

29. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Application of central limit theorem
Approximation for finite samples
What we have shown is that, irrespective of the distribution of X, the distribution of the sample mean is approximately normal in sufficiently large samples. This enables us to perform the usual tests. 28

30. THE CENTRAL LIMIT THEOREM
X has mean m and variance s2. X is an estimator of m.
Application of central limit theorem
Approximation for finite samples
Of course, this begs the question of what might be considered to be ‘sufficiently large n’. To answer this question, the analysis must be supplemented by simulation. 29

31. THE CENTRAL LIMIT THEOREM
10 million samples
The figure shows the distribution of for the uniform distribution when n = 1. It is, of course, just the uniform distribution itself, with the mean of 0.5 subtracted. 30

32. THE CENTRAL LIMIT THEOREM
10 million samples
Here is the distribution of when n = 10. It looks very like a normal distribution. 31

33. THE CENTRAL LIMIT THEOREM
10 million samples
Here is the same figure with the theoretical limiting normal distribution, in red. It confirms that the distribution for the sample mean has virtually converged to normality with a sample size of only 10. 32

34. THE CENTRAL LIMIT THEOREM
10 million samples
The curve for n = 25 has been added. There is hardly any change because convergence has already been achieved. 33

35. THE CENTRAL LIMIT THEOREM
10 million samples
Of course, the curve for n = 100 also coincides. In this case, n = 25 was ‘sufficiently large’. Perhaps even n = 10. 34

36. THE CENTRAL LIMIT THEOREM
10 million samples
Now consider the example of the lognormal distribution. Here is the distribution of for n = 1. It is just the lognormal distribution itself with the mean subtracted. 35

37. THE CENTRAL LIMIT THEOREM
limiting normal distribution
10 million samples
Here is the distribution of for n = 10. The theoretical limiting distribution is also shown. Clearly, n = 10 is far from being ‘sufficiently large’. 36

38. THE CENTRAL LIMIT THEOREM
limiting normal distribution
10 million samples
Here is the distribution of for n = 25. It is closer to the limiting distribution but there is still a long way to go. 37

39. THE CENTRAL LIMIT THEOREM
limiting normal distribution
10 million samples
Here is the distribution of for n = It is closer still to the limiting distribution but convergence has not been achieved. In the case of the lognormal distribution, even a sample size of 100 is clearly not ‘sufficiently large’. We should try 200, perhaps 500. 38

40. Copyright Christopher Dougherty 2016.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author.
The content of this slideshow comes from Section R.15 of C. Dougherty, Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre
Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course
EC212 Introduction to Econometrics
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics