1. Large Sample Distribution Theory
Burak Saltoğlu

2. Outline Convergence in Probability Laws of Large Numbers
Convergence of Functions
Convergence to a Random Variable
Convergence in Distribution: Limiting Distributions
Central Limit Theorems
Asymtotic Distributions

3. Convergence in Probability
Definition 1: Let xn be a sequence random variable where n is sample size,
the random variable xn converges in probability to a constant c if
the values that the x may take that are not close to c become
increasingly unlikely as n increases.
If xn converges to c, then we say,
All the mass of the probability distribution concentrates around c.

4. mean square Convergence
Definition mean square convergence:

5. Convergence in Probability
Definition 3: An estimator of a parameter is a consistent estimator iff
Theorem 1: The mean of a random sample from any distribution with finite mean μ and variance σ2, is a consistent estimator of μ.
Proof:

6. Convergence in Probability
Corrolary to Theorem 1: In random sampling, for any function g(x), if E[g(x)]
and Var[g(x)] are finite constants, then
Example: Sampling from a normal distribution,

7. Law of large numbers Weak Law of large numbers
Based on convergence in probability
Strong form of large numbers
Based on Almost sure convergence

8. Laws of Large Numbers Khinchine’s Weak Law of Large Numbers: Remarks:
1) No finite variance assumption (unllike definition 3).
2) Requires i.i.d sampling

9. 2. Chebychev’s weak law of large numbers
There are 2 differences between Khinchine and Chebychev’s LLN
Chebychev does not require a convergence to a constant
More importantly Chebyshev allows heterogeneity of distributions i.e. İt does not require iid’ness.

10. Convergence of Functions
Theorem 2(Slutsky): For a continious function, g(xn) that is not a function of n,
Using Slutsky theorem, we can write some rules of plim.

11. Convergence of Functions
Rules for Probability Limits
1) For plimx=c and plimy=d
i) plim (x+y) = c+d
ii) plim xy=cd
iii) plim (x/y)=c/d
2) For matrices X and Y with plimX=A and plimY=B
i) plim X-1=A-1
ii) plim XY=AB

12. Convergence in Distribution: Limiting Distributions
Definition 6: xn with cdf Fn(x) converges in distribution to a random variable with cdf, F(x) if
then F(x) is the limiting distribution of xn and can be shown as

13. Convergence in Distribution: Limiting Distributions
Definition 7: The limiting mean and variance of a distribution of a random variable is those of the limiting distribution.
Example: Think of Student–t distribution with n-1d.f. It has 0 mean and
(n-1)/(n-3) variance.This is the exact distribution of it. However as n grows, it converges to standard normal distribution, that is,
So it has 0 limitimg mean and 1 limiting variance.

14. Convergence in Distribution: Limiting Distributions
Rules for Limiting Distributions:
1) If and plim yn=c, then
2) As a corrolary to Slutsky theorem, if and g(x) is a cont. function
For example, exact distribution of t2n-1 is F(1,n) but limiting distribution of it is
(N(0,1))2=ChiSquare(1), that is

15. Convergence in Distribution: Limiting Distributions
3) If yn has a limiting distribution and plim(xn-yn)=0 then xn has the same limiting distribution with yn

16. Central Limit Theorems
Lindberg-Levy Central Limit Theorem:

17. Central Limit Theorems
Delta Method: To find the limiting normal distribution of a function, we use a method called ‘delta’, a linear Taylor approximation, to have the following theorem..
Theorem 4: If g(z) is continuous function not involving sample size, n, then

18. Asymtotic Distributions
Definition 8: An asymtotic distribution is a distribution that is used to approximate the true finite sample distribution of a random variable.
A good way to derive the asymtotic distribution is using a known limiting distribution. For example, if we know
then, we can approximately-asymtotically write
as the asymtotic distribution of sample mean

19. End of the Lecture