1. Approximations to Probability Distributions: Limit Theorems

2. Sequences of Random Variables Interested in behavior of functions of random variables such as means, variances, proportions For large samples, exact distributions can be difficult/impossible to obtain Limit Theorems can be used to obtain properties of estimators as the sample sizes tend to infinity –Convergence in Probability – Limit of an estimator –Convergence in Distribution – Limit of a CDF –Central Limit Theorem – Large Sample Distribution of the Sample Mean of a Random Sample

3. Convergence in Probability The sequence of random variables, X 1,…,X n, is said to converge in probability to the constant c, if for every  >0, Weak Law of Large Numbers (WLLN): Let X 1,…,X n be iid random variables with E(X i )=  and V(X i )=  2 < . Then the sample mean converges in probability to  :

4. Proof of WLLN

5. Other Case/Rules Binomial Sample Proportions Useful Generalizations:

6. Convergence in Distribution Let Y n be a random variable with CDF F n (y). Let Y be a random variable with CDF F(y). If the limit as n  of F n (y) equals F(y) for every point y where F(y) is continuous, then we say that Y n converges in distribution to Y F(y) is called the limiting distribution function of Y n If M n (t)=E(e tYn ) converges to M(t)=E(e tY ), then Y n converges in distribution to Y

7. Example – Binomial  Poisson X n ~Binomial(n,p) Let =np  p= /n M n (t) = (pe t + (1-p)) n = (1+p(e t -1)) n = (1+ (e t -1)/n) n Aside: lim n  (1+a/n) n = e a  lim n  M n (t) = lim n  (1+ (e t -1)/n) n = exp( (e t -1)) exp( (e t -1)) ≡ MGF of Poisson( )  X n converges in distribution to Poisson( =np)

8. Example – Scaled Poisson  N(0,1)

10. Central Limit Theorem Let X 1,X 2,…,X n be a sequence of independently and identically distributed random variables with finite mean , and finite variance  2. Then: Thus the limiting distribution of the sample mean is a normal distribution, regardless of the distribution of the individual measurements

11. Proof of Central Limit Theorem (I) Additional Assumptions for this Proof: The moment-generating function of X, M X (t), exists in a neighborhood of 0 (for all |t| 0). The third derivative of the MGF is bounded in a neighborhood of 0 (M (3) (t) ≤ B 0). Elements of Proof Work with Y i =(X i -  )/  Use Taylor’s Theorem (Lagrange Form) Calculus Result: lim n  [1+(a n /n)] n = e a if lim n  a n =a

12. Proof of CLT (II)

13. Proof of CLT (III)

14. Proof of CLT (IV)

15. Proof of CLT (V)