Large Sample Distribution Theory Burak Saltoğlu

Outline Convergence in Probability Laws of Large Numbers Convergence of Functions Convergence to a Random Variable Convergence in Distribution: Limiting Distributions Central Limit Theorems Asymtotic Distributions

Convergence in Probability Definition 1: Let xn be a sequence random variable where n is sample size, the random variable xn converges in probability to a constant c if the values that the x may take that are not close to c become increasingly unlikely as n increases. If xn converges to c, then we say, All the mass of the probability distribution concentrates around c.

mean square Convergence Definition mean square convergence:

Convergence in Probability Definition 3: An estimator of a parameter is a consistent estimator iff Theorem 1: The mean of a random sample from any distribution with finite mean μ and variance σ2, is a consistent estimator of μ. Proof:

Convergence in Probability Corrolary to Theorem 1: In random sampling, for any function g(x), if E[g(x)] and Var[g(x)] are finite constants, then Example: Sampling from a normal distribution,

Law of large numbers Weak Law of large numbers Based on convergence in probability Strong form of large numbers Based on Almost sure convergence

Laws of Large Numbers Khinchine’s Weak Law of Large Numbers: Remarks: 1) No finite variance assumption (unllike definition 3). 2) Requires i.i.d sampling

2. Chebychev’s weak law of large numbers There are 2 differences between Khinchine and Chebychev’s LLN Chebychev does not require a convergence to a constant More importantly Chebyshev allows heterogeneity of distributions i.e. İt does not require iid’ness.

Convergence of Functions Theorem 2(Slutsky): For a continious function, g(xn) that is not a function of n, Using Slutsky theorem, we can write some rules of plim.

Convergence of Functions Rules for Probability Limits 1) For plimx=c and plimy=d i) plim (x+y) = c+d ii) plim xy=cd iii) plim (x/y)=c/d 2) For matrices X and Y with plimX=A and plimY=B i) plim X-1=A-1 ii) plim XY=AB

Convergence in Distribution: Limiting Distributions Definition 6: xn with cdf Fn(x) converges in distribution to a random variable with cdf, F(x) if then F(x) is the limiting distribution of xn and can be shown as

Convergence in Distribution: Limiting Distributions Definition 7: The limiting mean and variance of a distribution of a random variable is those of the limiting distribution. Example: Think of Student–t distribution with n-1d.f. It has 0 mean and (n-1)/(n-3) variance.This is the exact distribution of it. However as n grows, it converges to standard normal distribution, that is, So it has 0 limitimg mean and 1 limiting variance.

Convergence in Distribution: Limiting Distributions Rules for Limiting Distributions: 1) If and plim yn=c, then 2) As a corrolary to Slutsky theorem, if and g(x) is a cont. function For example, exact distribution of t2n-1 is F(1,n) but limiting distribution of it is (N(0,1))2=ChiSquare(1), that is

Convergence in Distribution: Limiting Distributions 3) If yn has a limiting distribution and plim(xn-yn)=0 then xn has the same limiting distribution with yn

Central Limit Theorems Lindberg-Levy Central Limit Theorem:

Central Limit Theorems Delta Method: To find the limiting normal distribution of a function, we use a method called ‘delta’, a linear Taylor approximation, to have the following theorem.. Theorem 4: If g(z) is continuous function not involving sample size, n, then

Asymtotic Distributions Definition 8: An asymtotic distribution is a distribution that is used to approximate the true finite sample distribution of a random variable. A good way to derive the asymtotic distribution is using a known limiting distribution. For example, if we know then, we can approximately-asymtotically write as the asymtotic distribution of sample mean

End of the Lecture