1. 1 Sampling Distribution Theory ch6

2. 2  Two independent R.V.s have the joint p.m.f. = the product of individual p.m.f.s.  Ex6.1-1: X1is the number of spots on a fair die.  f 1 (x 1 )=1/6, x 1 =1,2,3,4,5,6.  X 2 is the number of heads on 4 indep. Tosses of a fair coin.  If X 1 and X 2 are indep.  If X 1 and X 2 have the same p.m.f., their joint p.m.f. is f(x 1 )*f(x 2 ).  This collection of X 1 and X 2 is a random sample of size n=2 from f(x). P(X 1 =1,2 & X 2 =3,4)

3. 3 Linear Functions of Indep. R.V.s  Suppose a function Y=X 1 +X 2, S 1 ={1,2,3,4,5,6}, S 2 ={0,1,2,3,4}.  Y will have the support S={1,2,…,9,10}.  The p.m.f. g(y) of Y is  The mathematical expectation (or expected value) of a function Y=u(X 1,X 2 ) is  If X 1 and X 2 are indep. 

4. 4 Example  Ex6.1-2: X 1 and X 2 are two indep. R.V. from casting a die twice.  E(X 1 )=E(X 2 )=3.5; Var(X 1 )=Var(X 2 )=35/12; E(X 1 X 2 )=E(X 1 )E(X 2 )=12.25;  E[(X 1 -3.5)(X 2 -3.5)]=E(X 1 -3.5)E(X 2 -3.5)=0.  Y=X 1 +X 2 →E(Y)= E(X 1 )+E(X 2 )=7;  Var(Y)=E[(X 1 +X 2 -7) 2 ]=Var(X 1 )+Var(X 2 )=35/6.  The p.m.f. g(y) of Y with S={2,3,4,…,12} is

5. 5 General Cases  If X 1,…,X n are indep., then their joint p.d.f. is f 1 (x 1 ) …f n (x n ).  The expected value of the product u 1 (x 1 ) …u n (x n ) is the product of the expected values of u 1 (x 1 ),…, u n (x n ).  If all these n distributions are the same, the collection of n indep. and identically distributed (iid) random variables, X 1,…,X n, is a random sample of size n from that common distribution.  Ex6.1-3: X 1, X 2, X 3, are a random sample from a distribution with p.d.f. f(x)=e -x, 0<x<∞.  The joint p.d.f. is P(0 < X 1 < 1,2 < X 2 < 4,3 < X 3 < 7)

6. 6 Distributions of Sums of Indep. R.V.s  Distributions of the product of indep. R.V.s are straightforward.  However, distributions of the sum of indep. R.V.s are fetching:  First, the joint p.m.f. or p.d.f. is a simple product.  However, through summation, these R.V.s interfere with each other.  Care must be taken to distinguish some sum value happens more frequently than the others.  Sampling distribution theory is to derive the distributions of the functions of R.V.s (random variables).  The sample mean and variance are famous functions.  The summation of R.V.s is another example.

7. 7 Example  Ex6.2-1: X 1 and X 2 are two indep. R.V.s from casting a 4- sided die twice.  The p.m.f. f(x)=, x=1,2,3,4.  The p.m.f. of Y=X 1 +X 2 with S={2,3,4,5,6,7,8} is g(y): (convolution formula) g(2)= g(3)=  g(y)=

8. 8 Theorems  Thm6.2-1: X 1,…,X n are indep. and have the joint p.m.f. is f 1 (x 1 ) …f n (x n ). Y=u(X 0,…,X n ) have the p.m.f. g(y)  Then if the summations exist.  For continuous type, integrals replace the summations.  Thm6.2-2: X 1,…,Xn are indep. and their means exist,  Then,  Thm6.2-3: If X 1,…,X n are indep. with means μ 1,…,μ n and variances σ 1 2,…,σ n 2, then Y=a 1 X 1 +…+a n X n, where a i ’s are real constants, have the mean and variance:  Ex6.2-3: X 1 & X 2 are indep. with μ 1 = -4, μ 2 =3 and σ 1 2 =4, σ 2 2 =9.  Y=3X 1 -2X 2 has

9. 9 Moment-generating Functions  Ex6.2-4: X 1,…,X n are a random sample of size n from a distribution with mean μand variance σ 2 ; then  The sample mean:  Thm6.2-4: If X 1,…,X n are indep. R.V.s with moment- generating functions, i=1..n, then Y=a 1 X 1 +…+a n X n, has the moment-generating  Cly6.2-1: If X 1,…,X n are indep. R.V.s with M(t),  then Y=X 1 +…+X n has M Y (t)=[M(t)] n.  has M Y (t)

10. 10 Examples  Ex6.2-5: X 1,…,X n are the outcomes on n Bernoulli trials.  The moment-generating function of X i, i=1..n, is M(t)=q+pe t.  Then Y=X 1 +…+X n has M Y (t)=[q+pe t ] n, which is b(n,p).  Ex6.2-6: X 1,X 2,X 3 are the outcomes of a random sample of size n=3 from the exponential distribution with mean θand M(t)=1/(1-θt), t<1/θ.  Then Y=X 1 +X 2 +X 3 has M Y (t)=[1/(1-θt)] 3 =(1-θt) -3, which is a gamma distribution with α=3, and θ.  has a gamma distribution with α=3, and θ/3.

11. 11 Statistics on Normal Distributions  Thm6.3-1: X 1,…,X n are the outcomes on a random sample of size n from the normal distribution N(μ,σ 2 ).  The distribution of the sample mean is N(μ,σ 2 /n).  Pf:  Thm6.3-2: X 1,…,X n are independent and have χ 2 (r 1 ),…, χ 2 (r n ) distributions, respectively ; Then, Y=X 1 +…+X n is χ 2 (r 1 +…+r n ).  Pf:  Thm6.3-3: Z 1,…,Z n are independent and all have N(0,1); Then, W=Z 1 2 +…+Z n 2 is χ 2 (n). Pf: Thm4.4-2: If X is N(μ,σ 2 ), then V=[(X-μ)/σ] 2 =Z 2 is χ 2 (1). Thm6.3-2: Y=X 1 +…+X n is χ 2 (r 1 +…+r n )= χ 2 (n) in this case. Thm4.4-2: If X is N(μ,σ 2 ), then V=[(X-μ)/σ] 2 =Z 2 is χ 2 (1). Thm6.3-2: Y=X 1 +…+X n is χ 2 (r 1 +…+r n )= χ 2 (n) in this case. M X (t) M Y (t)=

12. 12 Example  Fig.6.3-1: p.d.f.s of means of samples from N(50,16).  is N(50, 16/n).

13. 13 Theoretical Mean and Sample Mean  Cly6.3-1: Z 1,…,Z n are independent and have N(μ i,σ i 2 ),i=1..n; Then, W=[(Z 1 -μ 1 )/σ i 2 ] 2 +…+[(Z n -μ n )/σ n 2 ] 2 is χ 2 (n).  Thm6.3-4: X 1,…,X n are the outcomes on a random sample of size n from the normal distribution N(μ,σ 2 ). Then,  The sample mean & variance are indep.  is χ 2 (n-1) Pf: (a) omitted; (b): As theoretical mean is replaced by the sample mean, one degree of freedom is lost! M V (t)

14. 14 Linear Combinations of N(μ,σ 2 )  Ex6.3-2: X 1,X 2,X 3,X 4 are a random sample of size 4 from the normal distribution N(76.4,383). Then P(0.711  W  7.779)=0.9-0.05=0.85, P(0.352  V  6.251)=0.9-0.05=0.85  Thm6.3-5: If X 1,…,X n are n mutually indep. normal variables with means μ 1,…,μ n & variances σ 1 2,…,σ n 2, then the linear function has the normal distribution Pf: By moment-generating function, …  Ex6.3-3: X 1 :N(693.2,22820) and X 2 :N(631.7,19205) are indep. Find P(X 1 >X 2 )  Y=X 1 -X 2 is N(61.5,42025).  P(X 1 >X 2 )=P(Y>0)=

15. 15 Central Limit Theorem  Ex6.2-4: X 1,…,X n are a random sample of size n from a distribution with mean μand variance σ 2 ; then The sample mean:  Thm6.4-1: (Central Limit Theorem) If is the mean of a random sample X 1,…,X n of size n from some distribution with a finite mean μand a finite positive variance σ 2, then the distribution of is N(0, 1) in the limit as n →∞. Even if X i is not N(μ,σ 2 ). W= if n is large W

16. 16 More Examples  Ex6.4-1: Let denote the mean of a random sample of size n=15 from the distribution whose p.d.f. is f(x)=3x 2 /2, -1<x<1.  μ=0, σ 2 =3/5.  Ex6.4-2: Let X 1,…,X 20 be a random sample of size 20 from the uniform distribution U(0,1).  μ=½, σ 2 =1/12; Y=X 1 +…+X 20.  Ex6.4-3: Let denote the mean of a random sample of size n=25 from the distribution whose p.d.f. is f(x)=x 3 /4, 0<x<2  μ=1.6, σ 2 =8/75.

17. 17 How large of size n is sufficient?  If n=25, 30 or larger, the approximation is generally good.  If the original distribution is symmetric, unimodal and of continuous type, n can be as small as 4 or 5.  If the original is like normal, n can be lowered to 2 or 3.  If it is exactly normal, n=1 or more is just good.  However, if the original is highly skew, n must be quite large.  Ex6.4-4: Let X 1,…,X 4 be a random sample of size 4 from the uniform distribution U(0,1) with p.d.f. f(x)=1, 0<x<1.  μ=½, σ 2 =1/12; Y=X 1 +X 2. Y=X 1 +…+X 4.

18. 18 Graphic Illustration  Fig.6.4-1: Sum of n U(0, 1) R.V.s  N( n(1/2), n(1/12) ) p.d.f.

19. 19 Skew Distributions  Suppose f(x) and F(x) are the p.d.f. and distribution function of a random variable X with mean μ and variance σ 2.  Ex6.4-5: Let X 1,…,X n be a random sample of size n from a chi-square distribution χ 2 (1).  Y=X 1 +…+X n is χ 2 (n), E(Y)=n, Var(Y)=2n. n=20 or 100 →N(, ). n=20 or 100 →N(, ).

20. 20 Graphic Illustration  Fig.6.4-2: The p.d.f.s of sums of χ 2 (1), transformed so that their mean is equal to zero and variance is equal to one, becomes closer to the N(0, 1) as the number of degrees of freedom increases.

21. 21 Simulation of R.V. X with f(x) & F(x)  Random number generator will produce values y’s for U(0,1).  Since F(x)=U(0,1)=Y, x=F -1 (y) is an observed or simulated value of X.  Ex.6.4-6: Let X 1,…,X n be a random sample of size n from the distribution with f(x), F(x), mean μand variance σ 2.  1000 random samples are simulated to compute the values of W.  A histogram of these values are grouped into 21 classes of equal width. f(x)=(x+1)/2, F(x)=(x+1) 2 /4, f(x)=3x 2 /2, F(x)=(x 3 +1)/2, -1<x<1; μ=1/3, σ 2 =2/9. -1<x<1; μ=0, σ 2 =3/5. N(0,1) N(0,1)

22. 22 Approximation of Discrete Distributions  Let X 1,…,X n be a random sample from a Bernoulli distribution with μ=p and σ 2 =npq, 0<p<1.  Thus, Y=X 1 +…+X n is binomial b(n,p). →N(np,npq) as n →∞.  Rule: n is “sufficiently large” if np  5 and nq  5.  If p deviates from 0.5 (skew!!), n need to be larger.  Ex.6.5-1: Y, b(10,1/2), can be approximated by N(5,2.5).   Ex.6.5-2: Y, b(18,1/6), can be hardly approx. by N(3,2.5), ∵ 3<5.

23. 23 Another Example  Ex6.5-4: Y is b(36,1/2).  Correct Probability:  Approximation of Binomial Distribution b(n,p): Good approx.!

24. 24 Approximation of Poisson Distribution  Approximation of a Poisson Distribution Y with mean λ:  Ex6.5-5: X that has a Poisson distribution with mean 20 can be seen as the sum Y of the observations of a random sample of size 20 from a Poisson distribution with mean 1.  Correct Probability: Fig.6.5-3: Normal approx. of the Poisson Probability Histogram

25. 25 Student ’ s T Distribution  Gossett, William Sealy published “t-test” in Biometrika1908 to measure the confidence interval, the deviation of “small samples” from the “real”.  Suppose the underlying distribution is normal with unknown σ 2.  Fig.6.6-1(a): The T p.d.f. becomes closer to the N(0, 1) p.d.f. as the number of degrees of freedom increases. t α (r) is the 100(1-α) percentile, or the upper 100αpercent point. [Table VI, p.658] f(t)= Only depends on r!

26. 26 Examples  Ex6.6-1: Suppose T has a t distribution with r=7.  From Table VI on p.658,  Ex6.6-2: Suppose T has a t distribution with r=14.  Find a constant c, such that P(|T|<c)=0.9  From Table VI on p.658,

27. 27 F Distribution: F(r 1, r 2 )  From two indep. random samples of size n 1 & n 2 from N(μ 1,σ 1 2 ) & N(μ 2,σ 2 2 ),some comparisons can be performed, such as:  F α (r 1,r 2 )is the upper 100α percent point. [Table VII, p.659~663]  Ex6.6-4: Suppose F has a F(4,9) distribution.  Find constants c & d, such that P(F  c)=0.01, P(F  d)=0.05 If F is F(6,9): c=F 0.99 (4,9)