Presentation is loading. Please wait.

Presentation is loading. Please wait.

SOME GENERAL PROBLEMS 1. Problem A certain lion has three possible states of activity each night; they are ‘very active’ (denoted by θ 1 ), ‘moderately.

Similar presentations


Presentation on theme: "SOME GENERAL PROBLEMS 1. Problem A certain lion has three possible states of activity each night; they are ‘very active’ (denoted by θ 1 ), ‘moderately."— Presentation transcript:

1 SOME GENERAL PROBLEMS 1

2 Problem A certain lion has three possible states of activity each night; they are ‘very active’ (denoted by θ 1 ), ‘moderately active’ (denoted by θ 2 ), and ‘lethargic (lacking energy)’ (denoted by θ 3 ). Also, each night this lion eats people; it eats i people with probability p(i|θ), θ ϵ Θ={θ 1, θ 2, θ 3 }. Of course, the probability distribution of the number of people eaten depends on the lion’s activity state θ ϵ Θ. The numeric values are given in the following table. 2

3 Problem i01234 p(i|θ 1 ) p(i|θ 2 ) p(i|θ 3 ) If we are told X=x 0 people were eaten last night, how should we estimate the lion’s activity state (θ 1, θ 2 or θ 3 )?

4 Solution One reasonable method is to estimate θ as that in Θ for which p(x 0 |θ) is largest. In other words, the θ ϵ Θ that provides the largest probability of observing what we did observe. : the MLE of θ based on X (Taken from “Dudewicz and Mishra, 1988, Modern Mathematical Statistics, Wiley”) 4

5 Problem Consider the Laplace distribution centered at the origin and with the shape parameter β, which for all x has the p.d.f. Find MME and MLE of β. 5

6 Problem Let X 1,…,X n be independent r.v.s each with lognormal distribution, ln N( ,  2 ). Find the MMEs of ,  2 6

7 STATISTICAL INFERENCE PART III BETTER OR BEST ESTIMATORS, FISHER INFORMATION, CRAMER- RAO LOWER BOUND (CRLB) 7

8 8 RECALL: EXPONENTIAL CLASS OF PDFS If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

9 9 EXPONENTIAL CLASS and CSS Random Sample from Regular Exponential Class is a css for .

10 10 RAO-BLACKWELL THEOREM Let X 1, X 2,…,X n have joint pdf or pmf f(x 1,x 2,…,x n ;  ) and let S=(S 1,S 2,…,S k ) be a vector of jss for . If T is an UE of  (  ) and  (S)=E(T  S), then i)  (S) is an UE of  (  ). ii)  (S) is a fn of S, so it is free of . iii)Var(  (S) )  Var(T) for all .  (S) is a better unbiased estimator of  (  ).

11 RAO-BLACKWELL THEOREM Notes:  (S)=E(T  S) is at least as good as T. For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs. 11

12 Example Hogg & Craig, Exercise X 1,X 2 ~Exp(θ) Find joint p.d.f. of ss Y 1 =X 1 +X 2 for θ and Y 2 =X 2. Show that Y 2 is UE of θ with variance θ². Find φ(y 1 )=E(Y 2 |Y 1 ) and variance of φ(Y 1 ). 12

13 THE MINIMUM VARIANCE UNBIASED ESTIMATOR Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then  (S)=E(T  S) is – an UE of , i.e.,E[  (S)]=E[E(T  S)]=  and – with a smaller variance than Var(T). 13

14 14 LEHMANN-SCHEFFE THEOREM Let Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . Y css for . T(y)=fn(y) and E[T(Y)]= .  T(Y) is the UMVUE of .  So, it is the best unbiased estimator of .

15 15 THE MINIMUM VARIANCE UNBIASED ESTIMATOR Let Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of . Let U 1 (Y) and U 2 (Y) be two function of Y. Since they are UE’s, E( U 1 (Y)  U 2 (Y) )=0 imply W(Y)=U 1 (Y)  U 2 (Y)=0 for all possible values of Y. Therefore, U 1 (Y)=U 2 (Y) for all Y.

16 Example Let X 1,X 2,…,X n ~Poi(μ). Find UMVUE of μ. Solution steps: – Show that is css for μ. – Find a statistics (such as S*) that is UE of μ and a function of S. – Then, S* is UMVUE of μ by Lehmann-Scheffe Thm. 16

17 Note The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique. 17

18 18 RECALL: EXPONENTIAL CLASS OF PDFS If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

19 19 EXPONENTIAL CLASS and CSS Random Sample from Regular Exponential Class is a css for . If Y is an UE of , Y is the UMVUE of .

20 20 EXAMPLES Let X 1,X 2,…~Bin(1,p), i.e., Ber(p). This family is a member of exponential family of distributions. is a CSS for p. is UE of p and a function of CSS. is UMVUE of p.

21 21 EXAMPLES X~N( ,  2 ) where both  and  2 is unknown. Find a css for  and  2.

22 22 FISHER INFORMATION AND INFORMATION CRITERIA X, f(x;  ), , x  A (not depend on  ). Definitions and notations:

23 23 FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X : The Fisher Information in the random sample: Let’s prove the equalities above.

24 24 FISHER INFORMATION AND INFORMATION CRITERIA

25 25 FISHER INFORMATION AND INFORMATION CRITERIA

26 26 FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X : The Fisher Information in the random sample: Proof of the last equality is available on Casella & Berger (1990), pg

27 27 CRAMER-RAO LOWER BOUND (CRLB) Let X 1,X 2,…,X n be sample random variables. Range of X does not depend on . Y=U(X 1,X 2,…,X n ) : a statistic; does’nt contain . Let E(Y)=m(  ). Let prove this!

28 28 CRAMER-RAO LOWER BOUND (CRLB) -1  Corr(Y,Z)  1  0  Corr(Y,Z) 2  1   Take Z= ′(x 1,x 2,…,x n ;  ) Then, E(Z)=0 and V(Z)=I n (  ) (from previous slides).

29 29 CRAMER-RAO LOWER BOUND (CRLB) Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

30 30 CRAMER-RAO LOWER BOUND (CRLB) E(Y.Z)=m ʹ (  ), Cov(Y,Z)=m ʹ (  ), V(Z)=I n (  ) The Cramer-Rao Inequality (Information Inequality)

31 31 CRAMER-RAO LOWER BOUND (CRLB) CRLB is the lower bound for the variance of an unbiased estimator of m(  ). When V(Y)=CRLB, Y is the MVUE of m(  ). For a r.s., remember that I n (  )=n I(  ), so,

32 32 ASYMPTOTIC DISTRIBUTION OF MLEs : MLE of  X 1,X 2,…,X n is a random sample.

33 33 EFFICIENT ESTIMATOR T is an efficient estimator (EE) of  if – T is UE of , and, – Var(T)=CRLB T is an efficient estimator (EE) of its expectation, m(  ), if its variance reaches the CRLB. An EE of m(  ) may not exist. The EE of m(  ), if exists, is unique. The EE of m(  ) is the unique MVUE of m(  ).

34 34 ASYMPTOTIC EFFICIENT ESTIMATOR Y is an asymptotic EE of m(  ) if

35 35 EXAMPLES A r.s. of size n from X~Poi(θ). a)Find CRLB for any UE of θ. b)Find UMVUE of θ. c)Find an EE for θ. d)Find CRLB for any UE of exp{-2 θ}. Assume n=1, and show that is UMVUE of exp{- 2 θ}. Is this a reasonable estimator?

36 36 EXAMPLE A r.s. of size n from X~Exp(  ). Find UMVUE of , if exists.

37 Summary We covered 3 methods for finding good estimators (possibly UMVUE): – Rao-Blackwell Theorem (Use a ss T, an UE U, and create a new statistic by E(U|T)) – Lehmann-Scheffe Theorem (Use a css T which is also UE) – Cramer-Rao Lower Bound (Find an UE with variance=CRLB) 37

38 Problems Let be a random sample from gamma distribution, X i ~Gamma(2,θ). The p.d.f. of X 1 is given by: a) Find a complete and sufficient statistic for θ. b) Find a minimal sufficient statistic for θ. c) Find CRLB for the variance of an unbiased estimator of θ. d) Find a UMVUE of θ. 38

39 Problems Suppose X 1,…,X n are independent with density for θ>0 a) Find a complete sufficient statistic. b) Find the CRLB for the variance of unbiased estimators of 1/θ. c) Find the UMVUE of 1/θ if there is one. 39


Download ppt "SOME GENERAL PROBLEMS 1. Problem A certain lion has three possible states of activity each night; they are ‘very active’ (denoted by θ 1 ), ‘moderately."

Similar presentations


Ads by Google