Download presentation

1
SOME GENERAL PROBLEMS

2
Problem A certain lion has three possible states of activity each night; they are ‘very active’ (denoted by θ1), ‘moderately active’ (denoted by θ2), and ‘lethargic (lacking energy)’ (denoted by θ3). Also, each night this lion eats people; it eats i people with probability p(i|θ), θ ϵ Θ={θ1, θ2, θ3} . Of course, the probability distribution of the number of people eaten depends on the lion’s activity state θ ϵ Θ. The numeric values are given in the following table.

3
Problem i 1 2 3 4 p(i|θ1) 0.05 0.8 0.1 p(i|θ2) p(i|θ3) 0.9 0.08 0.02 If we are told X=x0 people were eaten last night, how should we estimate the lion’s activity state (θ1, θ2 or θ3)?

4
Solution One reasonable method is to estimate θ as that in Θ for which p(x0|θ) is largest. In other words, the θ ϵ Θ that provides the largest probability of observing what we did observe. : the MLE of θ based on X (Taken from “Dudewicz and Mishra, 1988, Modern Mathematical Statistics, Wiley”)

5
Problem Consider the Laplace distribution centered at the origin and with the shape parameter β, which for all x has the p.d.f. Find MME and MLE of β.

6
Problem Let X1,…,Xn be independent r.v.s each with lognormal distribution, ln N(,2). Find the MMEs of ,2

7
**STATISTICAL INFERENCE PART III**

BETTER OR BEST ESTIMATORS, FISHER INFORMATION, CRAMER-RAO LOWER BOUND (CRLB)

8
**RECALL: EXPONENTIAL CLASS OF PDFS**

If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

9
**EXPONENTIAL CLASS and CSS**

Random Sample from Regular Exponential Class is a css for .

10
**RAO-BLACKWELL THEOREM**

Let X1, X2,…,Xn have joint pdf or pmf f(x1,x2,…,xn;) and let S=(S1,S2,…,Sk) be a vector of jss for . If T is an UE of () and (S)=E(TS), then (S) is an UE of () . (S) is a fn of S, so it is free of . Var((S) ) Var(T) for all . (S) is a better unbiased estimator of () .

11
**RAO-BLACKWELL THEOREM**

Notes: (S)=E(TS) is at least as good as T. For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs.

12
**Example Hogg & Craig, Exercise 10.10 X1,X2~Exp(θ)**

Find joint p.d.f. of ss Y1=X1+X2 for θ and Y2=X2. Show that Y2 is UE of θ with variance θ². Find φ(y1)=E(Y2|Y1) and variance of φ(Y1).

13
**THE MINIMUM VARIANCE UNBIASED ESTIMATOR**

Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (S)=E(TS) is an UE of , i.e.,E[(S)]=E[E(TS)]= and with a smaller variance than Var(T).

14
**LEHMANN-SCHEFFE THEOREM**

Let Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . Y css for . T(y)=fn(y) and E[T(Y)]=. T(Y) is the UMVUE of . So, it is the best unbiased estimator of .

15
**THE MINIMUM VARIANCE UNBIASED ESTIMATOR**

Let Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of . Let U1(Y) and U2(Y) be two function of Y. Since they are UE’s, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.

16
**Example Let X1,X2,…,Xn ~Poi(μ). Find UMVUE of μ. Solution steps:**

Show that is css for μ. Find a statistics (such as S*) that is UE of μ and a function of S. Then, S* is UMVUE of μ by Lehmann-Scheffe Thm.

17
Note The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique.

18
**RECALL: EXPONENTIAL CLASS OF PDFS**

If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

19
**EXPONENTIAL CLASS and CSS**

Random Sample from Regular Exponential Class is a css for . If Y is an UE of , Y is the UMVUE of .

20
**EXAMPLES Let X1,X2,…~Bin(1,p), i.e., Ber(p).**

This family is a member of exponential family of distributions. is a CSS for p. is UE of p and a function of CSS. is UMVUE of p.

21
EXAMPLES X~N(,2) where both and 2 is unknown. Find a css for and 2 .

22
**FISHER INFORMATION AND INFORMATION CRITERIA**

X, f(x;), , xA (not depend on ). Definitions and notations:

23
**FISHER INFORMATION AND INFORMATION CRITERIA**

The Fisher Information in a random variable X: The Fisher Information in the random sample: Let’s prove the equalities above.

24
**FISHER INFORMATION AND INFORMATION CRITERIA**

25
**FISHER INFORMATION AND INFORMATION CRITERIA**

26
**FISHER INFORMATION AND INFORMATION CRITERIA**

The Fisher Information in a random variable X: The Fisher Information in the random sample: Proof of the last equality is available on Casella & Berger (1990), pg

27
**CRAMER-RAO LOWER BOUND (CRLB)**

Let X1,X2,…,Xn be sample random variables. Range of X does not depend on . Y=U(X1,X2,…,Xn): a statistic; does’nt contain . Let E(Y)=m(). Let prove this!

28
**CRAMER-RAO LOWER BOUND (CRLB)**

-1Corr(Y,Z)1 0 Corr(Y,Z)21 Take Z=′(x1,x2,…,xn;) Then, E(Z)=0 and V(Z)=In() (from previous slides).

29
**CRAMER-RAO LOWER BOUND (CRLB)**

Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

30
**CRAMER-RAO LOWER BOUND (CRLB)**

E(Y.Z)=mʹ(), Cov(Y,Z)=mʹ(), V(Z)=In() The Cramer-Rao Inequality (Information Inequality)

31
**CRAMER-RAO LOWER BOUND (CRLB)**

CRLB is the lower bound for the variance of an unbiased estimator of m(). When V(Y)=CRLB, Y is the MVUE of m(). For a r.s., remember that In()=n I(), so,

32
**ASYMPTOTIC DISTRIBUTION OF MLEs**

: MLE of X1,X2,…,Xn is a random sample.

33
**EFFICIENT ESTIMATOR T is an efficient estimator (EE) of if**

T is UE of , and, Var(T)=CRLB T is an efficient estimator (EE) of its expectation, m(), if its variance reaches the CRLB. An EE of m() may not exist. The EE of m(), if exists, is unique. The EE of m() is the unique MVUE of m().

34
**ASYMPTOTIC EFFICIENT ESTIMATOR**

Y is an asymptotic EE of m() if

35
**EXAMPLES A r.s. of size n from X~Poi(θ). Find CRLB for any UE of θ.**

Find UMVUE of θ. Find an EE for θ. Find CRLB for any UE of exp{-2θ}. Assume n=1, and show that is UMVUE of exp{-2θ}. Is this a reasonable estimator?

36
EXAMPLE A r.s. of size n from X~Exp(). Find UMVUE of , if exists.

37
Summary We covered 3 methods for finding good estimators (possibly UMVUE): Rao-Blackwell Theorem (Use a ss T, an UE U, and create a new statistic by E(U|T)) Lehmann-Scheffe Theorem (Use a css T which is also UE) Cramer-Rao Lower Bound (Find an UE with variance=CRLB)

38
Problems Let be a random sample from gamma distribution, Xi~Gamma(2,θ). The p.d.f. of X1 is given by: a) Find a complete and sufficient statistic for θ. b) Find a minimal sufficient statistic for θ. c) Find CRLB for the variance of an unbiased estimator of θ. d) Find a UMVUE of θ.

39
**Problems Suppose X1,…,Xn are independent with density for θ>0**

a) Find a complete sufficient statistic. b) Find the CRLB for the variance of unbiased estimators of 1/θ. c) Find the UMVUE of 1/θ if there is one.

Similar presentations

OK

1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.

1 12. Principles of Parameter Estimation The purpose of this lecture is to illustrate the usefulness of the various concepts introduced and studied in.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google