Presentation is loading. Please wait.

Presentation is loading. Please wait.

Point Estimation Notes of STAT 6205 by Dr. Fan. Overview Section 6.1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics.

Similar presentations


Presentation on theme: "Point Estimation Notes of STAT 6205 by Dr. Fan. Overview Section 6.1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics."— Presentation transcript:

1 Point Estimation Notes of STAT 6205 by Dr. Fan

2 Overview Section 6.1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics o Definition o Exponential family o Mean square error (how to choose an estimator) 6205-Ch6 2

3 Big Picture Goal: To study the unknown distribution of a population Method: Get a representative/random sample and use the information obtained in the sample to make statistical inference on the unknown features of the distribution Statistical Inference has two parts: o Estimation (of parameters) o Hypothesis testing Estimation: o Point estimation: use a single value to estimate a parameter o Interval estimation: find an interval covering the unknown parameter 6205-Ch6 3

4 Point Estimator Biased/unbiased: an estimator is called unbiased if its mean is equal to the parameter of estimate; otherwise, it is biased Example: X_bar is unbiased for estimating mu 6205-Ch6 4

5 Maximum Likelihood Estimation Given a random sample X1, X2, …, Xn from a distribution f(x;  ) where  is a (unknown) value in the parameter space W. Likelihood function vs. joint pdf Maximum Likelihood Estimator (m.l.e.) of , denoted as is the value  which maximizes the likelihood function, given the sample X1, X2, …, Xn Ch6 5

6 Examples/Exercises Problem 1: To estimate p, the true probability of heads up for a given coin. Problem 2: Let X1, X2, …, Xn be a random sample from a Exp(mu) distribution. Find the m.l.e. of mu. Problem 3: Let X1, X2, …, Xn be a random sample from a Weibull(a=3,b) distribution. Find the m.l.e. of b. Problem 4: Let X1, X2, …, Xn be a random sample from a N( ,  ^2) distribution. Find the m.l.e. of  and . Problem 5: Let X1, X2, …, Xn be a random sample from a Weibull(a,b) distribution. Find the m.l.e. of a and b Ch6 6

7 Method of Moments Idea: Set population moments = sample moments and solve for parameters Formula: When the parameter q is r-dimensional, solve the following equations for q : 6205-Ch6 7

8 Examples/Exercises Given a random sample from a population Problem 1: Find the m.m.e. of m for a Exp( m ) population. Exercise 1: Find the m.m.e. of m and s for a N( m,s ^2) population Ch6 8

9 Sufficient Statistics Idea: The “sufficient” statistic contains all information about the unknown parameter; no other statistic can provide additional information as to the unknown parameter. If for any event A, P[A|Y=y] does not depend on the unknown parameter, then the statistic Y is called “sufficient” (for the unknown parameter). Any one-to-one mapping of a sufficient statistic Y is also sufficient. Sufficient statistics do not need to be estimators of the parameter Ch6 9

10 Sufficient Statistics 6205-Ch6 10

11 Examples/Exercises Let X1, X2, …, Xn be a random sample from f(x) Problem: Let f be Poisson(a). Prove that 1.X-bar is sufficient for the parameter a 2.The m.l.e. of a is a function of the sufficient statistic Exercise: Let f be Bin(n, p). Prove that X-bar is sufficient for p (n is known). Hint: find a sufficient statistic Y for p and then show that X-bar is a function of Y 6205-Ch6 11

12 Exponential Family 6205-Ch6 12

13 Examples/Exercises Example 1: Find a sufficient statistic for p for Bin(n, p) Example 2: Find a sufficient statistic for a for Poisson(a) Exercise: Find a sufficient statistic for m for Exp( m ) 6205-Ch6 13

14 Joint Sufficient Statistics Example: Prove that X-bar and S^2 are joint sufficient statistics for m and s of N( m, s ^2) 6205-Ch6 14

15 Application of Sufficience 6205-Ch6 15

16 Example Consider a Weibull distribution with parameter(a=2, b) 1)Find a sufficient statistic for b 2)Find an unbiased estimator which is a function of the sufficient statistic found in 1) 6205-Ch6 16

17 Good Estimator? Criterion: mean square error 6205-Ch6 17

18 Example Which of the following two estimator of variance is better? 6205-Ch6 18


Download ppt "Point Estimation Notes of STAT 6205 by Dr. Fan. Overview Section 6.1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics."

Similar presentations


Ads by Google