1. Random Sampling, Point Estimation and Maximum Likelihood

2. Statistical Inference  The field of statistical inference consists of those methods used to make decisions or to draw conclusions about a population.  These methods utilize the information contained in a sample from the population in drawing conclusions.  Statistical inference may be divided into two major areas:  Parameter Estimation  Hypothesis Testing

3. Sampling If all members of a population are identical, the population is considered to be homogenous. When individual members of a population are different from each other, the population is considered to be heterogeneous (having significant variation among individuals).

4. What is Sampling? Population Sample Using data to say something (make an inference) with confidence, about a whole (population) based on the study of a only a few (sample). Sampling Frame Sampling Process What you want to talk about What you actually observe in the data Inference

5. Estimation of Parameters Sample mean and sample variance are the two most important parameters of a sample. µ measures the central location of the sample values and s 2 their spread (their variability). Small s 2 may indicate high quality of production, high accuracy of measurement, etc. Note that µ and s 2 will generally vary from sample to sample taken from the same population.

6. Point Estimation of Parameters A point estimate of a parameter is a number (point on real line: p in Binomial, µ and б in Normal Distribution ), which is computed from a given sample and serves as an approximation of the unknown exact value of the parameter of the population.

7. Interval Estimate A point estimate is a statistic taken from a sample and is used to estimate a population parameter. An Interval Estimate is an interval (Confidence Interval).

8. Point Estimates Estimate Population Parameters … with Sample Statistics Mean Proportion Variance Difference

9. Approximation of Mean Method of Moments K th Moment of a Sample x 1, x 2, x 3,…, x n

10. Likelihood Function Consider a random variable X whose probability/ density f(x) depends on a single parameter θ: Discrete Probability of n elements is l = f(x 1 ) f(x 2 ) f(x 3 )…f(x n ) where x j ≤ x ≤ x j + ∆x, j = 1, 2, …, n Since f(x j ) depend on θ, the function l depends on x 1, x 2, x 3,…, x n (given and fixed) and θ.

11. Likelihood Function The likelihood function is: Likelihood Function is an approximation for the unknown value of θ for which l is as large as possible. If l is differentiable function of θ, a necessary condition for to have a maximum in an interval is:

12. Problem 1 Find the maximum likelihood estimate for the parameter µ of a Normal distribution with known variance б 2 = б 0 2.

13. Problem 3 Derive the maximum likelihood estimate for the parameter p of a Binomial distribution.

14. Problem 5 Suppose that 4 times 5 trials were made and in the first 5 trials A happened 2, 1, 4, 4 times, respectively. E stimate p.

15. Problem 7 Consider X = number of independent trials until an event A occurs. Show that X has the probability function f(x) = pq x-1, x = 1, 2, …, where p is probability of A in a single trial and q = 1 – p. Find the maximum likelihood estimate for the parameter p corresponding to a single observation x of X.

16. Problem 9 Apply the maximum likelihood method to Poisson distribution.

17. Problem 11 Find the maximum likelihood estimate of θ in the density f(x) = θe -θx, if x ≥ 0 and f(x) = 0, if x < 0.

18. Problem 13 Compute θ ^ from the sample 1.8, 0.4, 0.8, 0.6, 1.4. Graph the Sample Distribution Function F^(x) and the Distribution Function F(x), with θ = θ^ on the same axes. Do they agree reasonably well.

19. 7-2 General Concepts of Point Estimation 7-2.1 Unbiased Estimators Definition

20. 7-2 General Concepts of Point Estimation Example 7-1

21. 7-2 General Concepts of Point Estimation Example 7-1 (continued)

22. 7-2 General Concepts of Point Estimation 7-2.3 Variance of a Point Estimator Definition Figure 7-1 The sampling distributions of two unbiased estimators

23. 7-2 General Concepts of Point Estimation 7-2.3 Variance of a Point Estimator Theorem 7-1

24. 7-2 General Concepts of Point Estimation 7-2.4 Standard Error: Reporting a Point Estimate Definition

25. 7-2 General Concepts of Point Estimation 7-2.4 Standard Error: Reporting a Point Estimate

26. 7-2 General Concepts of Point Estimation Example 7-2

27. 7-2 General Concepts of Point Estimation Example 7-2 (continued)

28. 7-2 General Concepts of Point Estimation 7-2.6 Mean Square Error of an Estimator Definition

29. 7-2 General Concepts of Point Estimation 7-2.6 Mean Square Error of an Estimator

30. 7-2 General Concepts of Point Estimation 7-2.6 Mean Square Error of an Estimator Figure 7-2 A biased estimator that has smaller variance than the unbiased estimator

31. 7-3 Methods of Point Estimation Definition

32. 7-3 Methods of Point Estimation Example 7-4

33. 7-3 Methods of Point Estimation 7-3.2 Method of Maximum Likelihood Definition

34. 7-3 Methods of Point Estimation Example 7-6

35. 7-3 Methods of Point Estimation Example 7-6 (continued)

36. 7-3 Methods of Point Estimation Figure 7-3 Log likelihood for the exponential distribution, using the failure time data. (a) Log likelihood with n = 8 (original data). (b) Log likelihood if n = 8, 20, and 40.

37. 7-3 Methods of Point Estimation Example 7-9

38. 7-3 Methods of Point Estimation Example 7-9 (continued)

39. 7-3 Methods of Point Estimation Properties of the Maximum Likelihood Estimator

40. 7-3 Methods of Point Estimation The Invariance Property

41. 7-3 Methods of Point Estimation Example 7-10

42. 7-3 Methods of Point Estimation Complications in Using Maximum Likelihood Estimation It is not always easy to maximize the likelihood function because the equation(s) obtained from dL(  )/d  = 0 may be difficult to solve. It may not always be possible to use calculus methods directly to determine the maximum of L(  ).

43. 7-3 Methods of Point Estimation Example 7-11

44. 7-3 Methods of Point Estimation Figure 7-4 The likelihood function for the uniform distribution in Example 7-11.

45. 7-4 Sampling Distributions Statistical inference is concerned with making decisions about a population based on the information contained in a random sample from that population. Definition

46. 7-5 Sampling Distributions of Means Theorem 7-2: The Central Limit Theorem

47. 7-5 Sampling Distributions of Means Figure 7-6 Distributions of average scores from throwing dice. [Adapted with permission from Box, Hunter, and Hunter (1978).]

48. Example 7-13

49. 7-5 Sampling Distributions of Means Figure 7-7 Probability for Example 7-13.

50. 7-5 Sampling Distributions of Means Definition

52. Point Estimation