Estimators and estimates: An estimator is a mathematical formula. An estimate is a number obtained by applying this formula to a set of sample data. 1 ESTIMATORS It is important to distinguish between estimators and estimates. Definitions are given above. EMU, ECON 503, M. Balcılar

Population characteristicEstimator Mean:  X 4 ESTIMATORS A common example of an estimator is the sample mean, which is the usual estimator of the population mean. EMU, ECON 503, M. Balcılar

Population characteristicEstimator Mean:  X 4 ESTIMATORS Here it is defined for a random variable X and a sample of n observations. EMU, ECON 503, M. Balcılar

Population characteristicEstimator Mean:  X Population variance: 4 ESTIMATORS Another common estimator is s 2, defined above. It is used to estimate the population variance,  X 2. EMU, ECON 503, M. Balcılar

Estimators are random variables 8 ESTIMATORS An estimator is a special kind of random variable. We will demonstrate this in the case of the sample mean. EMU, ECON 503, M. Balcılar

Estimators are random variables 8 ESTIMATORS We saw in the previous sequence that each observation on X can be decomposed into a fixed component and a random component. EMU, ECON 503, M. Balcılar

Estimators are random variables 8 ESTIMATORS So the sample mean is the average of n fixed components and n random components. EMU, ECON 503, M. Balcılar

Estimators are random variables 8 ESTIMATORS It thus has a fixed component  X and a random component u, the average of the random components in the observations in the sample. EMU, ECON 503, M. Balcılar

10 ESTIMATORS probability density function of X XX X XX X probability density function of X The graph compares the probability density functions of X and X. As we have seen, they have the same fixed component. However the distribution of the sample mean is more concentrated. EMU, ECON 503, M. Balcılar

10 ESTIMATORS Its random component tends to be smaller than that of X because it is the average of the random components in all the observations, and these tend to cancel each other out. probability density function of X XX X XX X probability density function of X EMU, ECON 503, M. Balcılar

Unbiasedness of X: 1 UNBIASEDNESS AND EFFICIENCY Suppose that you wish to estimate the population mean  X of a random variable X given a sample of observations. We will demonstrate that the sample mean is an unbiased estimator, but not the only one. EMU, ECON 503, M. Balcılar

Unbiasedness of X: 2 UNBIASEDNESS AND EFFICIENCY We use the second expected value rule to take the (1/n) factor out of the expectation expression. EMU, ECON 503, M. Balcılar

Unbiasedness of X: 3 UNBIASEDNESS AND EFFICIENCY Next we use the first expected value rule to break up the expression into the sum of the expectations of the observations. EMU, ECON 503, M. Balcılar

Unbiasedness of X: 4 UNBIASEDNESS AND EFFICIENCY Each expectation is equal to  X, and hence the expected value of the sample mean is  X. EMU, ECON 503, M. Balcılar

probability density function XX estimator B How do we choose among them? The answer is to use the most efficient estimator, the one with the smallest population variance, because it will tend to be the most accurate. UNBIASEDNESS AND EFFICIENCY estimator A 12 EMU, ECON 503, M. Balcılar

probability density function estimator B In the diagram, A and B are both unbiased estimators but B is superior because it is more efficient. UNBIASEDNESS AND EFFICIENCY estimator A 13 XX EMU, ECON 503, M. Balcılar

1 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE Suppose that you have alternative estimators of a population characteristic , one unbiased, the other biased but with a smaller population variance. How do you choose between them? probability density function  estimator B estimator A EMU, ECON 503, M. Balcılar

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE One way is to define a loss function which reflects the cost to you of making errors, positive or negative, of different sizes. 2 error (positive)error (negative) loss EMU, ECON 503, M. Balcılar

3 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE A widely-used loss function is the mean square error of the estimator, defined as the expected value of the square of the deviation of the estimator about the true value of the population characteristic. probability density function  estimator B EMU, ECON 503, M. Balcılar

4 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE The mean square error involves a trade-off between the population variance of the estimator and its bias. Suppose you have a biased estimator like estimator B above, with expected value  Z. probability density function ZZ bias estimator B EMU, ECON 503, M. Balcılar

5 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE The mean square error can be shown to be equal to the sum of the population variance of the estimator and the square of the bias. probability density function ZZ bias estimator B EMU, ECON 503, M. Balcılar

6 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE To demonstrate this, we start by subtracting and adding  Z. EMU, ECON 503, M. Balcılar

7 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE We expand the quadratic using the rule (a + b) 2 = a 2 + b 2 + 2ab, where a = Z -  Z and b =  Z - . EMU, ECON 503, M. Balcılar

8 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE We use the first expected value rule to break up the expectation into its three components. EMU, ECON 503, M. Balcılar

9 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE The first term in the expression is by definition the population variance of Z. EMU, ECON 503, M. Balcılar

10 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE (  Z -  ) is a constant, so the second term is a constant. EMU, ECON 503, M. Balcılar

11 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE In the third term, (  Z -  ) may be brought out of the expectation, again because it is a constant, using the second expected value rule. EMU, ECON 503, M. Balcılar

12 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE Now E(Z) is  Z, and E(-  Z ) is -  Z. EMU, ECON 503, M. Balcılar

13 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE Hence the third term is zero and the mean square error of Z is shown be the sum of the population variance of Z and the bias squared. EMU, ECON 503, M. Balcılar

14 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE In the case of the estimators shown, estimator B is probably a little better than estimator A according to the MSE criterion. probability density function  estimator B estimator A EMU, ECON 503, M. Balcılar

n The sample mean is the usual estimator of a population mean, for reasons discussed in the previous sequence. In this sequence we will see how its properties are affected by the sample size. probability density function of X n = 1 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x EMU, ECON 503, M. Balcılar

n Suppose that a random variable X has population mean 100 and standard deviation 50, as in the diagram. Suppose that we do not know the population mean and we are using the sample mean to estimate it n = 1 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n The sample mean will have the same population mean as X, but its standard deviation will be 50/, where n is the number of observations in the sample n = 1 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n The larger is the sample, the smaller will be the standard deviation of the sample mean n = 1 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n If n is equal to 1, the sample consists of a single observation. X is the same as X and its standard deviation is n = 1 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n We will see how the shape of the distribution changes as the sample size is increased n = 4 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n The distribution becomes more concentrated about the population mean n = 25 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n To see what happens for n greater than 100, we will have to change the vertical scale n = 100 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n We have increased the vertical scale by a factor of n = 100 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n The distribution continues to contract about the population mean n = 1000 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar

n In the limit, the variance of the distribution tends to zero. The distribution collapses to a spike at the true value. The sample mean is therefore a consistent estimator of the population mean n = 5000 EFFECT OF INCREASING THE SAMPLE SIZE ON THE DISTRIBUTION OF x probability density function of X EMU, ECON 503, M. Balcılar