TYPE II ERROR AND THE POWER OF A TEST A Type I error occurs when the null hypothesis is rejected when it is in fact true. A Type II error occurs when the.

Slides:

Advertisements

Similar presentations

EC220 - Introduction to econometrics (chapter 2)

Advertisements

EC220 - Introduction to econometrics (review chapter)

THE ERROR CORRECTION MODEL 1 The error correction model is a variant of the partial adjustment model. As with the partial adjustment model, we assume a.

1 This very short sequence presents an important definition, that of the independence of two random variables. Two random variables X and Y are said to.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 10) Slideshow: introduction to maximum likelihood estimation Original citation: Dougherty,

1 THE DISTURBANCE TERM IN LOGARITHMIC MODELS Thus far, nothing has been said about the disturbance term in nonlinear regression models.

EC220 - Introduction to econometrics (chapter 7)

1 XX X1X1 XX X Random variable X with unknown population mean  X function of X probability density Sample of n observations X 1, X 2,..., X n : potential.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: testing a hypothesis relating to a regression coefficient Original citation:

1 THE NORMAL DISTRIBUTION In the analysis so far, we have discussed the mean and the variance of a distribution of a random variable, but we have not said.

1 PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE red This sequence provides an example of a discrete random variable. Suppose that you.

MEASUREMENT ERROR 1 In this sequence we will investigate the consequences of measurement errors in the variables in a regression model. To keep the analysis.

1 ASSUMPTIONS FOR MODEL C: REGRESSIONS WITH TIME SERIES DATA Assumptions C.1, C.3, C.4, C.5, and C.8, and the consequences of their violations are the.

00  sd  0 –sd  0 –1.96sd  0 +sd 2.5% CONFIDENCE INTERVALS probability density function of X null hypothesis H 0 :  =  0 In the sequence.

EXPECTED VALUE OF A RANDOM VARIABLE 1 The expected value of a random variable, also known as its population mean, is the weighted average of its possible.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: expected value of a function of a random variable Original citation:

1 We will now consider the distributional properties of OLS estimators in models with a lagged dependent variable. We will do so for the simplest such.

TESTING A HYPOTHESIS RELATING TO THE POPULATION MEAN 1 This sequence describes the testing of a hypothesis at the 5% and 1% significance levels. It also.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: confidence intervals Original citation: Dougherty, C. (2012) EC220.

EC220 - Introduction to econometrics (review chapter)

TESTING A HYPOTHESIS RELATING TO A REGRESSION COEFFICIENT This sequence describes the testing of a hypotheses relating to regression coefficients. It is.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: continuous random variables Original citation: Dougherty, C. (2012)

1 A MONTE CARLO EXPERIMENT In the previous slideshow, we saw that the error term is responsible for the variations of b 2 around its fixed component 

Christopher Dougherty EC220 - Introduction to econometrics (chapter 3) Slideshow: prediction Original citation: Dougherty, C. (2012) EC220 - Introduction.

1 In the previous sequence, we were performing what are described as two-sided t tests. These are appropriate when we have no information about the alternative.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 10) Slideshow: maximum likelihood estimation of regression coefficients Original citation:

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: the normal distribution Original citation: Dougherty, C. (2012)

1 In a second variation, we shall consider the model shown above. x is the rate of growth of productivity, assumed to be exogenous. w is now hypothesized.

1 PREDICTION In the previous sequence, we saw how to predict the price of a good or asset given the composition of its characteristics. In this sequence,

1 UNBIASEDNESS AND EFFICIENCY Much of the analysis in this course will be concerned with three properties of estimators: unbiasedness, efficiency, and.

FIXED EFFECTS REGRESSIONS: WITHIN-GROUPS METHOD The two main approaches to the fitting of models using panel data are known, for reasons that will be explained.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: sampling and estimators Original citation: Dougherty, C. (2012)

THE DUMMY VARIABLE TRAP 1 Suppose that you have a regression model with Y depending on a set of ordinary variables X 2,..., X k and a qualitative variable.

1 CONTINUOUS RANDOM VARIABLES A discrete random variable is one that can take only a finite set of values. The sum of the numbers when two dice are thrown.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: conflicts between unbiasedness and minimum variance Original citation:

THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE 1 In this short sequence we shall decompose a random variable X into its fixed and random components.

1 General model with lagged variables Static model AR(1) model Model with lagged dependent variable Methodologically, in developing a regression specification.

Confidence intervals were treated at length in the Review chapter and their application to regression analysis presents no problems. We will not repeat.

CONSEQUENCES OF AUTOCORRELATION

ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE 1 This sequence derives an alternative expression for the population variance of a random variable. It provides.

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

1 t TEST OF A HYPOTHESIS RELATING TO A POPULATION MEAN The diagram summarizes the procedure for performing a 5% significance test on the slope coefficient.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 12) Slideshow: footnote: the Cochrane-Orcutt iterative process Original citation: Dougherty,

Simple regression model: Y =  1 +  2 X + u 1 We have seen that the regression coefficients b 1 and b 2 are random variables. They provide point estimates.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 9) Slideshow: instrumental variable estimation: variation Original citation: Dougherty,

1 We will now look at the properties of the OLS regression estimators with the assumptions of Model B. We will do this within the context of the simple.

1 Y SIMPLE REGRESSION MODEL Suppose that a variable Y is a linear function of another variable X, with unknown parameters  1 and  2 that we wish to estimate.

1 We will continue with a variation on the basic model. We will now hypothesize that p is a function of m, the rate of growth of the money supply, as well.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: alternative expression for population variance Original citation:

1 11 00 This sequence explains the logic behind a one-sided t test. ONE-SIDED t TESTS  0 +sd  0 –sd  0 –2sd  1 +2sd 2.5% null hypothesis:H 0 :

1 ASYMPTOTIC PROPERTIES OF ESTIMATORS: THE USE OF SIMULATION In practice we deal with finite samples, not infinite ones. So why should we be interested.

Definition of, the expected value of a function of X : 1 EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE To find the expected value of a function of.

4 In our case, the starting point should be the model with all the lagged variables. DYNAMIC MODEL SPECIFICATION General model with lagged variables Static.

HETEROSCEDASTICITY 1 This sequence relates to Assumption A.4 of the regression model assumptions and introduces the topic of heteroscedasticity. This relates.

INSTRUMENTAL VARIABLES 1 Suppose that you have a model in which Y is determined by X but you have reason to believe that Assumption B.7 is invalid and.

1 HETEROSCEDASTICITY: WEIGHTED AND LOGARITHMIC REGRESSIONS This sequence presents two methods for dealing with the problem of heteroscedasticity. We will.

1 ESTIMATORS OF VARIANCE, COVARIANCE, AND CORRELATION We have seen that the variance of a random variable X is given by the expression above. Variance.

Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: confidence intervals Original citation: Dougherty, C. (2012) EC220 -

1 We will illustrate the heteroscedasticity theory with a Monte Carlo simulation. HETEROSCEDASTICITY: MONTE CARLO ILLUSTRATION 1 standard deviation of.

WHITE TEST FOR HETEROSCEDASTICITY 1 The White test for heteroscedasticity looks for evidence of an association between the variance of the disturbance.

Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: independence of two random variables Original citation: Dougherty,

Christopher Dougherty EC220 - Introduction to econometrics (chapter 1) Slideshow: simple regression model Original citation: Dougherty, C. (2012) EC220.

FOOTNOTE: THE COCHRANE–ORCUTT ITERATIVE PROCESS 1 We saw in the previous sequence that AR(1) autocorrelation could be eliminated by a simple manipulation.

Introduction to Econometrics, 5th edition

Introduction to Econometrics, 5th edition

Introduction to Econometrics, 5th edition

Introduction to Econometrics, 5th edition

Introduction to Econometrics, 5th edition

Introduction to Econometrics, 5th edition

Presentation transcript:

TYPE II ERROR AND THE POWER OF A TEST A Type I error occurs when the null hypothesis is rejected when it is in fact true. A Type II error occurs when the null hypothesis is not rejected when it is in fact false. 1 f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

We will see that, in general, there is a trade-off between the risk of making a Type I error and the risk of making a Type II error. 2 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

We will consider the case where the null hypothesis, H 0 :  =  0 is false and the actual value of  is  1. This is shown in the figure. 3 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

If the null hypothesis is tested, it will be rejected only if X lies in one of the rejection regions associated with it. 4 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

To determine the rejection regions, we draw the distribution of X conditional on H 0 being true. The distribution is marked with a dashed curve to emphasize that H 0 is not actually true. 5 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

The rejection regions for a (two-sided) 5 percent test, given this distribution, are marked on the diagram. 6 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

If X lies in the acceptance region, H 0 will not be rejected, and so a Type II error will occur. What is the probability of this happening? To determine this, we now turn to the actual distribution of X, given that  =  1. This is the solid curve on the right. 7 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 2.5% X rejection region acceptance region (5% test) 00 11

The probability of X lying in the acceptance region for H 0 is the area under this curve in the acceptance region. It is the blue area in the figure. In this particular case, the probability of X lying within the acceptance region for H 0, thus causing a Type II error, is TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. X 0.15 true distribution false distribution rejection region acceptance region (5% test) 00 11

The probability of rejecting the null hypothesis, when it is false, is known as the power of a test. By definition, it is equal to 1 minus the probability of making a Type II error. It is therefore 0.85 in this example. 9 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. X 0.15 true distribution false distribution 0.85 rejection region acceptance region (5% test) 00 11

The power depends on the distance between the value of  under the false null hypothesis and its actual value. The closer that the actual value is to  0, the harder it is to demonstrate that H 0 :  =  0 is false. 10 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X)  0 –1.96s.d. 11 X 0.15 true distribution false distribution 0.85 rejection region acceptance region (5% test) 00

This is illustrated in the figure.  0 is the same as in the previous figure, and so the acceptance region and rejection regions for the test of H 0 :  =  0 are the same as in the previous figure. 11 TYPE II ERROR AND THE POWER OF A TEST rejection region acceptance region (5% test) f(X)f(X) X  0 –1.96s.d. 00 22 false distribution true distribution

As in the previous figure, H 0 is false, but now the true value is  2, and  2 is closer to  0. As a consequence, the probability of X lying in the acceptance region for H 0 is much greater, 0.68 instead of 0.15, and so the power of the test, 0.32, is much lower. 12 TYPE II ERROR AND THE POWER OF A TEST rejection region acceptance region (5% test) f(X)f(X) X  0 –1.96s.d. 00 22 false distribution true distribution

The figure plots the power of a 5 percent significance test as a function of the distance separating the actual value of  and  0, measured in terms of the standard deviation of the distribution of X. 13 TYPE II ERROR AND THE POWER OF A TEST Distance between the actual value of  and  0 (standard deviations)

As is intuitively obvious, the greater is the discrepancy, the greater is the probability of H 0 :  =  0 being rejected. 14 TYPE II ERROR AND THE POWER OF A TEST Distance between the actual value of  and  0 (standard deviations)

We now return to the original value of  1 and again consider the case where H 0 :  =  0 is false and H 1 :  =  1 is true. What difference does it make if we perform a 1 percent test, instead of a 5 percent test? 15 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% rejection region acceptance region (1% test)

The figure shows the acceptance region for the 1 percent test. 16 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% rejection region acceptance region (1% test)

The probability of X lying in this region, given that it is actually distributed with mean  1, is shown as the yellow shaded area. It is The probability of making a Type II error is therefore TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% 0.34 rejection region acceptance region (1% test)

We have seen that the probability of making a Type II error with a 5 percent test, given by the blue shaded area, was This illustrates the trade-off between the risks of Type I and Type II error. 18 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% 0.15 rejection region acceptance region (1% test)

If we perform a 1 percent test instead of a 5 percent test, and H 0 is true, the risk of mistakenly rejecting it (and therefore committing a Type I error) is only 1 percent instead of 5 percent. 19 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% 0.15 rejection region acceptance region (1% test)

However, if H 0 happens to be false, the probability of not rejecting it (and therefore committing a Type II error) is larger. 20 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% 0.15 rejection region acceptance region (1% test)

How much larger? This is not fixed. It depends on the distance between  0 and  1, measured in terms of standard deviations. In this particular case, it has increased from 0.15 to 0.34, so it has about doubled. 21 TYPE II ERROR AND THE POWER OF A TEST f(X)f(X) X  0 –2.58s.d. 00 false distribution 11 true distribution 0.5% 0.15 rejection region acceptance region (1% test)

To generalize, we plot the power functions for the 5 percent and 1 percent tests. 22 TYPE II ERROR AND THE POWER OF A TEST 5% test1% test Distance between the actual value of  and  0 (standard deviations)

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.10 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics