EC220 - Introduction to econometrics (chapter 2) Christopher Dougherty EC220 - Introduction to econometrics (chapter 2) Slideshow: type 1 error and type 2 error   Original citation: Dougherty, C. (2012) EC220 - Introduction to econometrics (chapter 2). [Teaching Resource] © 2012 The Author This version available at: http://learningresources.lse.ac.uk/128/ Available in LSE Learning Resources Online: May 2012 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/   http://learningresources.lse.ac.uk/

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 5% level 2.5% 2.5% b2 b2-1.96sd b2-sd b2 b2+sd b2+1.96sd In the previous sequence a Type I error was defined to be the rejection of a null hypothesis when it happens to be true. 1

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 5% level 2.5% 2.5% b2 b2-1.96sd b2-sd b2 b2+sd b2+1.96sd In hypothesis testing there is also a possibility of failing to reject the null hypothesis when it is in fact false. This is known as a Type II error. 2

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 5% level 2.5% 2.5% b2 b2-1.96sd b2-sd b2 b2+sd b2+1.96sd This sequence will demonstrate that there is a trade-off between the risk of making a Type I error and the risk of making a Type II error. 3

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 5% level 2.5% 2.5% b2 b2-1.96sd b2-sd b2 b2+sd b2+1.96sd The diagram show the acceptance region and the rejection regions for a 5% significance test. The risk of making a Type I error, if the null hypothesis happens to be true, is 5%. 4

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level 5% level 0.5% 0.5% b2 b2-2.58sd b2-sd b2 b2+sd b2+2.58sd Using a 1% significance test, instead of a 5% test, reduces the risk of making a Type I error to 1%, if the null hypothesis is true. 5

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level 5% level 0.5% 0.5% b2 b2-2.58sd b2-sd b2 b2+sd b2+2.58sd We will consider the implications of the choice of significance test for the case where the null hypothesis happens to be false. 6

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level 5% level 0.5% 0.5% b2 b2-2.58sd b2-sd b2 b2+sd b2+2.58sd The diagram above explains how the test decisions are made, but it does not give the actual distribution of b2. (For that reason the curve has been drawn with a dashed line.) 7

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 Suppose that H1: b2 = b21 is in fact true and the distribution of b2 is therefore governed by the right-hand curve. 8

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 If we obtain some data and run a regression, the estimate b2 might be as shown. In this case we would make the right decision and reject H0, no matter which test we used. 9

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 Here is another estimate. Again, we would make the right decision and reject the null hypothesis, no matter whether we use the 5% test or the 1% test. 10

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 In the case shown, we would make a Type II error and fail to reject the null hypothesis, using either significance level. 11

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 But in the case of this estimate, we would make the right decision if we used a 5% test but we would make a Type II error if we used a 1% test. 12

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 The probability of making a Type II error if we use a 1% test is given by the probability of b2 lying within the 1% acceptance region, the interval between the red vertical dotted lines. 13

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 Given that H1 is true, the probability of b2 lying in the acceptance region is that area under the distribution for H1 in the diagram - the pink shaded area in the diagram. 14

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 If instead we use a 5% significance test, the probability of making a Type II error if H1 is true is given by the area under the distribution for H1 in the acceptance region for the 5% test. 15

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 This is the gray shaded area in the diagram. In this particular case, using a 5% test instead of a 1% test would approximately halve the risk of making a Type II error. 16

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 The problem, of course, is that you never know whether H0 is true of false. If you did, why would you be performing a test? 17

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 If H0 happens to be true, using a 1% test instead of a 5% test greatly reduces the risk of making a Type I error (you cannot make a Type II error). 18

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b2 1% level actual distribution under 5% level 0.5% 0.5% b2 b2 b2-2sd 1 b2-sd 1 b2 1 b2+sd 1 b2+2sd 1 However, if H0 is false, using a 1% test instead of a 5% test increases the risk of making a Type II error (you cannot make a Type I error). 19

Copyright Christopher Dougherty 2011. These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 2.6 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre http://www.oup.com/uk/orc/bin/9780199567089/. Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx or the University of London International Programmes distance learning course 20 Elements of Econometrics www.londoninternational.ac.uk/lse. 11.07.25