Presentation is loading. Please wait.

Presentation is loading. Please wait.

Type I & Type II errors Brian Yuen 18 June 2013. Slide - 2 Definition of Type I error ( α ) Concluded a statistical significant effect size from the sample.

Similar presentations


Presentation on theme: "Type I & Type II errors Brian Yuen 18 June 2013. Slide - 2 Definition of Type I error ( α ) Concluded a statistical significant effect size from the sample."— Presentation transcript:

1 Type I & Type II errors Brian Yuen 18 June 2013

2 Slide - 2 Definition of Type I error ( α ) Concluded a statistical significant effect size from the sample when this does not exist in the whole population (H 0 is true) Reject the null hypothesis (H 0 ) when it is true False positive result Level of significance - P value Usually allowed for 5% - arbitrary but acceptable rate! –Repeatedly sample and test from the same population, you will expect to make 1 wrong conclusion (with statistically significant result) in 20 of these tests if the null hypothesis is in fact true 2

3 Slide - 3 Definition of Type II error ( β ) Concluded a non-statistical significant effect size from the sample when this exists in the whole population (H 1 true) Fail to reject the null hypothesis (H 0 ) when it is not true False negative result Power of study = 100% - False negative rate Maximum accepted false negative rate at 20%  Minimum power of study at 80% 3

4 Slide - 4 What does a significant result mean? Remember we are trying to find enough evidence to reject the null hypothesis i.e. to show that the observed effect size has exceeded our pre-set threshold of expected events, hence a statistical significant result Observed a significant result, are we –lucky to find out null hypothesis is indeed incorrect? A genuine finding –unlucky to observed an unlikely result/event? Type I error 4 Chance findings fall into this region Findings fall in these 2 regions are too extreme to say these by chance

5 Slide - 5 Can we eliminate these errors completely? Type I error –Not really, there is always a pre-set (5%) chance for a Type I error –You could minimise this by reducing the level of significance from 5% to 1%, but you would expect a larger sample size Type II error –Not really, there is always a pre-set (20%) chance for a Type II error –You could minimise this by reducing the error rate from 20% to 10%, but you would compromise this with a larger sample size 5

6 Slide - 6 Can we ever work out if Type I error exist in published results? Yes, if the result is statistically significant, then there is a possibility of a Type I error Since we allowed for 5% chance to have this error, there is always a possibility of getting such error and there is no way we can get rid of this doubt We could work out if there is any hint of an unusual/unlikely result by comparing results from other similar studies Testing multiple outcomes issue P<0.05 means it is unlikely that the observed effect size is by chance (less than 5%) P<0.01 means it is unlikely that the observed effect size is by chance (less than 1%) 6

7 Slide - 7 Can we ever work out if Type II error exist in published results? Yes, if the result is not statistically significant, then there is a possibility of a Type II error The key point is to identify if the author had performed a sample size calculation –with a pre-defined power before the study starts –on the one primary outcome they made conclusive statement –whether they had recruited the desired number at the analysis stage –whether they had pre-defined a clinically worthwhile effect size to be detected and was observed Then the chance of having a Type II error would be the pre-set value, i.e. 20% 7

8 Slide No True Difference True Difference No Observed Difference Well Designed Trial (100-  ) Type II Error (  ) Observed Difference Type I Error (  ) Well Powered Trial (100-  ) Type I error ( α ) & Type II error ( β )

9 Slide - 9 Mind twisting quizzes! In a published paper, a statistically significant result of a fully powered 2-group study was reported on a single primary outcome. What can you comment on this result regarding Type I/II error? In a published paper, a non statistically significant result of a fully powered 2-group study was reported on a single primary outcome. What can you comment on this result regarding Type I/II error? You have reviewed 20 studies of a similar kind regarding the same outcome measure. You have found 5 studies with statistically significant results, while the others were not. What is your view on these results? 9

10 Slide - 10 Reference readers/publications/statistics-square-one/5- differences-between-means-type-i-anhttp://www.bmj.com/about-bmj/resources- readers/publications/statistics-square-one/5- differences-between-means-type-i-an 10


Download ppt "Type I & Type II errors Brian Yuen 18 June 2013. Slide - 2 Definition of Type I error ( α ) Concluded a statistical significant effect size from the sample."

Similar presentations


Ads by Google