Download presentation

Presentation is loading. Please wait.

Published byRay Bardy Modified over 2 years ago

1
Power and Effect Size Cal State Northridge 320 Andrew Ainsworth PhD

2
2 Major Points Review What is power? What controls power? Effect size Power for one sample t Power for related-samples t Power for two sample t Psy 320 - Cal State Northridge

3
3 Important Concepts Concepts critical to hypothesis testing –Decision –Type I error –Type II error –Critical values –One- and two-tailed tests Psy 320 - Cal State Northridge

4
4 Decisions When we test a hypothesis we draw a conclusion; either correct or incorrect. –Type I error Reject the null hypothesis when it is actually correct. –Type II error Retain the null hypothesis when it is actually false. Psy 320 - Cal State Northridge

5
5 Type I Errors Null Hypothesis really is true We conclude the null is false. This is a Type I error –Probability set at alpha ( ) usually at.05 –Therefore, probability of Type I error =.05 Psy 320 - Cal State Northridge

6
6 Type II Errors The Alternative Hypothesis is true We conclude that the null is true This is also an error (Type II) –Probability denoted beta ( ) We can’t set beta easily. We’ll talk about this issue later. Power = (1 - ) = probability of correctly rejecting false null hypothesis. Psy 320 - Cal State Northridge

7
7 Confusion Matrix Psy 320 - Cal State Northridge

8
8 Critical Values These represent the point at which we decide to reject null hypothesis. e.g. We might decide to reject null when (p|null) <.05. –In the null distribution there is some value with p =.05 –We reject when we exceed that value. –That value is called the critical value. Psy 320 - Cal State Northridge

9
9 One- and Two-Tailed Tests Two-tailed test rejects null when obtained value too extreme in either direction –Decide on this before collecting data. One-tailed test rejects null if obtained value is too low (or too high) –We only set aside one direction for rejection. Psy 320 - Cal State Northridge

10
10 One- & Two-Tailed Example One-tailed test –Reject null if IQPLUS group shows an increase in IQ Probably wouldn’t expect a reduction and therefore no point guarding against it. Two-tailed test –Reject null if IQPLUS group has a mean that is substantially higher or lower. Psy 320 - Cal State Northridge

11
11 What Is Power? Probability of rejecting a false H 0 Probability that you’ll find difference that’s really there 1 - , where = probability of Type II error Psy 320 - Cal State Northridge

12
12 What Controls Power? The significance level ( ) True difference between null and alternative hypotheses 1 - 2 Sample size Population variance The particular test being used Psy 320 - Cal State Northridge

13
13 Distributions Under 1 and 0 Psy 320 - Cal State Northridge

14
14 Effect Size The degree to which the null is false –Depends on distance between and –Also depends on standard error (of mean or of difference between means) Psy 320 - Cal State Northridge

15
15 What happened to n? It doesn’t relate to how different the two population means are. It controls power, but not effect size. We will add it in later. Psy 320 - Cal State Northridge

16
16 Estimating Effect Size Judge your effect size by: –Past research –What you consider important –Cohen’s conventions Psy 320 - Cal State Northridge

17
17 Combining Effect Size and n We put them together and then evaluate power from the result. General formula for Delta –where f (n) is some function of n that will depend on the type of design Psy 320 - Cal State Northridge

18
18 Power for One-Sample or Related samples t First calculate delta with: –where n = size of sample, and and as above Look power up in table using and significance level ( ) Psy 320 - Cal State Northridge

19
19 Power for Single Sample IQPLUS Study One sample z and t –Compared IQPLUS group with population mean = 100, sigma = 10 –Used 25 subjects –We got a sample mean of 106 and s = 7.78 Psy 320 - Cal State Northridge

20
20 IQPLUS Assuming we don’t know sigma – = 0.77 – n = 25 – –We are testing at =.05 –Use Appendix D.5 Psy 320 - Cal State Northridge

21
21 Appendix D.5 This table is severely abbreviated. Power for = 3.85, =.05 Psy 320 - Cal State Northridge

22
22 Conclusions If we can trust our estimates in the IQPLUS study then if this study were run repeatedly, 97% of the time the result would be significant. Psy 320 - Cal State Northridge

23
23 How Many Subjects Do I Really Need (Single/Related Sample(s))? Run calculations backward –Start with anticipated effect size ( ) –Determine required for power =.80. Why.80? –Calculate n What if we wanted to rerun the IQPLUS study, and wanted power =.80? Psy 320 - Cal State Northridge

24
24 Calculating n We estimated =.77 Complete Appendix D.5 shows we need = 2.80 Calculations on next slide Psy 320 - Cal State Northridge

25
25 IQPLUS n Psy 320 - Cal State Northridge

26
26 Power for Two Independent Groups What changes from preceding? –Effect size deals with two sample means –Take into account both values of n Effect size Psy 320 - Cal State Northridge

27
27 Estimating d We could calculate d directly if we knew populations. We could estimate from previous data. Here we will calculate using Violent Video Games example Psy 320 - Cal State Northridge

28
28 Example: Violent Videos Games Two independent randomly selected/assigned groups –GTA (violent: 8 subjects) VS. NBA 2K7 (non- violent: 10 subjects) –We want to compare mean number of aggressive behaviors following game play –GTA had a mean of 10.25 and s = 1.669 –NBA 2K7 had a mean of 8.4 and s = 1.647 –s 2 pooled = 2.745, so s pooled = 1.657 Psy 320 - Cal State Northridge

29
29 Two Independent Groups Then calculate from effect size Note: The above formula assumes that the 2 groups have equal n Psy 320 - Cal State Northridge

30
30 Two Independent Groups Our data do not have equal n, but let’s pretend they do for a moment (both 10) For our data Psy 320 - Cal State Northridge

31
31 Appendix D.5 This table is severely abbreviated. Power for = 2.5, =.05 Estimate =.71 Psy 320 - Cal State Northridge

32
32 Conclusions If we had equal n and we can trust our estimates in the violent video game study then if this study were run repeatedly, 71% of the time the result would be significant. Psy 320 - Cal State Northridge

33
33 Unequal Sample Sizes With unequal samples use harmonic mean of sample sizes Where k is the number of groups (i.e. 2), n i is each group size Standard arithmetic average will work well if n ’s are close. Psy 320 - Cal State Northridge

34
34 Two Independent Groups Our data do not have equal n, so we need to find the harmonic mean For our data Psy 320 - Cal State Northridge

35
35 Two Independent Groups Our data do not have equal n, so… For our data Psy 320 - Cal State Northridge

36
36 Appendix D.5 This table is severely abbreviated. Psy 320 - Cal State Northridge Power for = 2.4, =.05 Estimate =.67

37
37 Conclusions If we we can trust our estimates in the violent video game study then if this study were run repeatedly, 67% of the time the result would be significant. Psy 320 - Cal State Northridge

38
38 How Many Subjects Do I Really Need (Independent Samples)? Run calculations backward –Start with anticipated effect size ( ) –Determine required for power =.80. Why.80? –Calculate n What if we wanted to rerun the violent video game study, and wanted power =.80? Psy 320 - Cal State Northridge

39
39 Calculating n We estimated = 1.116 Complete Appendix E.5 shows we need = 2.80 Calculations on next slide Psy 320 - Cal State Northridge

40
40 Violent Video Games n Psy 320 - Cal State Northridge

Similar presentations

OK

Major Points Formal Tests of Mean Differences Review of Concepts: Means, Standard Deviations, Standard Errors, Type I errors New Concepts: One and Two.

Major Points Formal Tests of Mean Differences Review of Concepts: Means, Standard Deviations, Standard Errors, Type I errors New Concepts: One and Two.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google