Download presentation

Presentation is loading. Please wait.

Published byChristopher Liggett Modified about 1 year ago

1
PowerPower

2
The Four Components to a Statistical Conclusion The number of units (e.g., people) accessible to study The salience of the program relative to the noise The odds the observed result is due to chance The odds you’ll observe a treatment effect when it occurs Sample size Effect size Alpha level Power

3
The Four Components to a Statistical Conclusion Amount of information Salience of program Willingness to risk Ability to see effect that’s there Sample size Effect size Alpha level Power

4
The Effect Size Is a ratio of...

5
The Effect Size Is a ratio of... Signal Noise

6
The Effect Size Is a ratio of... Difference between groups Standard error of the difference Signal Noise

7
Given Values for Any Three, You Can Compute the Fourth. l n = f(effect size, a, power) l effect size = f(n, a, power) l a = f(n, effect size, power) l power = f(n, effect size, a)

8
The Decision Matrix In reality What we conclude

9
The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

10
The Decision Matrix In reality What we conclude Null true Alternative false In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

11
The Decision Matrix In reality What we conclude Null true Alternative false In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong 1- THE CONFIDENCE LEVEL The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none

12
The Decision Matrix In reality What we conclude Null true Alternative false In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

13
The Decision Matrix In reality What we conclude Null true Alternative false In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong TYPE I ERROR The odds of saying there is an effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is one

14
The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

15
The Decision Matrix In reality What we conclude Null false Alternative true In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

16
The Decision Matrix In reality What we conclude Null false Alternative true In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct TYPE II ERROR The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none

17
The Decision Matrix In reality What we conclude Null false Alternative true In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

18
The Decision Matrix In reality What we conclude Null false Alternative true In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- POWER The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is one

19
The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- THE CONFIDENCE LEVEL TYPE II ERROR The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none 1- TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one

20
The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- THE CONFIDENCE LEVEL TYPE II ERROR 1- TYPE I ERROR POWER

21
The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- THE CONFIDENCE LEVEL TYPE II ERROR 1- TYPE I ERROR POWER CORRECT CORRECT

22
The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- THE CONFIDENCE LEVEL TYPE II ERROR The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none 1- TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one If you try to increase power, you increase the chance of winding up in the bottom row and of Type I error.

23
The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1- THE CONFIDENCE LEVEL TYPE II ERROR The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none 1- TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one If you try to decrease Type I errors, you increase the chance of winding up in the top row and of Type II error.

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google