Presentation is loading. Please wait.

Presentation is loading. Please wait.

PowerPower. The Four Components to a Statistical Conclusion The number of units (e.g., people) accessible to study The salience of the program relative.

Similar presentations


Presentation on theme: "PowerPower. The Four Components to a Statistical Conclusion The number of units (e.g., people) accessible to study The salience of the program relative."— Presentation transcript:

1 PowerPower

2 The Four Components to a Statistical Conclusion The number of units (e.g., people) accessible to study The salience of the program relative to the noise The odds the observed result is due to chance The odds you’ll observe a treatment effect when it occurs Sample size Effect size Alpha level Power

3 The Four Components to a Statistical Conclusion Amount of information Salience of program Willingness to risk Ability to see effect that’s there Sample size Effect size Alpha level Power

4 The Effect Size Is a ratio of...

5 The Effect Size Is a ratio of... Signal Noise

6 The Effect Size Is a ratio of... Difference between groups Standard error of the difference Signal Noise

7 Given Values for Any Three, You Can Compute the Fourth. l n = f(effect size, a, power) l effect size = f(n, a, power) l a = f(n, effect size, power) l power = f(n, effect size, a)

8 The Decision Matrix In reality What we conclude

9 The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

10 The Decision Matrix In reality What we conclude Null true Alternative false In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

11 The Decision Matrix In reality What we conclude Null true Alternative false In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong 1-  THE CONFIDENCE LEVEL The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none

12 The Decision Matrix In reality What we conclude Null true Alternative false In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong

13 The Decision Matrix In reality What we conclude Null true Alternative false In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong  TYPE I ERROR The odds of saying there is an effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is one

14 The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

15 The Decision Matrix In reality What we conclude Null false Alternative true In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

16 The Decision Matrix In reality What we conclude Null false Alternative true In reality... Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct TYPE II ERROR  The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none

17 The Decision Matrix In reality What we conclude Null false Alternative true In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct

18 The Decision Matrix In reality What we conclude Null false Alternative true In reality... Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  POWER The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is one

19 The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  THE CONFIDENCE LEVEL TYPE II ERROR  The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none  1-  TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one

20 The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  THE CONFIDENCE LEVEL TYPE II ERROR   1-  TYPE I ERROR POWER

21 The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  THE CONFIDENCE LEVEL TYPE II ERROR   1-  TYPE I ERROR POWER CORRECT CORRECT

22 The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effect There is no difference, gain Our theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  THE CONFIDENCE LEVEL TYPE II ERROR  The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none  1-  TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one If you try to increase power, you increase the chance of winding up in the bottom row and of Type I error.

23 The Decision Matrix In reality What we conclude Null true Null false Alternative false Alternative true In reality... Accept null Reject alternative Reject null Accept alternative We say... There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong We say... There is a real program effect There is a difference, gain Our theory is correct There is no real program effectThere is no real program effect There is no difference, gainThere is no difference, gain Our theory is wrongOur theory is wrong There is a real program effectThere is a real program effect There is a difference, gainThere is a difference, gain Our theory is correctOur theory is correct 1-  THE CONFIDENCE LEVEL TYPE II ERROR  The odds of saying there is no effect or gain when in fact there is none # of times out of 100 when there is no effect, we’ll say there is none The odds of saying there is no effect or gain when in fact there is one # of times out of 100 when there is an effect, we’ll say there is none  1-  TYPE I ERROR POWER The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one If you try to decrease Type I errors, you increase the chance of winding up in the top row and of Type II error.


Download ppt "PowerPower. The Four Components to a Statistical Conclusion The number of units (e.g., people) accessible to study The salience of the program relative."

Similar presentations


Ads by Google