# Power.

## Presentation on theme: "Power."— Presentation transcript:

Power

The Four Components to a Statistical Conclusion
Sample size The number of units (e.g., people) accessible to study The salience of the program relative to the noise The odds the observed result is due to chance The odds you’ll observe a treatment effect when it occurs Effect size Alpha level Power

The Four Components to a Statistical Conclusion
Sample size Amount of information Salience of program Willingness to risk Ability to see effect that’s there Effect size Alpha level Power

The Effect Size Is a ratio of...

The Effect Size Is a ratio of... Signal Noise

Difference between groups Standard error of the difference
The Effect Size Is a ratio of... Signal Noise Difference between groups Standard error of the difference

Given Values for Any Three, You Can Compute the Fourth.
n = f(effect size, a, power) effect size = f(n, a, power) a = f(n, effect size, power) power = f(n, effect size, a)

The Decision Matrix In reality What we conclude

The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong

The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong

# of times out of 100 when there is no effect, we’ll say there is none
The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Accept null 1- Reject alternative THE CONFIDENCE LEVEL We say... The odds of saying there is no effect or gain when in fact there is none There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none

The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct

# of times out of 100 when there is no effect, we’ll say there is one
The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Reject null Accept alternative TYPE I ERROR We say... The odds of saying there is an effect or gain when in fact there is none There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one

The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct

The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong

# of times out of 100 when there is an effect, we’ll say there is none
The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Accept null Reject alternative TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is an effect, we’ll say there is none

The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct

# of times out of 100 when there is an effect, we’ll say there is one
The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Reject null 1- Accept alternative POWER We say... The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is an effect, we’ll say there is one

The Decision Matrix In reality What we conclude Null true Null false
Alternative false Alternative true In reality... In reality... What we conclude There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one

The Decision Matrix In reality What we conclude THE CONFIDENCE LEVEL
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... There is no real program effect There is no difference, gain Our theory is wrong Reject null 1- Accept alternative TYPE I ERROR POWER We say... There is a real program effect There is a difference, gain Our theory is correct

The Decision Matrix In reality What we conclude THE CONFIDENCE LEVEL
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... CORRECT There is no real program effect There is no difference, gain Our theory is wrong Reject null 1- Accept alternative TYPE I ERROR POWER We say... CORRECT There is a real program effect There is a difference, gain Our theory is correct

The Decision Matrix In reality What we conclude
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one If you try to increase power, you increase the chance of winding up in the bottom row and of Type I error. There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one

The Decision Matrix In reality
What we conclude Null true Null false Alternative false Alternative true If you try to decrease Type I errors, you increase the chance of winding up in the top row and of Type II error. In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one