1. Chapter 6
Making Sense of Statistical Significance: Decision Errors, Effect Size and Statistical Power
Part 1: Sept. 18, 2014

2. Decision Errors
Due to use of samples to estimate effects in populations
Type I error
Reject the null hypothesis when in fact it is true
Ex?
Equal to alpha (α) – probability of Type 1 error
α = .05, run a 5% risk of making a type 1 error
Type II error
Not rejecting the null hypothesis when in reality it is false (being too conservative)
Equal to beta (β) - probability of making a Type II error

3. Possible Decisions in Hypothesis Testing

4. Effect Size Figuring effect size (d)
We may reject the null and conclude there is a significant effect, but how large is it?
Effect size will estimate that; it is the amount that two populations (from our sample vs. comparison population) do not overlap
Figuring effect size (d)
Example:
Population SD
1 = experimental group
(use M from sample) 2 = population or
comparison group mean

6. Effect Size
Effect size conventions – make conclusions about how large/important effect is
Cohen’s (1977) guidelines -
Small effect size around d = .2 (or -.2)
Medium effect size around d = .5 (or -.5)
Large effect size around d = .8 (or -.8)
Example interpretation?
Effect size speaks to ‘practical significance’ – an indication of the importance of a statistically significant effect

7. Effect Size Interpretation
What is a desired effect size?
Interpretation:
For an experiment…
For a group comparison…
For a correlational study…
Used in meta-analysis:

8. Importance of Effect Size
3 applications –
1. Before starting the study, estimate effect size to project the N needed to get statistically signif results
2. Info about practical significance
3. Standardized comparisons across studies

9. Possible research outcomes
High Practical Signif (Effect size)
Low Practical Signif (Effect size)
High Statistical Signif
Low Statistical Signif

10. Example of journal article reporting effect size (d)
for gender comparisons on several variables:
d = effect size for this stat
p = signif level (if < .05, it’s statistically signif