1. Multiple comparisons - multiple pairwise tests - orthogonal contrasts
- independent tests
- labelling conventions

2. Card example number 1

3. Multiple tests Problem:
Because we examine the same data in multiple comparisons, the result of the first comparison affects our expectation of the next comparison.

4. Multiple tests ANOVA shows at least one different, but which one(s)?
T-tests of all pairwise combinations
significant
significant
Not significant

5. Multiple tests T-test: <5% chance that this difference was a fluke…
affects likelihood of finding a difference in this pair!

6. Multiple tests Solution:
Make alpha your overall “experiment-wise” error rate
T-test: <5% chance that this difference was a fluke…
affects likelihood (alpha) of finding a difference in this pair!

7. Multiple tests Solution:
Make alpha your overall “experiment-wise” error rate
e.g. simple Bonferroni:
Divide alpha by number of tests
Alpha / 3 =
Alpha / 3 =
0.0167
Alpha / 3 =

8. Card example 2

9. Orthogonal contrasts Orthogonal = perpendicular = independent
Contrast = comparison
Example. We compare the growth of three types of plants: Legumes, graminoids, and asters.
These 2 contrasts are orthogonal:
1. Legumes vs. non-legumes (graminoids, asters)
2. Graminoids vs. asters

10. Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
Legumes Graminoids Asters

11. Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”.
Legumes Graminoids Asters
/ /2

12. Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”.
3. Repeat for all other contrasts.
Legumes Graminoids Asters
/ /2

13. Trick for determining if contrasts are orthogonal:
4. Multiply each column, then sum these products.
Legumes Graminoids Asters
/ /2
/ /2
Sum of products = 0

14. Trick for determining if contrasts are orthogonal:
4. Multiply each column, then sum these products.
5. If this sum = 0 then the contrasts were orthogonal!
Legumes Graminoids Asters
/ /2
/ /2
Sum of products = 0

15. What about these contrasts?
1. Monocots (graminoids) vs. dicots (legumes and asters).
2. Legumes vs. non-legumes

16. } } How do you program contrasts in JMP (etc.)? Treatment SS

17. How do you program contrasts in JMP (etc.)?
Legumes
vs. non-legumes
Normal treatments
“There was a significant treatment effect (F…). About 53% of the variation between treatments was due to differences between legumes and non-legumes (F …).”
Legume 1 1
Graminoid 2 2
Aster 3 2 SS
Df 2 1 MS

18. Even different statistical tests may not be independent !
Example. You use regression to examine if plant size determines seed number. You then divide the data into 2 treatments, large plants and small plants, and analyze seed number with a t-test.
Problem?

19. Multiple tests b Convention:
Treatments with a common letter are not significantly different a
a,b
significant
Not significant
Not significant