1. Comparing Multiple Groups:
Scientific Practice
Comparing Multiple Groups:
Analysis of Variance
ANOVA (1-way)
@UWE_JT9
@dave_lush

2. Where We Are/Where We Are Going
We have looked at the 1-sample t-test and the 2-sample t-test
1-sample compares sample mean vs expected mean
paired t-test is a special case of ‘Before minus After’
2-sample compares two samples against each other
eg Males vs Females
But what if we want to compare lots of different situations?
repeatedly applying a t-test…
takes time
increase the chance of anomalous result
does not consider all data together

3. One-Way ANOVA
One-Way refers to the type of experiment being considered, not the number of conditions
eg we can compare effects of 10 different BP drugs
compare growth rates of 7 different bacteria
Like t-tests, we want to know if means differ
but ANOVA examines variability of underlying data
Assumptions…
data are normally distributed
samples are independent
variances of each group are similar (not that strict)
ANOVA partitions total variability into…
between-groups and within-groups

4. One-Way ANOVA Step 1: Formulate the Null Hypothesis
groups do not vary in terms of their means
Step 2: Generate the Test Statistic, F
F = MSbetween/MSwithin
where MS = s2
note if Null Hypo is true, then the random things determining MSwithin soley determine MSbetween
ie F = 1
but the more the total variability can be assigned to between-group and the less to within-groups…
…the bigger F becomes
ie a bigger F implies differences between groups that could be significant

5. One-Way ANOVA Step 3: Work out probability
the F statistic has a distribution, the F distribution
note one tail
origin of 0
peak around 1
no negatives
the usual logic applies…
a ‘critical value’ for F can be looked up in a table
if our F is greater, then p < 0.05 (or other level)
critical F depends on DoF of between-groups and within-groups
between-groups DoF = #groups – 1
within-groups DoF = total readings – #groups

6. One-Way ANOVA

7. One-Way ANOVA

8. One-Way ANOVA Step 4: Interpret the probability ‘Issues’ with ANOVA
if p < 0.05, then the Null Hypo is rejected…
…in favour of the Alt Hypo, that there is at least one significantly different mean in the groups we are investigating
‘Issues’ with ANOVA
My ANOVA tells me something is going on, but it doesn’t tell me what!
how do I find out which group(s) is/are sig different?
Eg which of my 10 BP drugs is having an effect?
Terminology
eg MS

9. One-Way ANOVA: Terminology
Between groups aka…
factor (Minitab), treatment
Within groups aka…
error (Minitab), residual
Example using Minitab…
4 cages of rats kept on a control diet
are there differences in weights between cages?

10. One-Way ANOVA: Terminology
Minitab output (part)…
Source DF SS MS F P
Factor
Error
Total
Factor (between), Error (within)
DF = degrees of freedom
SS = sum of the squares
each weight is a distance (+/-) from the mean
and each group mean is (+/-) from overall mean
square each ‘distance’ to remove the sign
sum each together to get indication of variability

11. One-Way ANOVA: Sum of Squares

12. One-Way ANOVA: Terminology
Minitab output (part)…
Source DF SS MS F P
Factor
Error
Total
MS = Mean Square
simple…
MS = SS / DF
less simple…
MS is the variance of each factor
variance = sum of squared differences / N
where N is DoF, not number of observations
F = MSFactor / MSError = 9078/3997 = 2.27

13. One-Way ANOVA: What changed?
Say, unknown to us, a cage was unlocked and rats kept sneaking out to eat extra food…
Source DF SS MS F P
Factor
Error
Total
Critical F (3, 16 DoF; 0.05 level) = 3.24
note Minitab calculates actual p (<0.0005% )
v low chance variation between groups by chance
reject Null Hypo; but which one(s) are different?

14. One-Way ANOVA: What changed?
When ANOVA flags an effect, working out what has changed is called post hoc testing
comparing all groups with each other – Tukey test
comparing groups to a control – Dunnett test
Tukey in Minitab
for a ‘family’ of comparisons, Minitab ‘ups’ the 95% confidence level for individual comparisons so the ‘family’ level is 95%
Grouping Information Using Tukey Method
N Mean Grouping
Cage A
Cage B
Cage B
Cage B
Means that do not share a letter are significantly different.

15. One-Way ANOVA: What changed?
Dunnett test
creates CIs for differences between mean of each ‘expt’ group and mean of control group
if a CI contains 0, then no significant difference
as before, multiple comparisons means that individual confidence levels increased so ‘family’ confidence level appropriate (95%)
It’s stupid, but say our unlocked cage was the control
eg looking at effect of locking cages on weight of rats

16. One-Way ANOVA: What changed?
Level N Mean Grouping
Cage 2 (control) A
Cage
Cage
Cage
Means not labeled A are sig different to control level mean
Family error rate = 0.05
Individual error rate =
Intervals for treatment mean minus control mean
Level Lower Center Upper
Cage
Cage
Cage
Tells us the ‘expt’ cages are different to the control cage
CI does not include 0 difference

17. One-Way ANOVA: Is that it?
2-sample t-test and 1-way ANOVA
differ as 1-way ANOVA can deal with >2 groups
but both rely on sets of data being…
normally distributed
independent of each other
ie unpaired data
Is there an equivalent ANOVA to ‘paired t-test’?
eg same group of subjects given a series of drugs
two solutions
the repeated measures ANOVA
but not trivial to do! Leave until later?
randomised block design

18. Summary
1-way ANOVA allows many groups to be compared in terms of one factor (treatment)
Partitions total variability between-groups and within-groups
Test Statistic, F = MSbetween/MSwithin
big F implies significant differences
Post hoc testing needed if ANOVA significant
Tukey test for testing all groups against each other
Dunnett test for comparing ‘expt’ groups v a control
1-way ANOVA an ‘extension’ of unpaired t-test
Repeated Measured/Randomised Block ANOVA for paired data