1. Lecture 7: Single classification analysis of variance (ANOVA)
When to use ANOVA
ANOVA models and partitioning sums of squares
ANOVA: hypothesis testing
ANOVA: assumptions
A non-parametric alternative: Kruskal-Wallis ANOVA
Power analysis in single classification ANOVA
Bio 4118 Applied Biostatistics
2001

2. When to use ANOVA
Tests for effect of “discrete” independent variables.
Each independent variable is called a factor, and each factor may have two or more levels or treatments (e.g. crop yields with nitrogen (N) or nitrogen and phosphorous (N + P) added).
ANOVA tests whether all group means are the same.
Use when number of levels (groups) is greater than two.
Control
Experimental (N)
Experimental (N+P)
Frequency
Yield mC mN
mN+P
Bio 4118 Applied Biostatistics
2001

3. Why not use multiple 2-sample tests?
Yield mC mN
mN+P
Control
Experimental (N)
Experimental (N+P)
mc:mN
mN:mN+P
mC: mN+P
Frequency
For k comparisons, the probability of accepting a true H0 for all k is (1 - a)k.
For 4 means, (1 - a)k = (0.95)6 = .735.
So a (for all comparisons) =
So, when comparing the means of four samples from the same population, we would expect to detect significant differences among at least one pair 27% of the time.
Bio 4118 Applied Biostatistics
2001

4. What ANOVA does/doesn’t do
Yield
Frequency mC mN
mN+P
Tells us whether all group means are equal (at a specified a level)...
...but if we reject H0, the ANOVA does not tell us which pairs of means are different from one another.
Control
Experimental (N)
Experimental (N+ P)
Bio 4118 Applied Biostatistics
2001

5. Model I ANOVA: effects of temperature on trout growth
3 treatments determined (set) by investigator.
Dependent variable is growth rate (l), factor (X) is temperature.
Since X is controlled, we can estimate the effect of a unit increase in X (temperature) on l (the effect size)...
…and can predict l at other temperatures.
Water temperature (°C) 16 20 24 28
0.00
0.04
0.08
0.12
0.16
0.20
Growth rate l (cm/day)
Bio 4118 Applied Biostatistics
2001

6. Model II ANOVA: geographical variation in body size of black bears
3 locations (groups) sampled from set of possible locations.
Dependent variable is body size, factor (X) is location.
Even if locations differ, we have no idea what factors are controlling this variability...
…so we cannot predict body size at other locations.
Body size (kg)
120
160
200
240
280
Riding
Mountain
Kluane
Algonquin
Bio 4118 Applied Biostatistics
2001

7. Model differences
In Model I, the putative causal factor(s) can be manipulated by the experimenter, whereas in Model II they cannot.
In Model I, we can estimate the magnitude of treatment effects and make predictions, whereas in Model II we can do neither.
In one-way (but NOT multi-way!) ANOVA, calculations are identical for both models.
Bio 4118 Applied Biostatistics
2001

8. How is it done? And why call it ANOVA?
In ANOVA, the total variance in the dependent variable is partitioned into two components:
among-groups: variance of means of different groups (treatments)
within-groups (error): variance of individual observations within groups around the mean of the group
Bio 4118 Applied Biostatistics
2001

9. Statistical analysis as model building
All statistical analyses begin with a mathematical model that supposedly “describes” the data, e.g., regression, ANOVA.
“Model fitting” is then the process by which model parameters are estimated. X
Linear regression
ANOVA
e42 m2 a2 Y m
Group 1
Group 2
Group 3
Bio 4118 Applied Biostatistics
2001

10. Least squares estimation (LSE)
SSR
An ordinary least squares (OLS) estimate of a model parameter Q is that which minimizes the sum of squared differences between observed and predicted values:
OLS Q
Predicted values are derived from some model whose parameters we wish to estimate
Bio 4118 Applied Biostatistics
2001

11. Example: LSE of model parameters in simple linear regression
Data consists of a set of n paired observations (x1, y1), …, (xnyn)
The “model” for the I th observation is:
What is the LSE of the model parameters a and b? Y ei X
Residual:
Bio 4118 Applied Biostatistics
2001

12. The general ANOVA modelY m2 a2 m
e42
The general model is:
ANOVA algorithms fit the above model (by ordinary least squares) to estimate the ai’s.
H0: all ai’s = 0
m =m1 = m2 = m3 Y m
Group 1
Group 2
Group 3
a1 =a2 =a3 = 0
Group
Bio 4118 Applied Biostatistics
2001

13. Partitioning the total sums of squaresY m m3 m1
Total SS
Model (Groups) SS
Error SS
Group 1
Group 2
Group 3
Bio 4118 Applied Biostatistics
2001

14. The ANOVA table Source of Variation Sum of Squares Degrees of
freedom (df)
Mean
Square F
Total
n - 1
SS/df
MSgroups
MSerror
Groups
k - 1
SS/df
Error
n - k
SS/df
Bio 4118 Applied Biostatistics
2001

15. Variance components and group means
Yield
Frequency mC mN
mN+P
MSgroups measures average squared difference among group means.
MSerror is a measure of precision.
Control
Experimental (N)
Experimental (N+ P)
Bio 4118 Applied Biostatistics
2001

16. ANOVA: the null hypothesis
Yield
Frequency mC mN
mN+P
H0: all group means are the same, or...
H0: all group effects (ai) are zero, or...
H0: F = MSgroups/ MSerror = 1
For k groups and N observations, compare with F distribution at desired a level with k - 1 and N - k degrees of freedom.
Control
Experimental (N)
Experimental (N+ P)
Bio 4118 Applied Biostatistics
2001

17. Lab example: temporal variation in size of sturgeon (Model II ANOVA)
Prediction: dam construction resulted in loss of large sturgeon
Test: compare sturgeon size before and after dam construction
H0: mean size is the same for all years (?)
1954
1958
1965
1966
YEAR
35.0
38.8
42.6
46.4
50.2
54.0
FKLNGTH
Dam construction
Bio 4118 Applied Biostatistics
2001

18. Temporal variation in size of sturgeon (ANOVA results)
Conclusion: reject H0
Bio 4118 Applied Biostatistics
2001

19. ANOVA assumptions Residuals are independent of one another.
Residuals are normally distributed.
Variance of residuals within groups is the same for all groups (homoscedasticity).
Note: all assumptions apply to the residuals, not the raw data.
Since all assumptions apply to the residuals, not the raw data…
…all tests of assumptions are done after the analysis is completed (and residuals have been generated).
Bio 4118 Applied Biostatistics
2001

20. The general ANOVA modelY m2 a2 m
e42
The general model is:
… so the predicted value of all observations in the ith group is
The difference between the predicted value for an observation and the observed value is its residual.
Group 1
Group 2
Group 3
m =m1 = m2 = m3 Y m
a1 =a2 =a3 = 0
Group
Bio 4118 Applied Biostatistics
2001

21. Residual independence
Lack of residual independence usually arises because observations are correlated in time or space
E.g. measures of phosphorous concentrations upstream and downstream of a point source
Upstream site
Downstream site
Bio 4118 Applied Biostatistics
2001

22. What are degrees of freedom?
The degrees of freedom for any estimated quantity (e.g. the mean, variance, etc.) is the total sample minus one…
… because if you know the quantity (e.g. the mean) and the values of all n-1 observations, you know the value of the nth observation.
The degrees of freedom for any statistical model is the total sample size minus the number of parameters
Bio 4118 Applied Biostatistics
2001

23. Why do degrees of freedom matter?
Larger df
The distribution of the test statistic is different for different degrees of freedom.
Therefore, depending on the degrees of freedom, the same difference in sample means will give different p values.
Smaller df
Probability -3 -2 -1 1 2 3 t
Bio 4118 Applied Biostatistics
2001

24. Why does observations need to be independent?
If observations are not independent, then the true degrees of freedom is less (sometimes much less) than the calculated degrees of freedom …
… the distribution used to calculate p will be wrong …
… and p will be smaller than it ought to be.
calcuated df
true df
Probability
Calculated t -3 -2 -1 1 2 3 t
Bio 4118 Applied Biostatistics
2001

25. Checking independence of observations (residuals)
Does the experimental design suggest that sampling units may not be independent (e.g. spatiotemporal correlation?)
Calculate intraclass R correlation
Do autocorrelation plots to check for serial autocorrelation correlation.
Bio 4118 Applied Biostatistics
2001

26. Testing independence of residuals: ACF plots
Sort residuals by estimate and run ACF
Do any correlations fall outside the 95% confidence intervals?
Bio 4118 Applied Biostatistics
2001

27. Testing normality of residuals
-20
-10 10 20 30
RESIDUAL -3 -2 -1 1 2 3
Expected Value for Normal Distribution
Outliers?
Generate normal probability plot of residuals and check for linearity.
If warranted, run Lilliefors test, keeping in mind the power issue!
Bio 4118 Applied Biostatistics
2001

28. Testing homoscedasticity I: plotting residuals against estimates
Does “spread” of residuals appear the same for each group? 42 43 44 45 46 47 48 49
ESTIMATE -3 -2 -1 1 2 3 4 5 6
STUDENT
Outlier?
Bio 4118 Applied Biostatistics
2001

29. Testing homoscedasticity II: Levene’s test
Least Squares Means
1954
1958
1965
1966
YEAR 2 4 6
ABSRES
Calculate mean absolute residual for each group.
Does this value vary among groups?
Bio 4118 Applied Biostatistics
2001

30. Testing homoscedasticity II: Levene’s test (cont’d)
Bio 4118 Applied Biostatistics
2001

31. Effects of violations of assumptions4 1 2 3 5
0.0
0.2
0.4
0.6
0.8
1.0
Probability
Calculation of p assumes p(F) = p(F*)
… but as residuals conform less to required assumptions, the deviation between the two increases.
Therefore, calculated p values are incorrect.
F, low conformity
F, high conformity
True F (F*)
Bio 4118 Applied Biostatistics
2001

32. Robustness of ANOVA with respect to violation of assumptions
Bio 4118 Applied Biostatistics
2001

33. Residual analysis: questions
Which assumptions are not met, and how robust is ANOVA to their violation?
What is the sample size?
Is the violation of assumptions due to a couple of outliers?
How close is p to a?
Eliminate outliers and rerun analysis.
Transform data.
Try a non-parametric alternative (generally recommended if sample sizes are small, i.e. < 10 per group) such as Kruskal-Wallis ANOVA.
Bio 4118 Applied Biostatistics
2001

34. A non-parametric alternative: Kruskal-Wallis ANOVA
Calculate rank sum (Rg) for each group.
H0: RC = R1 = R2
Calculate K-W H statistic:
… which is distributed as c2 with k-1 df if N for each group is not too small, otherwise use critical values for H.
A non-parametric alternative: Kruskal-Wallis ANOVA
Bio 4118 Applied Biostatistics
2001

35. Power and sample size in single-classification ANOVA
If H0 is true, then variance ratio MSgroups/MSerror follows central F distribution.
But, if H0 is false, then MSgroups/MSerror follows non-central F, defined by n1, n2 and non-centrality parameter f.
So, power calculations depend on non-central F.
Frequency
Yield mC mN
mN+P
Control
Experimental (N)
Experimental (N+P)
Bio 4118 Applied Biostatistics
2001

36. Power and sample size in single-classification ANOVA
Power of a test involving k groups with n replicates per group at specified a when (1) group means are known; (2) minimal detectable distance is specified.
estimation of minimum sample size and minimal detectable difference among groups
Frequency
Yield mC mN
mN+P
Control
Experimental (N)
Experimental (N+P)
Bio 4118 Applied Biostatistics
2001

37. Power and sample size in single-classification ANOVA
ANOVA with k groups with n replicates per group at specified a .
If we have an estimate of the within-group variability s2 (MSerror), we can calculate f:
Frequency
Yield mC mN
mN+P
Control
Experimental (N)
Experimental (N+P)
Bio 4118 Applied Biostatistics
2001

38. Calculating power given f
1-b
Decreasing n2
n1 = 2
a = .05 2 3 4 5
a = .01 1
1.5
2.5
Given n1 ,n2, a and f, we can read 1-b from suitable tables or curves (e.g. Zar (1996), Appendix Figure B.1).
f(a = .01)
f(a = .05)
Bio 4118 Applied Biostatistics
2001

39. Model I ANOVA: minimal detectable difference
Frequency mC mN
mN+P
Control
Experimental (N)
Experimental (N+P) d
Model I ANOVA: minimal detectable difference
Suppose we want to detect a difference between the two most different sample means of at least d.
To test at the a significance level with 1 - b power, we can calculate the minimal sample size nmin required to detect d, given a sample group variance s2 by solving iteratively.
Bio 4118 Applied Biostatistics
2001

40. Model I ANOVA: power of the test
Control
Experimental (N)
Experimental (N+P)
Frequency
Yield mC mN
mN+P
Model I ANOVA: power of the test
If H0 is accepted, it is good practice to calculate power!
Knowing MSgroups , s2 (= MSerror), and k, we can calculate f.
Bio 4118 Applied Biostatistics
2001

41. Power of the test: an example
Effect of temperature on insect development time
4 eggs each at two temperatures, 5 at the third (k = 3, n1 = n2 = 4, n3 = 5)
So, there is a 67% chance of committing a Type II error.
Bio 4118 Applied Biostatistics
2001

42. Factors determining power in single classification ANOVA
Power increases with increasing f.
Therefore, power increases with (1) increasing sample size n; (2) increasing differences among group means (MSgroups); (3) decreasing number of groups; (4) decreasing within-group variability s2 (MSerror).
Bio 4118 Applied Biostatistics
2001

43. Power in single-classification Model II ANOVA
120
160
200
240
280
Riding
Mountain
Kluane
Algonquin
In this case, we can calculate 1- b from central F:
Knowing n1, n2, a and MSgroups, we can estimate 1 - b.
Body size (kg)
Bio 4118 Applied Biostatistics
2001

44. Power in non-parametric single-classification ANOVA
If assumptions of parametric ANOVA are met, then non-parametric ANOVA is 3/p = 95% as powerful.
If non-parametric ANOVA is used, calculate power for parametric ANOVA to get a rough estimate of power of non-parametric test.
Bio 4118 Applied Biostatistics
2001