1. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 1 Multiple-classification ANOVA What is it, and why use it? Multiple-classification ANOVA models Multiple-classification ANOVA designs The general multiple-classification ANOVA model Hypothesis testing and multiple comparisons in: factorial designs nested designs repeated measures designs blocked designs non-parametric ANOVA

2. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 2 What is multiple classification ANOVA? Extension of single classification ANOVA but with several (instead of one) independent factors E.g. 2-way ANOVA examines effects of nitrogen (factor 1) and phosphorous (factor 2) addition on corn yield. Unlike single-classification, in multi-way ANOVA’s there are several H 0 ’s which can be tested. Cell entries are means of N = 5 plots, in tonnes/acre.

3. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 3 The advantage of multi-way ANOVAs I: controlling  e We can run a single experiment which tests the effects of several different factors without increasing  e. E.g. in 1-way design, we would need 3 experiments to test for a nitrogen effect, one for each phosphorous level. Probability of accepting true H 0 in all experiments is (.95) 3 =.86, so probability of rejecting at least one true H 0 is  e =.14. Experiment 1 Experiment 2 Experiment 3

4. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 4 The advantage of multi-way ANOVAs II: testing interactions We can run a single experiment which allows us to test whether the effect of one factor depends on another. E.g. 1-way design would allow us to test for the effect of nitrogen given a certain level of phosphorous, but we cannot test whether this effect varies among different phosphorous levels. Experiment 1 Experiment 2 Experiment 3

5. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 5 Model I multiple-classification ANOVA: effects of temperature and acidity on trout growth Treatment levels for temperature and pH are set by investigator. Dependent variable is growth rate ( ); factors are temperature and pH. Since factors are controlled, we can estimate the effect of a unit increase in temperature and pH on  the  effect size  …and can predict  at other temperatures and pH’s. pH = 6.5 pH = 4.5 Water temperature (°C) 16202428 0.00 0.04 0.08 0.12 0.16 0.20 Growth rate (cm/day)

6. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 6 Model II multiple-classification ANOVA: size variation in Norops townsendi in the Cocos archipelago Dependent variable is snout- vent length; factors are Island (random) and elevation (random). Even if size differs between islands or elevations within islands, we have no idea what factors are controlling this variability... … so we cannot predict size at other islands or at other elevations on a given island. Sea level Mid elevation Peak Isla Manuelita Island Cocos Snout-vent length (mm) 35 40 45 50 55 60

7. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 7 Model III multiple-classification ANOVA: sexual and geographical variation in body size of black bears Dependent variable is body size; factors are sex (fixed) and location (random). Even if locations differ, we have no idea what factors are controlling this variability. So we cannot predict body size of each sex at other locations... …but we can (perhaps) predict the difference between the sexes. Body size (kg) 120 160 200 240 280 Riding Mountain Kluane Algonquin males females

8. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 8 Fixed versus random factors in ANOVA

9. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 9 Types of multiple classification ANOVA I: Factorial designs Levels of each factor are common to all levels of all other factors, i.e. the levels of each factor have consistent properties across other factors. e.g. effect of age (factor 1) and sex (factor 2) on size of lizards Observation

10. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 10 Types of ANOVA II: Nested designs Levels of at least one factor are random subset of the set of all possible levels, with levels not necessarily having consistent properties across other factors. e.g. effect of island (A, B) on size of lizards, with two random sites per island Observation

11. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 11 Types of ANOVA III: Balanced versus unbalanced designs Observation BalancedUnbalanced

12. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 12 Types of ANOVA IV: Replicated versus unreplicated designs Temperature measurement Unreplicated Replicated

13. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 13 Choosing the appropriate design/model Type of model: Model I, Model II or Model III Factorial versus nested Balanced versus unbalanced Replicated versus unreplicated Note: calculations differ among different designs and models, so make sure you choose the right one!

14. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 14 The general multiple- classification ANOVA model The general model for a two-factor ANOVA is ANOVA algorithms fit the above model (by least squares) to estimate  i,  j and (  ) ij. H 01 : all  i = 0 H 02 : all  j = 0 H 03 : (  ij = 0 For random effects, replace  i,  j with A i, B j. For three-factor ANOVA, the model extends to:

15. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 15 The two-factor model: effects of age and sex on lizard body size Size Juvenile females Juvenile males Adult females Adult males

16. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 16 The two-factor model: effects of age and sex on lizard body size  Size Adult females Juvenile females Juvenile males Adult males

17. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 17 The two-factor model: effects of age and sex on lizard body size Size Juvenile females Juvenile males Adult females Adult males   

18. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 18 The two-factor model: effects of age and sex on lizard body size Size Juvenile females Juvenile males Adult females Adult males     

19. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 19 The two-factor model: effects of age and sex on lizard body size Size Juvenile females Juvenile males Adult females Adult males            

20. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 20 The two-factor model: effects of age and sex on lizard body size Size Juvenile females Juvenile males Adult females Adult males              

21. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 21 Hypothesis testing in multiple classification ANOVA Proceeds in the same manner as single- classification ANOVA by partitioning the total sums of squares into components due to each factor and their interactions.

22. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 22 The one-way ANOVA table Source of Variation Sum of Squares Mean Square Degrees of freedom F Total Error n - 1 n - k SS/df Groupsk - 1SS/df MS groups MS error i1 k ij j1 n 2 ( Y Y ) i    i i i k n Y Y()    1 2 i1 k i j1 n 2 ( Y Y i)i) i    j

23. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 23

24. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 24 Hypothesis testing in 2-factor factorial design ANOVA H 01 : age has no effect on size: all  i ’s = 0 H 02 : sex has no effect on size: all  j ’s = 0 H 03 : The effects of size and age are independent (no interaction): (  ) ij = 0

25. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 25 Possible outcomes of two-factor ANOVA Decision: accept all H 0 ’s. Inference: body size in lizards is not influenced by sex or age. Size Juvenile Adult Male Female

26. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 26 Possible outcomes of two-factor ANOVA Decision: reject H 01, accept other H 0 ’s. Inference: body size in lizards depends on age but not sex. Size Juvenile Adult Male Female

27. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 27 Possible outcomes of two-factor ANOVA Decision: reject H 02, accept other H 0 ’s. Inference: body size in lizards depends on sex, but not age. Size Juvenile Adult Male Female

28. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 28 Possible outcomes of two-factor ANOVA Decision: reject all H 0 ’s. Inference: body size in lizards is influenced by both age and sex, but the effect of age varies between the sexes (or the effect of sex varies between the ages). Size Juvenile Adult Male Female

29. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 29 Possible outcomes of two-factor ANOVA Decision: reject H 03, accept other H 0 ’s. Inference: there is an effect of sex, but it depends on age (or vice versa); overall, different sexes or different ages do not differ. Size JuvenileAdult Male Female

30. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 30 Possible outcomes of two-factor ANOVA Decision: reject H 02 and H 03, accept H 01. Inference: there is an effect of sex, but it depends on age (or vice versa); overall, different sexes differ in size but different ages do not. Size Juvenile Adult Male Female

31. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 31 Possible results from two-factor ANOVA no effects of any factor effect of sex only effect of age only effect of sex and age, no interaction effect of sex and interaction, no age effect of age and interaction, no sex effect of sex, age, and interaction effect of interaction, no effect of sex or age

32. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 32 Significance testing in 2-factor ANOVA: Model I factorial design Test MS effect over Ms error. Then compare resulting F values to critical values with appropriate degrees of freedom.

33. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 33 Significance testing in 2-factor ANOVA: Model II, III factorial design Test MS I over MS error. If H 0I rejected, test MS effect over MS interaction. Note: in Model III, some authors recommend testing random effect over MS error.

34. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 34 Significance testing in 2-factor ANOVA: Model II, III factorial design Test MS I over MS error. If H 0I accepted, then either (1) test MS effect over MS interaction or (2) pool MS error and MS interaction to get pooled “error” MS E’ with increased df.

35. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 35 Important note If interaction is present, it is not clear whether one should even test main effects! This decision usually depends on the biological hypothesis being tested, NOT on the resulting statistics!

36. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 36 Lab example: effects of sex and location on sturgeon size Note: F-tests assume (incorrectly) Model I ANOVA! Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) SEX 1 1745.36 1745.358 17.72204 0.0000405 LOCATION 1 8.78 8.778 0.08913 0.7656296 SEX:LOCATION 1 48.69 48.692 0.49441 0.4828844 Residuals 178 17530.36 98.485

37. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 37 Multiple comparisons: factorial design If interaction exists, pairwise contrasts usually done across different levels of one factor (e.g. B) within a level of another factor (e.g. A). E.g. for 2-factor ANOVA, each with 3 levels, there are 9 possible contrasts.

38. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 38 Multiple comparisons: factorial design If interaction does not exist, pairwise contrasts are done between treatments for significant effects, pooling over other factor. E.g. compare treatment means of B pooled over factor A (so 3 possible contrasts).

39. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 39 Nested ANOVA: effects of genotype on dessication resistance in fruit flies 3 genotypes (groups, fixed factor) 3 chambers per group (sub-groups, random factor) 5 larvae per chamber, dependent variable is survival in hours.

40. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 40 Significance testing in 2-factor ANOVA: Model II, III nested design Test MS subgroups over Ms error. Test MS groups over Ms subgroups. Note: for nested design, there are only two hypotheses to be tested (versus three in factorial design).

41. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 41 Nested ANOVA: effects of genotype on dessication resistance in fruit flies Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) GENOTYPE2 2952.220 1476.110 292.6081 0.0000000 CHAMBER %in% GENOTYPE 6 40.655 6.776 1.3432 0.2638893 Residuals36 181.608 5.045

42. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 42 Multiple comparisons: nested design If H 0subgroups is accepted, do pairwise contrasts between groups pooling over subgroups. If H 0subgroups is rejected, be careful! E.g. if factor A (subgroups) is non- significant, then compare pooled (over subgroups) group means (3 possible contrasts).

43. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 43 Unreplicated designs Each cell has only a single observation, so cannot compute within-group MS (MS error in replicated designs). So MS interaction becomes MS error. So there is no way of testing for interactions; statistical inferences are based on the assumption (which may not be valid) that no interaction is present. Observation

44. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 44 Significance testing in 2-factor ANOVA: unreplicated Model I factorial design Test MS effect over Ms error... … but remember any inference from this test depends critically on the no interaction assumption!

45. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 45 Significance testing in 2-factor ANOVA: unreplicated Model II factorial design Test MS effect over MS error. Inference(s) from this test are valid whether an interaction is present or not.

46. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 46 Significance testing in 2-factor ANOVA: unreplicated Model III factorial design Test MS effect over Ms error. For random factor, inferences from this test are valid whether an interaction is present or not. For fixed factor, inferences are valid only if an interaction is not present.

47. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 47 Repeated measures ANOVA Used when the same sampling units (individuals, plots etc.) are used in different “treatments”. e.g. blood pressure of patients before, 1 month and 2 months after treatment with anti-hypertensive drug. Can consider this a special Model III ANOVA, with time (factor 1) fixed and patient (factor 2) random serving as replicates within time blocks.

48. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 48 Covariance The sample covariance between two variables X and Y is: Cov(X,Y) = 0 if X and Y are independent, 0 if they are positively correlated. A covariance matrix is a matrix whose diagonal elements are variances and off-diagonal elements are covariances. Variance Covariance

49. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 49 Assumptions of repeated measures ANOVA All standard ANOVA assumptions plus… …data must be compound symmetric, i.e. diagonal elements of the covariance matrix must be equal, as must all the off-diagonal elements. if this assumption is not met, use Greenhouse- Geisser or Huyndh-Feldt test statistics. Variance Covariance A compound symmetric covariance matrix

50. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 50 Example: changes in head dimensions of female children with age Sums of squares are partitioned into a within-individuals among ages (MS age ) component and an among individuals within-age component MS error. H 0 : F = MS age /MS error = 1 Age (yrs) 567 Face width (cm) 6.50 6.75 7.00 7.25 7.50 7.75 8.00

51. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 51 Lab example: influence of age on face width of girls

52. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 52 Non-parametric multiple-classification ANOVA Effect of temperature (fixed) and plant type (fixed) on net primary production (g C/m 2 /day), 4 replicate plots per treatment Calculate ranks, and do 2-factor Model I ANOVA on ranks.

53. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 53 Significance testing in 2-factor non- parametric ANOVA Test H = SS effect over MS total. Compare H to  2 distribution with appropriate degrees of freedom.

54. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 54 Effects of sex and location on sturgeon round weight Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) SEX 1 58393.6 58393.63 22.49791 0.0000042 LOCATION 1 1128.2 1128.21 0.43468 0.5105359 SEX:LOCATION 1 1229.8 1229.82 0.47383 0.4921091 Residuals 182 472383.5 2595.51 H sex = 58393.63/2881.27 = 20.3 (p = 6.620058e-006) H location = 1128.2/2881.27 = 0.39 (p = 0.5322994) H sex:location = 1229.8/2881.27 = 0.43 (p = 0.5119889)

55. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 55 1423 2134 4 3 1 2 3124 431 2

56. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 56 Multiple-classification ANOVA: the randomized block design In factorial designs (completely randomized design) each observation is assumed independent of all other observations. But in some designs (randomized block designs) one observation in one treatment is related in some way to one observation in all other treatments. The set of “related” observations is said to constitute a “block”.

57. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 57 Completely randomized versus randomized block designs From each of 20 sows, 1 offspring is chosen at random and assigned at random to one of 4 experimental diets. So, observations are completely randomized, and the appropriate model is a Model I single-classification ANOVA.

58. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 58 Completely randomized versus randomized block designs From each of 5 sows, 4 offspring are selected at random. Each offspring is assigned at random to one of 4 experimental diets. Since 4 offspring are related, each set of siblings (litter) constitutes a block and the appropriate model is two-factor Model III ANOVA: diets (fixed) and litters (blocks, random).

59. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 59 Randomized blocks: Model III 2-factor ANOVA 2-factor randomized block design is treated as a Model III 2- factor ANOVA without replication, with Factor 1 (fixed) and Factor 2 (blocks) always the random factor.

60. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 60 Example: plant growth under different fertilizer treatments Environmental gradients in temperature, moisture, light etc. may exist in a field or greenhouse, so 5 blocks are created, with 4 plots in each, and plots assigned randomly to one of 4 fertilizer treatments (1,2,3,4). 1423 2134 4 3 1 2 3124 431 2

61. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 61 Example: plant growth under different fertilizer treatments H 0 : growth rate is the same for all treatments. reject H 0 : p(treatment) =.0007 p(blocks) = 0.08, which is suspicious, perhaps indicating environmental variability in the field.

62. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 62 Non-parametric randomized blocks If there is evidence that the k levels of the fixed factor do not met parametric assumptions, use Friedman’s test. For a groups (treatments) and b blocks, the test statistic is: R i is the rank sum for group I, and the test statistic is approximately  2 distributed.

63. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 63 Example: plant growth under different fertilizer treatments H 0 : R i is the same for all treatments. Therefore, reject H 0 : p(treatment) = 0.008

64. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 64 Power with G*Power ANOVA (Type I) metric of effect size : –f –f 2 2 2 2 1 p error R f R  

65. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 65 Power and sample size in Model I two- factor ANOVA Maximal power for given total sample size N is obtained with a balanced design (equal number of observations in each cell). Cell entries are means of N = 5 plots, in tonnes/acre.

66. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 66 Power and sample size in two-factor Model I ANOVA Two factors (A and B), a levels of factor A, b levels of factor B (so k’ = a or b) at specified . Balanced design with n’ observations per cell,  m is cell mean. Given within-group variability s 2 (MS error ), we can calculate  for each factor:

67. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 67 Calculating power given  Given 1 (factor df, e.g. a-1, b-1), 2 (within-cell (error) df),  and , can read 1-  from suitable tables or curves (e.g. Zar (1996), Appendix Figure B.1). 1-  Decreasing 2 1 = 2  =.05 2345  =.01 11.522.5  =.05)  =.01)

68. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 68 Model I ANOVA: minimal detectable difference Suppose we want to detect a difference between the two most different sample means of each level of some factor (A, B) at least . To test at the  significance level with 1 -  power, we can calculate the minimal sample size n min required to detect , given a sample group variance s 2, by solving iteratively.

69. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 69 Power of the test: main effects If H 0 is accepted, it is good practice to calculate power! Knowing MS factor, s 2 (= MS error ), and k’, we can calculate  for each main effect.

70. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 70 Power of the test: interaction effect If H 0 is accepted, it is good practice to calculate power! Knowing MS interaction and s 2 (= Ms error ) we can calculate  for the interaction.

71. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 71 Power of the test: an example Effect of hormone treatment on bird plasma concentrations 5 individuals of each sex (Factor B, male (M) or female (F)) with or without hormone treatment (Factor A) H 0 accepted for main effect B (sex) and interaction.

72. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 72 Power of the test: an example What is the power of the test for a sex effect? So there is about a 71% chance of committing a Type II error.

73. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 73 Power and sample size in Model III two-factor ANOVA: replicated design In Model III ANOVA with replication, we can calculate power with respect to hypotheses concerning the fixed factor by substituting MS interaction for MS error, and using df interaction for 2. Body size (kg) 120 160 200 240 280 Riding Mountain Kluane Algonquin males females

74. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 74 Power and sample size in Model III two-factor ANOVA: unreplicated design In Model III ANOVA without replication, we can calculate power with respect to hypotheses concerning the fixed factor by substituting MS remainder for MS error, and using df remainder for 2.

75. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 75 Multi-way ANOVA In principle, the procedures for 2-way ANOVAs can be extended to 3 or more factors, e.g. effect of species (factor 1), temperature (factor 2) and sex (factor 3) on respiration rate in crabs. In practice, higher-order ANOVAs are often difficult to interpret because of the large number of possible null hypotheses.

76. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 76 3-way Model I factorial ANOVA In 3-way factorial model I ANOVA, there are 7 null hypotheses. All effects MS are tested over MS error.

77. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 77 Number of testable hypotheses in multi- way factorial ANOVA As the number of factors increases, the number of possible hypotheses more than doubles.

78. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 78 Multi-way ANOVA: blocked and repeated measures designs Experimental designs with 3 or more factors where one of the factors is a block or a subject upon which repeated measures are taken Analysis proceeds as a factorial ANOVA with the block or repeated factor considered a random- effects factor. Hence, all such designs are Model III.

79. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 79 Distinguishing blocked and repeated measures designs If each block consists of a number of independent observations (e.g. individuals), then the design is blocked. In a split-plot design, the same block is applied to some - but not all - of the combination of factors. In repeated measures designs, the same individual is exposed to some - but not all - of the combination of factors.

80. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 80 Effects of diet and exercise on animal body weight: blocked design Body weight measured on 2 diets (D 1, D 2 ) and 2 exercise regimens (E 1, E 2 ), 4 blocks per diet, and 2 animals per block, one animal assigned at random to each of the 2 exercise regimens (16 animals in total). Diet and exercise are fixed factors.

81. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 81 Effects of diet and exercise on animal body weight: repeated measures design Body weight measured on 2 diets (D 1, D 2 ) and 2 exercise regimens (E 1, E 2 ), 4 subjects per diet, each subject exposed to both exercise regimens (8 animals in total). Diet and exercise are fixed factors. Note that N here is half of the blocked design!

82. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 82 Multi-way ANOVA - repeated measures designs Effect of sex (factor 1) and diet (factor 2) on plasma protein levels of turtles (subjects, repeated factor), 8 animals in total

83. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 83 Variance decomposition Test between (within)-subject effects over between(within) -subjects error, e.g.

84. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 84 Multi-way ANOVA - blocked design Effect of sex (factor 1) and diet (factor 2) on plasma protein levels of turtles, 4 blocks per sex, 3 animals per block (24 animals total)

85. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 85 Variance decomposition: blocked design Test fixed factor MS over remainder MS. Treat test of MS blocks /MS remainder with caution!

86. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 86 Power and sample size in multi-way ANOVA The principles and procedures for two-way ANOVA can be extended quite simply to multi-way ANOVA.

87. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 87 Model I multi-way ANOVA For calculating , k’ is number of levels of (fixed) factor under consideration, n’ is total number of data (i.e. pooled over other factors) in each level of the factor, s 2 is MS error.

88. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 88 Power and sample size in 3-way Model I ANOVA: an example Effect of species, sex and temperature on crab O 2 consumption (crabO2.sys)

89. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 89 Model I ANOVA: minimal detectable difference Suppose we want to detect a difference between the two most different sample means of each level of sex of at least . For 1 = 1, 2 = 54, to test at the .05 significance level with 1 - .95 (say), we require  = 5.2, giving n min = 1590!

90. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 90 Power of the test: main effects What is the power of the test for an effect of sex? So there is about a 79% chance of committing a Type II error.

91. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 91 Power of the test: interaction effect What is the power to detect a sex X temp interaction? So there is about a 75% chance of detecting an interaction.

92. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 92 Model III multi-way ANOVA To estimate power to detect fixed effects in model III ANOVA, use previous procedures, except…...for within-cells effect (s 2 ), substitute the appropriate denominator in the F test for significance of the effect. This is often NOT MS error !

93. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 93 Example: calculating in repeated measures designs Effect of sex (fixed factor 1) and diet (fixed factor 2) on plasma protein levels of turtles (subjects, repeated factor), 8 animals in total

94. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 94 Power to detect a sex effect To calculate , substitute subjects within sex MS and df for s 2.

95. University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 14/08/2015 3:33 AM 95 Multi-way ANOVA: critical questions How many factors? Model type (I, II, or III)? Model design –factorial? –nested? –blocked? –repeated measures? –mixed? Replication –balanced/unbalanced? –unreplicated? Parametric or non- parametric? The answers to these questions determines what hypotheses can be tested and what the appropriate test is. So, get them right!