January 7, afternoon session 1 Multi-factor ANOVA and Multiple Regression January 5-9, 2008 Beth Ayers

January 7, afternoon session 2 Thursday Session ANOVA ‒ One-way ANOVA ‒ Two-way ANOVA ‒ ANCOVA ‒ With-in subject ‒ Between subject ‒ Repeated measures ‒ MANOVA ‒ etc. Comparisons of different designs

January 7, afternoon session 3 Some Terminology Between subjects design – each subject participates in one and only one group Within subjects design – the same group of subjects serves in more than one treatment ‒ Subject is now a factor Mixed design – a study which has both between and within subject factors Repeated measures – general term for any study in which multiple measurements are measured on the same subject ‒ Can be either multiple treatments or several measurements over time

January 7, afternoon session 4 With-in Subjects New methods are needed that do not make the assumption of no correlation (independence) of errors Since subjects are receiving more than one treatment in within-subjects designs, we expect outcomes to be correlated

January 7, afternoon session 5 Why With-in Subjects Designs? We may want to study the change of an outcome over time Studying multiple outcomes for each subject allows each subject to be his or her own “control”

January 7, afternoon session 6 Advantages All sources of variability between subjects is excluded from the experimental error Repeated measures economizes subjects, which is important when only a few subjects can be utilized for the experiment Increased power

January 7, afternoon session 7 Disadvantages Interference/confounding ‒ Order effect ‒ Connected to the position in the treatment order ‒ Carryover effect ‒ Connected with the preceding treatment or treatments ‒ Practice effect ‒ Students get better with practice on preceding treatment Various steps can be taken to minimize the dangers of interference

January 7, afternoon session 8 Fixed vs. Random Factors Fixed factors – the levels are the same levels you would use if you repeated the experiment ‒ Treatments are usually fixed factors Random factors – a different set of levels would be used if you repeated the experiment ‒ Subjects are normally considered a random factor

January 7, afternoon session 9 Repeated Measures Analysis Repeated measures analysis is appropriate when one or more factors is a within-subjects factor Planned (main effect) contrasts are appropriate for both factors if there is no significant interaction Post-hoc comparisons can also be performed ‒ Must take ® level into consideration if doing post-hoc testing

January 7, afternoon session 10 Assumptions of Repeated Measures Normal distribution of the outcome for each level of the with-in subjects factor Errors are assumed to be uncorrelated between subjects Within a subject, the multiple measurements are assumed to be correlated A technical condition called sphericity must be met ‒ Population variances of repeated measures are equal ‒ Population correlations among all pairs of measures are equal ‒ Statistical packages can check this!

January 7, afternoon session 11 Relation to Paired t-test If we have a treatment with two levels and each subject received both, a paired T-test gives the same results as a two- way ANOVA with subject and treatment as factors

January 7, afternoon session 12 Keyboard Example Paired t-test results ANOVA results

January 7, afternoon session 13 Example An experiment is conducted to compare energy requirements of three activities: running, walking, and biking 12 subjects are asked to run, walk, and bike a required distance and the number of kilocalories burned is measured The activities are done in a random order with recovery time in between Each subject does each activity once

January 7, afternoon session 14 Example Why is random order used? Why can’t we used a paired t-test?

January 7, afternoon session 15 Example Why is random order used? ‒ Concerned about carry-over effect Why can’t we used a paired t-test? ‒ There are three levels to the explanatory variable

January 7, afternoon session 16 Exploratory Analysis

January 7, afternoon session 17 Exploratory Analysis Mean energy output for each activity

January 7, afternoon session 18 Analysis Use Sphericity Assumed row, assuming that we’ve run the check and the assumption is met

January 7, afternoon session 19 Contrasts Since there are k=3 levels of exercise, we can only do 2 Level 1 = cycling, level 2 = walking, level 3 = running Can say that walking consumes more energy than cycling and that running consumes more than walking

January 7, afternoon session 20 Comparisons Need to run comparisons to compare cycling to running The 1 vs. 3 shows us that there is a significant difference

January 7, afternoon session 21 Mixed between/within ANOVA One factor is varied between subjects and the other is within subjects Need to check interaction

January 7, afternoon session 22 Example Add gender to the previous within subjects exercise and energy consumption example

January 7, afternoon session 23 Exploratory Analysis

January 7, afternoon session 24 Exploratory Analysis

January 7, afternoon session 25 Analysis Conclusions?

January 7, afternoon session 26 Analysis Unfortunately SPSS doesn’t allow you to remove the interaction in repeated measures Options ‒ Interpret main effects in presence of the non- significant interaction ‒ Use another statistical package

January 7, afternoon session 27 Power A simple Google search for power repeated measures ANOVA turns up pages worth of online applets Pick one that you understand

January 7, afternoon session 28 Name that Experimental Design X1X1 X 2 Level 1 Level 2 Level 1 s1s2s3s4S5s1s2s3s4S5 s 16 s 17 s 18 s 19 s 20 Level 2s 6 s 7 s 8 s 9 s 10 s 21 s 22 s 23 s 24 s 25 Level 3 s 11 s 12 s 13 s 14 s 15 s 26 s 27 s 28 s 29 s 30 X1X1 X 2 Level 1 Level 2 Level 1 s1s2s3s4s5s1s2s3s4s5 s1s2s3s4s5s1s2s3s4s5 Level 2s1s2s3s4s5s1s2s3s4s5 s1s2s3s4s5s1s2s3s4s5 Level 3 s1s2s3s4s5s1s2s3s4s5 s1s2s3s4s5s1s2s3s4s5 X1X1 X 2 Level 1 Level 2 Level 1 s1s2s3s4s5s1s2s3s4s5 s 6 s 7 s 8 s 9 s 10 Level 2s1s2s3s4s5s1s2s3s4s5 s 6 s 7 s 8 s 9 s 10 Level 3 s1s2s3s4s5s1s2s3s4s5 s 6 s 7 s 8 s 9 s

January 7, afternoon session 29 Notes on designs All three give interaction and main effects information, but vary in the number of subjects needed Two-factor repeated measures – provides good precision since all sources of variability between subjects is excluded Mixed design – reduce carryover effects since each subject is exposed to less treatments The mixed design is usually the design of choice when the researcher is studying learning and the process that influences the speed with which learning takes place

January 7, afternoon session 30 MANOVA An extension of ANOVA where there is more than one dependent variable and the dependent variables can not be combined