1. Factorial Analysis of Variance One dependent variable, more than one independent variable (“factor”)

2. KNR 445 FACTORIAL ANOVA Slide 2 Two factors, more reality  Imagine you want to describe what influences GPA, body fat, a team’s winning %, the outcome of an electoral poll…  Surely, the person who says “it was because [just one thing]” is a fool?  Well, then people who conduct one way ANOVA’s are fools!  More variables simply get closer to the truth

3. KNR 445 FACTORIAL ANOVA Slide 3 Two factors, more reality  But there’s more to it than that (of course)  Consider this experiment:  Take 2 sets of golfers: 1 set (A 1 ) is high anxious, 1 set (A 2 ) is low anxious  Assign 1/3 of each set of golfers to a different performance scenario: Low pressure (B 1 ), Moderate pressure (B 2 ), High pressure (B 3 )

4. KNR 445 FACTORIAL ANOVA Slide 4 Two factors, more reality  So for assignment to groups we get: Situation Low Pressure Moderate pressure High Pressure Anxiety Level Low Anxiety n cell = 2 n(A 1 ) =6 High Anxiety n cell = 2 n(A 2 ) =6 n(B 1 ) = 4n(B 2 ) = 4n(B 3 ) = 4n total = 12

5. KNR 445 FACTORIAL ANOVA Slide 5 Two factors, more reality  Suppose that the performance scores are… Situation Low Pressure Moderate pressure High Pressure Anxiety Level Low Anxiety M = 5 (3, 7) M = 8 (7, 9) M = 11 (10, 12) M A1 = 8 High Anxiety M = 4 (3, 5) M = 6 (5, 7) M = 2 (2, 2) M A2 = 4 M B1 = 4.5M B2 = 7M B3 = 6.5M total = 12

6. KNR 445 FACTORIAL ANOVA Slide 6 MAIN EFFECTS  What do we find?  We can consider the overall effect of anxiety (Factor A) on performance  The null hypothesis here would be  This is analogous to doing a 1-way ANOVA on the row means of M A1 (8) and M A2 (4) NB: if you were to do a 1-way ANOVA, you’d ignore the effect of situation completely

7. KNR 445 FACTORIAL ANOVA Slide 7 MAIN EFFECTS  This overall effect of anxiety is called the main effect of anxiety

8. KNR 445 FACTORIAL ANOVA Slide 8 MAIN EFFECTS  What do we find?  We can also consider the overall effect of situation (Factor B) on performance  The null hypothesis here would be  This is analogous to doing a 1-way ANOVA on the row means of M B1 (4.5), M B2 (7) and M B3 (6.5) NB: here, you’d ignore the effect of anxiety completely

9. KNR 445 FACTORIAL ANOVA Slide 9 MAIN EFFECTS  This overall effect of situation is called the main effect of situation  In each of the main effects, note that each mean within the main effect has been computed by averaging across levels of the factor not considered in the main effect  This is how it is ignored, statistically. Its effects are, quite literally, averaged out WHENEVER YOU INTERPRET A MAIN EFFECT, YOU SHOULD PAY ATTENTION TO THE FACT THAT IT AVERAGES ACROSS LEVELS OF THE OTHER FACTOR – ESPECIALLY WHEN YOU GET…

10. KNR 445 FACTORIAL ANOVA Slide 10 INTERACTIONS  Note the difference between each pair of means in our original table of data Situation Low Pressure Moderate pressure High Pressure Anxiety Level Low Anxiety M = 5 (3, 7) M = 8 (7, 9) M = 11 (10, 12) M A1 = 8 High Anxiety M = 4 (3, 5) M = 6 (5, 7) M = 2 (2, 2) M A2 = 4 M B1 = 4.5M B2 = 7M B3 = 6.5M total = 12 5-4 = 18-6 = 2 11-2 = 9

11. KNR 445 FACTORIAL ANOVA Slide 11 INTERACTIONS  The magnitude of the difference changes depending on the situation Situation Low Pressure Moderate pressure High Pressure Anxiety Level Low Anxiety M = 5 (3, 7) M = 8 (7, 9) M = 11 (10, 12) M A1 = 8 High Anxiety M = 4 (3, 5) M = 6 (5, 7) M = 2 (2, 2) M A2 = 4 M B1 = 4.5M B2 = 7M B3 = 6.5M total = 12 5-4 = 1 8-6 = 2 11-2 = 9

12. KNR 445 FACTORIAL ANOVA Slide 12 INTERACTIONS  The magnitude of the difference changes depending on the situation  In other words, the effect of anxiety on performance depends on the situation in which the participants are asked to perform  In other words, the situation moderates the effect of anxiety on performance  In other words, the anxiety-performance relationship differs depending on the situation

13. KNR 445 FACTORIAL ANOVA Slide 13 INTERACTIONS  Also, the magnitude of the difference changes depending on anxiety level  In other words, the effect of situation on performance depends on the anxiety level of the participants  In other words, anxiety level moderates the effect of situation on performance  In other words, the situation-performance relationship differs depending on anxiety level

14. KNR 445 FACTORIAL ANOVA Slide 14 INTERACTIONS  Whatever way you slice it, it’s the same thing, and it’s easier to see in a graph: Ordinal interaction = lines do not cross

15. KNR 445 FACTORIAL ANOVA Slide 15 INTERACTIONS  The essential point is, when the lines are non-parallel (significantly so), you have an interaction, and the effect of one factor on the dependent variable depends on the level of other factor being considered Non-parallelism implies interaction

16. KNR 445 FACTORIAL ANOVA Slide 16 INTERACTIONS  So, is this an interaction?

17. KNR 445 FACTORIAL ANOVA Slide 17 INTERACTIONS  How about this?

18. KNR 445 FACTORIAL ANOVA Slide 18 INTERACTIONS  This is a disordinal interaction

19. KNR 445 FACTORIAL ANOVA Slide 19 Why bother with interactions?  With figure B, it seems we have a main effect of anxiety level  That implies that the effect of anxiety on performance can be generalized across different pressure conditions.  With figures A and C, generalization across situations would be a serious mistake  A main effect would fail to acknowledge that the effect of anxiety changes across situations  In which figure, A or C, would the main effect of anxiety be more likely?

20. KNR 445 FACTORIAL ANOVA Slide 20 Why bother with interactions?  Note: With disordinal interactions, post-hoc tests can be confusing, as they may result in no apparent significant differences. Here the interaction is not necessarily caused by a large difference between pairs of means, but may be a function of the change in direction of difference across pairs of means as well

21.  Enough for this week. Now to the practice…  (next week we’ll do the rest of this set) KNR 445 FACTORIAL ANOVA Slide 21

22. KNR 445 FACTORIAL ANOVA Slide 22 Statistically speaking…  With Factor A, Factor B, and the interaction A x B, the sums of squares are as follows: Recall: But: So:

23. KNR 445 FACTORIAL ANOVA Slide 23 Statistically speaking…  These are converted to variance estimates by dividing by the d. of f.: So… And: Variance for main effect of factor A Variance for main effect of factor B Variance for interaction between A & B

24. KNR 445 FACTORIAL ANOVA Slide 24 Statistically speaking…  And remember, these are estimates:

25. KNR 445 FACTORIAL ANOVA Slide 25 Statistically speaking…  Or, in another form: Total sum of squares Within-groups SS Between-groups SS SS associated with A SS associated with B SS associated with interaction

26. KNR 445 FACTORIAL ANOVA Slide 26 Statistically speaking…  So the F-tests are: For Factor A:

27. KNR 445 FACTORIAL ANOVA Slide 27 Statistically speaking…  So the F-tests are: For Factor B:

28. KNR 445 FACTORIAL ANOVA Slide 28 Statistically speaking…  So the F-tests are: …and for the interaction:

29. KNR 445 FACTORIAL ANOVA Slide 29 Assumptions  Similar to 1-way ANOVA  Assumptions now stated for cells, not groups. It is assumed that…  Observations are independent (uncorrelated) from one cell to the next   2 is the same for all cells (homogeneity of variance)  Cell populations are normally distributed  Last 2 are mostly a concern for small sample sizes Situation Low Pressure Moderat e pressure High Pressure Anxiety Level Low Anxiety M = 5 (3, 7) M = 8 (7, 9) M = 11 (10, 12) M A1 = 8 High Anxiety M = 4 (3, 5) M = 6 (5, 7) M = 2 (2, 2) M A2 = 4 M B1 = 4.5M B2 = 7M B3 = 6.5 M total = 12

30. KNR 445 FACTORIAL ANOVA Slide 30 Assumptions  Note – we have been discussing equal cell sizes throughout.  This is important, as it guarantees independence of statistical effects in the analysis  If cell sizes are unequal, certain adjustments must be made…  These involve reporting different types of sums of squares (Types 1 to 3 as reported in SPSS, Stevens, 1986, 1996) But that is probably a step too far for this class…just make sure your cell sizes are equal!

31. KNR 445 FACTORIAL ANOVA Slide 31 Factorial ANOVA in SPSS  Data considerations  Dependent variable: interval/ratio, normally distributed within each cell of the analysis)  Independent variables: must be discrete categories  If the independent variable is continuous, the number of categories can be created artificially by doing a median split or quartile split, BUT you’d be better off with regression  Must be independent (see slide 28)

32. KNR 445 FACTORIAL ANOVA Slide 32 Factorial ANOVA in SPSS  Data considerations Each row is one subject There is now more than one grouping factor, so “group” is not a good name for either Dependent variable

33. KNR 445 FACTORIAL ANOVA Slide 33 Factorial ANOVA in SPSS  Performing the analysis The general linear model is the same family of statistical techniques as we used in regression – we could dummy code these variables and get exactly the same answer using regression techniques 1 dependent variable, so choose univariate

34. KNR 445 FACTORIAL ANOVA Slide 34 Factorial ANOVA in SPSS  Performing the analysis They just need to be slid over to the right places Here are the variables as listed in the data file

35. KNR 445 FACTORIAL ANOVA Slide 35 Factorial ANOVA in SPSS  Performing the analysis This is just a descriptor – it has no function in the analysis Here are the factors Here is the dependent variable Further options

36. KNR 445 FACTORIAL ANOVA Slide 36 Factorial ANOVA in SPSS  Performing the analysis “Plots” lets you request a graph of the 2 factors

37. KNR 445 FACTORIAL ANOVA Slide 37 Factorial ANOVA in SPSS  Performing the analysis 2. Click on “Add” to request the plot 3. Selected plot appears here 1. Slide the factors across to the correct boxes 4. Click “continue” to do so!

38. KNR 445 FACTORIAL ANOVA Slide 38 Factorial ANOVA in SPSS  Performing the analysis “Post Hoc” lets you request follow-ups, but only to the main effects – and that is pointless here, because…?

39. KNR 445 FACTORIAL ANOVA Slide 39 Factorial ANOVA in SPSS  Performing the analysis If you wanted to do a post hoc on the main effects, you’d simply: 1. select the variables 2. Slide them over 3. Select the post hoc test 4. Continue

40. KNR 445 FACTORIAL ANOVA Slide 40 Factorial ANOVA in SPSS  Performing the analysis “Options” lets you request descriptives, effect sizes, power statistics, and homogeneity tests. It is the most important box to choose

41. KNR 445 FACTORIAL ANOVA Slide 41  Performing the analysis Factorial ANOVA in SPSS Here you see them: Descriptives Effect sizes Power calculation Homogeneity tests

42. KNR 445 FACTORIAL ANOVA Slide 42 Factorial ANOVA in SPSS  Performing the analysis Finally, click “ok” once you’ve specified what you want

43. KNR 445 FACTORIAL ANOVA Slide 43 Factorial ANOVA in SPSS  The output Here’s the table showing the allocation of subjects to levels of each factor Here’s the table showing the descriptive stats – note the balanced design!

44. KNR 445 FACTORIAL ANOVA Slide 44 Factorial ANOVA in SPSS  The output Here’s the homogeneity test – we’re OK – it is not significant, which means the assumption is met

45. KNR 445 FACTORIAL ANOVA Slide 45 Factorial ANOVA in SPSS  The output Here’s the ANOVA summary table Main Effects Interaction Observed Power Significance  2 – effect sizes – cite in write up (like R 2 ’s for each effect)

46. KNR 445 FACTORIAL ANOVA Slide 46 Factorial ANOVA in SPSS  The output  2 – effect sizes – cite in write up (like R 2 ’s for each effect) We need to say a bit more about this. What is displayed here is partial  2. It’s worth pausing for a moment here to consider what kind of an effect size that is

47. KNR 445 FACTORIAL ANOVA Slide 47 Factorial ANOVA in SPSS  Eta-squared (  2 ) & partial eta-squared.   2 is exactly like an R 2. So it can be treated the same.  As the number of factors in the equation increases, so the SS total will increase  So the  2 of any one factor will diminish with the introduction of other factors  All  2 in the analysis will sum to 1 or less (as per R 2 ) From: Levine & Hullet, 2002 (see web site for full citation)

48. KNR 445 FACTORIAL ANOVA Slide 48 Factorial ANOVA in SPSS  Eta-squared (  2 ) & partial eta-squared.  Partial  2 is calculated so that it does not shrink with the introduction of more factors:  For 1-way ANOVA, identical to  2. For multi- factorial designs, can be drastically different  Partial  2 can sum to more than one when there are several strong effects in the analysis. From: Levine & Hullet, 2002 (see web site for full citation)

49. KNR 445 FACTORIAL ANOVA Slide 49 Factorial ANOVA in SPSS  Eta-squared (  2 ) & partial eta-squared.  SPSS reports partial  2. Despite labeling it just eta squared.  What does this imply?  Partial is still ok as a measure of effect size, and is accepted by journals.  But you should be aware that it overestimates effect size when sample size is small (it is a “biased estimator”)  The best estimates are omega (  ) and epsilon (  )(neither of which are available in SPSS, of course) – each is unbiased, and so doesn’t change with sample size  So, just keep using partial eta squared, but report it as that, and be aware of the bias. From: Levine & Hullet, 2002 (see web site for full citation)

50. KNR 445 FACTORIAL ANOVA Slide 50 Factorial ANOVA in SPSS  And now back to...The output Here, we have significant main effects AND a significant interaction

51. KNR 445 FACTORIAL ANOVA Slide 51 Factorial ANOVA in SPSS  The output It is normal in such circumstances to report the means for the main effects but to note that these are superceded by the interaction, which implies that the main effects are not genuine

52. KNR 445 FACTORIAL ANOVA Slide 52 Factorial ANOVA in SPSS  The output This is an important statistic. It has become conventional to report a measure of effect size as well as the significance when reporting your results. This is the one most often chosen in papers using ANOVAs. It is cited as  2 in SPSS but is in fact a partial  2.

53. KNR 445 FACTORIAL ANOVA Slide 53 Factorial ANOVA in SPSS  The output LASTLY, here is the plot. You can clearly (?) see the main effects are not genuine, and that the only way to really describe what is going on here is through the interaction

54. KNR 445 FACTORIAL ANOVA Slide 54 Factorial ANOVA in SPSS  Next:  More on follow-up tests  What to do if the homogeneity assumption is not met  What to do if you have multiple measures on each subject (repeated measures!)

55. Factorial Analysis of Variance II Follow ups and Repeated Measures More fun than a rub down with a cheese grater

56. KNR 445 FACTORIAL ANOVA II Slide 56 Follow-ups for Factorial ANOVA  Recall possible outcomes from Factorial ANOVA:  Main effects  Interactions  What might be missing (not specified) from these results?

57. KNR 445 FACTORIAL ANOVA II Slide 57 Follow-ups for Factorial ANOVA  What might be missing from (not specified by) these results?  Differences between pairs of means within each factor (if levels of factor are > 2)  Differences between cells giving rise to interactions

58. KNR 445 FACTORIAL ANOVA II Slide 58 Follow-ups for Factorial ANOVA  Differences between pairs of means: “Post Hoc” lets you request follow-ups, but only to the main effects

59. KNR 445 FACTORIAL ANOVA II Slide 59 Follow-ups for Factorial ANOVA  Differences between pairs of means: To do a post hoc on the main effects: 1. select the variables 2. Slide them over 3. Select the post hoc test 4. Continue

60. KNR 445 FACTORIAL ANOVA II Slide 60 Follow-ups for Factorial ANOVA  Leaves one problem:  Differences between cells giving rise to interactions  How do we do follow-ups on significant interactions?  Consider the meaning of “follow-up”  It is asking: “where is the difference within the collection of cells that specifically gives rise to the significant effect?”  Implies breaking down the overall picture to examine patterns of significance and non- significance among smaller groups of cells

61. KNR 445 FACTORIAL ANOVA II Slide 61 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Method one: By inspection  Advantage – no information lost  Look at the overall shape of the graph and describe how the effect of one factor on the dependent variable changes as the level of the other factor changes. Try it for these...

62. KNR 445 FACTORIAL ANOVA II Slide 62 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Method two: By subsequent simpler analyses (what you should do)  These should be chosen according to the kind of research question you are trying to answer  For instance in figure A you might be interested in changes in performance due to levels of pressure

63. KNR 445 FACTORIAL ANOVA II Slide 63 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Method two: By subsequent simpler analyses  In this instance it would make sense to do separate follow up 1-way ANOVAs for each of the 2 Anxiety levels, using pressure as the factor, and performance as the d.v.

64. KNR 445 FACTORIAL ANOVA II Slide 64 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Method two: By subsequent simpler analyses  Because you are now conducting multiple tests, you should adjust your significance threshold to control for type 1 error.  The Bonferroni adjustment is suitable here – divide  by the number of tests being run (2 here – giving  =.05/2 =.025)

65. KNR 445 FACTORIAL ANOVA II Slide 65 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Method two: By subsequent simpler analyses  If in figure A you were interested in changes in performance due to levels of anxiety...  Then it might make more sense to run three t- tests, one for each level of pressure, with anxiety as the factor and performance as the d.v.  Again you would divide  by the number of tests being run (3 now – giving  =.05/3 =.017)

66. KNR 445 FACTORIAL ANOVA II Slide 66 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Bear in mind that any test conducted after the initial interaction is less powerful than the initial test  That’s why interpreting the picture sometimes makes sense to me  What you can’t do after looking at the picture is then claim that the difference is “primarily between these two conditions” – that would be claiming a pairwise difference without evidence

67. KNR 445 FACTORIAL ANOVA II Slide 67 Follow-ups for Factorial ANOVA  Follow-ups on significant interactions :  Note on ordinal (uncrossed) and disordinal (crossed) interactions  Regardless of whether the interaction crosses or not, there is a good chance that main effects found in these analyses are not genuine (that is their existence depends on the level of the other factor)  Always interpret a main effect with caution if there is a significant interaction involving that main effect Uncrossed – genuine main effect Uncrossed – no genuine main effect Crossed – no genuine main effect

68. KNR 445 FACTORIAL ANOVA II Slide 68 Follow-ups for Factorial ANOVA  Summary  No significant effects -No follow ups  Significant main effect only  Pairwise comparisons within significant effects  Significant main effects and a significant interaction  Caution in interpreting main effects (examine graph of interaction)  Try to find the locus of the interaction (visually or by further ANOVAs and t-tests with bonferroni adjustment)  Significant interaction only

69. Factorial ANOVA Follow-ups: SPSS  Let’s say we have a 2 x 2 ANOVA  Two independent variables with 2 levels (practice type and trial length)  No need to do follow-ups on main effects (why?)  For an interaction, what follow-ups would we do? KNR 445 FACTORIAL ANOVA Slide 69