1. 11/19/2015Slide 1 We can test the relationship between a quantitative dependent variable and two categorical independent variables with a two-factor analysis of variance (ANOVA). The two variables are referred to as “factors.” The question we want to answer in analysis of variance is whether or not one or both of the categorical independent variables has a relationship to the quantitative dependent variable, indicated by a difference in the means across the categories of the factors. The two-factor analysis tests for an interaction (combined) effect as well as main (individual) effects for the two independent variables. An interaction effect implies that we cannot interpret one of the independent variables without taking the other into account. The two-factor analysis of variance actually tests three null hypotheses: The means are equal for all combinations of the two factors (interaction effect) The means are equal for the categories of the first factor (first main effect) The means are equal for the categories of the second factor (second main effect) If we fail to reject the null hypothesis for the interaction effect, i.e. conclude that there is an interaction effect, we do not interpret the main effects because the interpretation of the interaction contradicts the interpretation of one or both of the main effects.

2. 11/19/2015Slide 2 We can have the following outcomes to our analysis: The interaction is statistically significant and we interpret the interaction effect, but not the main effects The interaction is not statistically significant, but one or both of the main effects are significant and we interpret the significant effects The interaction effect is not statistically significant, and neither are either of the main effects NOTE: in these problems the word “effect” is used twice to mean two different things. Main effects and interaction effects define a relationship between the variables. Effect size is a measure of the strength of the relationship, i.e. variance accounted for. On the following slides we will examine the relationship between income, self- employment, and sex from the GSS2000R.sav data set.

3. 11/19/2015Slide 3 If we do a one-way analysis of variance of income by sex, both the ANOVA table and the chart indicate that there is a statistically significant difference between income for males and females, with females earning significantly less.

4. 11/19/2015Slide 4 If we do a one-way analysis of variance of income by self-employment, both the ANOVA table and the chart indicate that there is no statistically significant difference between income for persons who are self-employed and persons who work for someone else. Based on these two one-way analyses of variance, we might expect that males would earn more than females regardless of their employment status, and self-employed persons earn roughly the same as persons who work for some else regardless of their gender.

5. 11/19/2015Slide 5 When we examine the combination of sex and self-employment on income, the two-factor analysis of variance indicates a significant relationship for the interaction of sex and self-employment. The line chart shows us the pattern that produced the significant interaction. The blue line for males slopes downward from self-employed, indicating that males who were self-employed earned more than males who worked for someone else. The green line for females slopes upward from self-employed, indicating that females who were self-employed earned less than females who worked for someone else.

6. 11/19/2015Slide 6 What about the main effect for sex? Can we say that males earn more than females whether they are self- employed or not? Yes, that is true, since the blue line is higher than the green line for both the self-employed group and the group who work for someone else. The gender difference is re- enforced by the ANOVA table which indicates a statistically significant difference in means by gender, even after the difference attributed to the interaction is taken into account.

7. 11/19/2015Slide 7 The lack of significance is re-enforced by the ANOVA table which indicates not statistically significant difference in means by self-employment What about the main effect for self- employment? Can we say that self-employed persons earn more than persons who work for someone else? No, that is true for males where the blue line slopes downward from self-employed, but not for females where the green line slopes upward from self-employed, indicating that females who work for someone else earn more than females who are self-employed.

8. 11/19/2015Slide 8 The author of your text suggests that main effects are not interpreted when there is a significant interaction, and this is the advice we will follow in working our problems. Others suggest that main effects can be interpreted even when there is a significant interaction, and this example is one in which I think that is reasonable.

9. 11/19/2015Slide 9 This is the screen for a two- factor analysis of variance problem.

10. 11/19/2015Slide 10 The first paragraph identifies the analysis to be conducted (two-factor) ANOVA and the factors to be tested (main effects and interaction effect).

11. 11/19/2015Slide 11 The first sentence in the next paragraph asks which variable is the target of preliminary data screening. The correct answers is the dependent variable. There is no expectation that the factors (categorical variables) be normally distributed.

12. 11/19/2015Slide 12 The second sentence in the paragraph asks about the normality of the dependent variable. Based on the information in the problem narrative (skewness = 0.56, kurtosis = -0.33, 0 outliers), we would conclude that the variable is nearly normal.

13. 11/19/2015Slide 13 The next blanks expect us to enter the value of the F ratio and p-value for the Levene test of homogeneity of variance. The Levene test of homogeneity of variance tests the null hypothesis that the variance of the groups is equal versus the alternative hypothesis that the variance of the one or several groups is different from the variance of the other groups. Unlike the independent samples t-test, there is not an alternative formula to use when we violate this assumption. The two-factor Anova is robust to the violation of this assumption if the counts in the groups are similar. In our problems, a violation of this assumption would result in answers of na for all subsequent questions.

14. 11/19/2015Slide 14 We compute factorial analysis by selecting General Linear Model > Univariate from the Analyze menu. To answer this question, we run the two-factor ANOVA.

15. 11/19/2015Slide 15 First, we move poverty to the Dependent Variable text box. Second, we move freeMove and freeReli to the Fixed Factors list box. Factors are fixed if all of the possible values are included in our data set. Random factors are used for variables where there are more possible responses than we have included in our data set. Third, click on the Options button to specify the needed output options.

16. 11/19/2015Slide 16 We highlight all of the variables listed in the Factor(s) and Factor interactions list box and click on the arrow button to move them to the Display Means for list box. Note: the interaction term is included by default in SPSS. If we wanted to exclude it, we would have to create a different model specifically. In addition to testing for the main effects and the interaction effects, we want to compute post hoc tests for differences in freeReli within categories of freeMove. Other options in SPSS support post hoc tests for individual factors, but this is what we must use to compute post hocs for combinations of two factors.

17. 11/19/2015Slide 17 With all of the entities listed in the Display Means for list box, we mark the check box Compare main effects. To control the alpha error rate for the multiple comparisons, we select the Bonferroni adjustment from the Confidence interval adjust drop down menu.

18. 11/19/2015Slide 18 We mark the other options to display: Descriptive statistics for the number of cases in each cell Estimated effect size for partial eta squared, and Homogeneity tests for the Levene test. Click on the Continue button to close the dialog box.

19. 11/19/2015Slide 19 Plots can be very helpful in interpreting an interaction, so we click on the Plots button.

20. 11/19/2015Slide 20 The dependent variable will be plotted on the vertical axis. We choose which of the factors is plotted on the horizontal axis and which factor is represented by different colored lines. We will plot the first named factor as separate lines and the second factor on the horizontal axis. Move freeReli to the Horizontal Axis text box. Move freeMove to the Separate Lines text box. After specifying the variables for the plot, click on the Add button, to include the plot in the list of Plots. If you do not Add the plot, it will not be drawn.

21. 11/19/2015Slide 21 Click on the Continue button to close the dialog box. We could add the plot with the role of the variables reversed if we thought that would be helpful.

22. 11/19/2015Slide 22 One part of the output we need is only available through syntax. To create and edit the syntax, click on the Paste button.

23. 11/19/2015Slide 23 When we click on the Paste button, the syntax for the command is copied to the syntax editor which is then opened. I have highlighted the line we need to edit by enclosing it in a red box.

24. 11/19/2015Slide 24 We need to add the post hoc tests for the interaction term. We add the phrase: COMPARE (freeReli) ADJ(BONFERRONI) to the end of the line. This will compute post hocs for the freeReli, within the categories of the freeMove. Note: we enter the name of the second factor in parentheses after the COMPARE. SPSS would also compute the post hocs for freeMove, within categories of freeReli.

25. 11/19/2015Slide 25 To run the command from the syntax editor, highlight all of the text and click on the Run Current button.

26. 11/19/2015Slide 26 The Univariate procedure produces a lot of output. The Between-Subjects Factors table shows the coding for each factor, and the number of cases in each category. The Descriptive Statistics table contains the means, standard deviations, and counts for each combination of factors. The Levene Test evaluates the assumption of homogeneity of variance.

27. 11/19/2015Slide 27 The Tests of Between-Subjects Effects table contains the f-tests for the main effect and the interaction effect. Estimated Marginal Means are the un-weighted mean that are actually tested for differences in the analysis of variances. They are not necessarily the same as the weighted means reported in the table of descriptive statistics.

28. 11/19/2015Slide 28 For example, the total weighted mean for all 135 cases in the descriptives table is 34.68. The un-weighted estimated marginal mean is 37.725 (the average of the four group means: 35.06, 49.98, 34.68, 31.17).

29. 11/19/2015Slide 29 These tables are used to test and interpret the main effect for freedom of movement. The Univariate Tests table provides the statistical evidence for the significance test of the mean. We would use the Pairwise Comparisons table for post hoc tests, but since the factors in our problems have only two categories, the difference tested by the Anova is identical to the single pairwise comparison. The Estimates table contains the un-weighted means that are compared in the test of the main effect. (See the calculation below.) 45.524 is the average of 35.06 and 49.98 in the table of Descriptive Statistics. 32.925 is the average of 34.68 and 31.17 in the table of Descriptive Statistics.

30. 11/19/2015Slide 30 These tables are used to test and interpret the main effect for freedom of religion. The Univariate Tests table provides the statistical evidence for the significance test of the mean. The Estimates table contains the un-weighted means that are compared in the test of the main effect. (See the calculation below.) We would use the Pairwise Comparisons table for post hoc tests, but since the factors in our problems have only two categories, the difference tested by the Anova is identical to the single pairwise comparison. 34.872 is the average of 35.06 and 34.68 in the table of Descriptive Statistics. 40.577 is the average of 49.98 and 31.17 in the table of Descriptive Statistics.

31. 11/19/2015Slide 31 These tables are used to test and interpret the interaction effects. The Univariate Tests table provides the statistical evidence for the significance test of the means for each of the variables included in the interaction. The Estimates table contains the un-weighted means that are compared in the test of the main effect. Since they are not a combination of the divisions of other cells, they are identical to the weighted means.

32. 11/19/2015Slide 32 The Profile Plot helps interpret the interaction. When the lines slope in different directions, it indicates the presence of an interaction (though we rely on the ANOVA table to determine its statistical significance). In this plot, the blue line for the Restricted group shows that there is a much higher mean for the unrestricted religion category, but a lower mean for the unrestricted religion category on the green line representing the unrestricted travel group.

33. 11/19/2015Slide 33 We transfer the degrees of freedom from the table of Levene’s Test of Equality of Error Variances to the problem narrative. The question preceding the production of the output asked about the assumption of homogeneity of variance. We proceed with answering this question.

34. 11/19/2015Slide 34 The F statistic (0.09) and the Sig. value (0.964) are transferred to the problem narrative.

35. 11/19/2015Slide 35 In this problem, the interpretation of equal variance is supported by the Levene statistic of 0.09 with a probability of p =.964, greater than the alpha of p =.05. The null hypothesis is not rejected. The assumption of equal variance is supported, and we find no significant violation. The uniformity of the variance of the dependent variable across groups defined by the independent variable is evaluated with the Levene Test of Equality of Error Variances. The Levene statistic tests the null hypothesis that the variances for all of the groups are equal. When the probability of Levene statistic is less than or equal to alpha, we reject the null hypothesis, supporting a finding that the variances of one or more groups is different and we do not satisfy the assumption of equal variances.

36. 11/19/2015Slide 36 The first sentence of the third paragraph asks about the significance of the interaction of the two factors.

37. 11/19/2015Slide 37

38. 11/19/2015Slide 38

39. 11/19/2015Slide 39 When the p-value for the F-test is less than or equal to alpha, we reject the null hypothesis that the means of the populations represented by the groups formed by all combinations of the factors in the sample were all equal, and we interpret the results of the test. If the p-value is greater than alpha, we fail to reject the null hypothesis and do not interpret the result. The p-value for the ANOVA test (p =.018) was less than or equal to the alpha level of significance (.05) supporting the conclusion to reject the null hypothesis. At least one of the means of the populations represented by the combinations of factors in the sample was different from the other means. the ANOVA test was statistically significant.

40. 11/19/2015Slide 40 The next sentence asks about the effect size. For these problems, we will use partial eta-squared (ηp²) as the measure of effect, because this is what SPSS computes. We transfer the effect measure from the table to the problem narrative. Eta-squared (η²) is computed as the sum of squares for the effect divided by the total sum of squares, i.e. 2047.650 ÷ 50578.401 =.040. Partial eta- squared (ηp²) is computed as the sum of squares for the effect divided by the sum of squares of the effect plus the sum of squares for the errors, i.e. 2047.650 ÷ (2047.650 + 46776.017) = 0.042. Partial eta squared is equivalent to the R² obtained in a regression analysis.

41. 11/19/2015Slide 41 Comparing the computed value for ηp² to the interpretative criteria in note 3, we find that the effect is small.

42. 11/19/2015Slide 42 The remainder of the paragraph is devoted to the interpretation of the interaction effect. The next sentence compare the averages for the categories of religious restrictions within the factor category that restricts travel. First, we will enter the means for the two groups of religious restrictions.

43. 11/19/2015Slide 43 Within the group of countries that restrict travel, countries that restrict religious practices had a estimated marginal mean of 35.06. Within the group of countries that restrict travel, countries that don’t restrict religious practices had a estimated marginal mean of 49.98.

44. 11/19/2015Slide 44 The mean of 35.06 was lower than the mean of 49.98.

45. 11/19/2015Slide 45 The next sentence asks whether or not the difference in means was statistically significant.

46. 11/19/2015Slide 46 First, we enter the degrees of freedom for the tests within the restricted travel group from the table of Univariate Tests. Second, we enter the f-statistic and probability for the tests within the restricted travel group from the table of Univariate Tests.

47. 11/19/2015Slide 47 The p-value for the ANOVA test (p =.015) was less than or equal to the alpha level of significance (.05) supporting the conclusion to reject the null hypothesis. The categories of religious restrictions do not have the same mean in the population. The univariate ANOVA test was statistically significant.

48. 11/19/2015Slide 48 Since the statistical test was significant, we interpret the effect statistic.

49. 11/19/2015Slide 49 Transfer the value of partial eta-squared to the problem narrative. We interpret the effect size as small based on the criteria in Note 3.

50. 11/19/2015Slide 50 The next sentence compare the averages for the categories of religious restrictions within the factor category that doesn’t restrict travel. First, we will enter the means for the two groups of religious restrictions.

51. 11/19/2015Slide 51 Within the group of countries that don’t restrict travel, countries that restrict religious practices had a estimated marginal mean of 34.68. Within the group of countries that don’t restrict travel, countries that don’t restrict religious practices had a estimated marginal mean of 31.17.

52. 11/19/2015Slide 52 The mean of 34.68 was higher than the mean of 31.17.

53. 11/19/2015Slide 53 The next sentence asks whether or not the difference in means was statistically significant.

54. 11/19/2015Slide 54 First, we enter the degrees of freedom for the tests within the restricted travel group from the table of Univariate Tests.

55. 11/19/2015Slide 55 Second, we enter the f-statistic and probability for the tests within the restricted travel group from the table of Univariate Tests.

56. 11/19/2015Slide 56 The p-value for the ANOVA test (p =.464) was greater than the alpha level of significance (.05) supporting the conclusion to fail to reject the null hypothesis. The categories of religious restrictions do not have different means in the population. The univariate ANOVA test was not statistically significant.

57. 11/19/2015Slide 57 When the difference in means is not statistically significant, we do not interpret the relationship and we enter na for the effect size blanks.

58. 11/19/2015Slide 58 When the interaction is statistically significant, we cannot make a direct statement about the main effects because the main effects do not follow the same pattern. We therefore do not interpret the main effects and enter na for all remaining responses.

59. 11/19/2015Slide 59 When we submit the problem for grading, the green shading on the answers indicates that they are all correct.

60. Interpreting main effects

61. 11/19/2015Slide 61 This problem demonstrates how to answer a problem when the interaction is not significant, but one or both main effects is statistically significant. The answers to the question in the second paragraph on data screening is the same as the previous problem that did have a significant interaction.

62. 11/19/2015Slide 62 The first sentence of the third paragraph asks about the significance of the interaction of the two factors.

63. 11/19/2015Slide 63 We transfer the degrees of freedom from the ANOVA table to the problem narrative.

64. 11/19/2015Slide 64 We transfer the f statistic and the sig value from the ANOVA table to the problem narrative.

65. 11/19/2015Slide 65 The p-value for the ANOVA test (p =.091) was greater than the alpha level of significance (.05). We cannot reject the null hypothesis. The ANOVA test of the interaction was not statistically significant. We do not interpret the interaction, and we do test the main effects.

66. 11/19/2015Slide 66 When the interaction is not statistically significant, all of the blanks in the remainder of the paragraph are na.

67. 11/19/2015Slide 67 The first sentence in paragraph four asks about the main effect for the first factor on movement or travel restrictions.

68. 11/19/2015Slide 68 We transfer the degrees of freedom, the F statistic, and the p-value from the ANOVA table to the problem narrative.

69. 11/19/2015Slide 69 The p-value for the ANOVA test (p <.001) was less than or equal to the alpha level of significance (.05) supporting the conclusion to reject the null hypothesis. At least one of the means of the populations defined by freedom of movement in the sample was different from the other means. The ANOVA test was statistically significant.

70. 11/19/2015Slide 70 The next sentence compare the averages for the categories of travel restrictions. First, we will enter the means for the two groups of religious restrictions.

71. 11/19/2015Slide 71 Countries that restrict travel had a estimated marginal mean of 43.56. Countries that didn’t restrict travel had a estimated marginal mean of 69.38.

72. 11/19/2015Slide 72 The mean of 43.56 was lower than the mean of 69.38.

73. 11/19/2015Slide 73 The final sentence of the paragraph asks about the effect size of the relationship.

74. 11/19/2015Slide 74 We transfer the effect measure from the table to the problem narrative.

75. 11/19/2015Slide 75 Comparing the computed value for ηp² to the interpretative criteria in note 3, we find that the effect of 0.12 is moderately large.

76. 11/19/2015Slide 76 The first sentence in paragraph five asks about the main effect for the second factor on religious restrictions.

77. 11/19/2015Slide 77 We transfer the degrees of freedom, the F statistic, and the p-value from the ANOVA table to the problem narrative.

78. 11/19/2015Slide 78 The p-value for the ANOVA test (p = 0.365) was greater than the alpha level of significance (.05). We cannot reject the null hypothesis. The ANOVA test was not statistically significant.

79. 11/19/2015Slide 79 When the test of the relationship is not statistically significant, the answers to all of the remaining questions in the paragraph is na.

80. 11/19/2015Slide 80 When we submit the problem for grading, the green shading on the answers indicates that they are all correct.