Statistical Analysis of Data1 of 38 1 of 42 Department of Cognitive Science Adv. Experimental Methods & Statistics PSYC 4310 / COGS 6310 MANOVA Multivariate Analysis of Variance Michael J. Kalsher PSYC 4310 Advanced Experimental Methods and Statistics © 2012, Michael Kalsher

Statistical Analysis of Data2 of 38 2 of 42 MANOVA: What is it? Used to determine main and interaction effects of categorical variables on multiple DVs. ANOVA tests the differences in means of a single DV for two or more categories of IVs. MANOVA tests whether the vectors of means for the two or more groups are sampled from the same sampling distribution. In other words, MANOVA gives a measure of the overall likelihood of picking two or more random vectors of means out of the same hat. Purposes for MANOVA To compare groups formed by categorical IVs on group differences in a set of interval-level DVs. To use lack of difference for a set of DVs as a criterion for reducing a set of IVs to a smaller, more easily modeled number of variables. To identify the IVs which differentiate a set of DVs the most.

Statistical Analysis of Data3 of 38 3 of 42 Why Use MANOVA? Advantages: Improves chances of discovering changes as a result of different variables and their interactions. Protects against inflated Type I error due to multiple tests of correlated dependent variables. Can detect whether groups differ along a combination of variables (i.e., a variate), whereas ANOVA can detect only if groups differ along a single variable. Disadvantages: More complicated and less powerful than ANOVA. Analysis often ambiguous in interpretation of the effects of IVs on any single DV.

Statistical Analysis of Data4 of 38 4 of 42 MANOVA: Parts of the Analysis Main Analysis Four commonly used ways of assessing the overall significance of a MANOVA: Pillai’s trace (V); Hotelling’s trace, Wilks’s lambda (  ), and Roy’s largest root. Debate over which method is best in terms of power and sample size considerations. Rules of thumb: If group differences are concentrated on the first variate, Roy’s statistic most powerful followed by Hotelling’s trace, Wilks’s lambda and Pillai’s trace. When groups differ along more than one variate, the power ordering is reversed. Approaches to Follow-up Analysis Perform separate ANOVAs for each DV (controlling for Type I error). Transform linear combinations of DVs to z scores, add them together, then evaluate the combined scores using ANOVA. Use discriminant function analysis (DFA) (yields one or more uncorrelated linear combinations of DVs that maximize differences among the groups)

Statistical Analysis of Data5 of 38 5 of 42 MANOVA test statistics: A Comparison

Statistical Analysis of Data6 of 38 6 of 42 Pillai-Bartlett trace (V) (Pillai’s trace) Given by the equation below in which represents the eigenvalues for each of the discriminant variates, and s represents the number of variates. Pillai’s trace is the sum of the proportion of explained variance on the discriminant functions and is similar to the ratio of SS M /SS T i 1 + i  i=1 s V =

Statistical Analysis of Data7 of 38 7 of 42 Hotelling’s trace The Hotelling-Lawley trace is the sum of the eigenvalues for each variate and is computed by the equation below. This test statistic is the sum of SS M /SS R for each of the variates and so it compares directly to the F ratio in ANOVA. i  i=1 s T =

Statistical Analysis of Data8 of 38 8 of 42 Wilks’s lambda (  ) Wilks’ lambda is the product of the unexplained variance on each of the variates. The  symbol is similar to the summation symbol (  ) except that it means multiply rather than add up. Wilks’s lambda represents the ratio of error variance to total variance (SS R /SS T ) for each variate.  i=1 s  = i

Statistical Analysis of Data9 of 38 9 of 42 Roy’s largest root Roy’s largest root is the eigenvalue for the first variate. In a sense, it is the same as the Hotelling-Lawley trace, except for the first variate only. This statistic represents the proportion of explained variance to unexplained variance (SS M /SS R ) for the first discriminant function. This value is conceptually the same as the F-ratio in univariate ANOVA and represents the maximum possible between-group difference given the data collected. Largest root = largest

Statistical Analysis of Data10 of of 42 MANOVA: Assumptions Independence: Observations should be statistically independent. Random Sampling: Data should be score level and randomly sampled from the population of interest. Multivariate Normality: In ANOVA, we assume that our DV is normally distributed within each group. In MANOVA, we assume that the DVs (collectively) have multivariate normality within groups (cannot be tested directly by SPSS). Homogeneity of Covariance Matrices: In ANOVA, it is assumed that the variances in each group are roughly equal (homogeneity of variance). In MANOVA, we assume this is true for each DV, but also that the correlation between any two DVs is the same in all groups. As a preliminary test, Levene’s test should not be significant for any of the DVs. Since Levene’s test doesn’t take account of the covariances, Box’s M test should be used to test whether the population variance-covariance matrices of the different groups in the analysis are equal Note: The F test from Box’s M statistics should be interpreted cautiously in that a significant result may be due to violation of the multivariate normality assumption for the Box’s M test, and a non-significant result may be due to a lack of power.

Statistical Analysis of Data11 of of 42 Effect Size Statistics for MANOVA The multivariate GLM procedure computes a multivariate effect size index. The multivariate effect size associated with Wilks’s lambda (  ) is the multivariate eta square. 1s1s Multivariate  2 = 1 -  Here, s is equal to the number of levels of the factor minus 1 or the number of DVs, whichever is smaller. This statistic should be interpreted similar to a univariate eta square and ranges in value from 0 to 1. A 0 represents no relationship between the factor and the DV, while a 1 indicates the strongest possible relationship. Unclear what should be considered a small, medium, and large effect size for this statistic.

Statistical Analysis of Data12 of of 42 Controversies and MANOVA Ability of MANOVA to detect true effects – Ramsey (1982): As correlation between DVs increases, power of MANOVA decreases. – Tabachnick & Fidell (2001): MANOVA works best with highly negatively correlated DVs and acceptably well with moderately correlated DVs in either direction; MANOVA wasteful when DVs are uncorrelated. – Cole et al. (1994): Power of MANOVA depends on combination of the correlation between DVs and effect size. Expecting large effect: MANOVA most powerful if the measures are somewhat different and if the group differences are in the same direction for each measure. If 2 DVs differ in terms of group differences (one large, one small), then power increased if DVs are highly correlated.

Statistical Analysis of Data13 of of 42 The Matrix: Revisited The MANOVA test statistic is derived by comparing the ratio of a matrix representing the systematic variance of all DVs to a matrix representing the unsystematic variance of all DVs Square Matrix Identity Matrix Diagonal components Off-diagonal components 2648 Row Vector Single person’s score on four different variables Column Vector Four participants’ score on one variable

Statistical Analysis of Data14 of of 42 Partitioning the Variance SS T = Sums of Squares Total SS m = Sums of Squares Model (Systematic Variance) SS R = Sums of Squares Error (Unexplained Variance) SS T SS M SS R MANOVA Test Statistic Sum of squares and cross-products matrices Systematic Variance HHypothesis sum of squares and cross-product matrix or hypothesis SSCP Unsystematic Variance EError sum of squares and cross-products matrix or error SSCP Total VarianceTTotal sum of squares and cross-products matrix or total SSCP Cross-product deviations represent a total value for the combined error between two variables, so in some sense, they represent an unstandardized estimate of the total correlation between two variables.

Statistical Analysis of Data15 of of 42 MANOVA: Performing MANOVA Using SPSS

Statistical Analysis of Data16 of of 42 Move “Social Dimension Summed Scale” and “Pragmatic Dimension Summed Scale” to the Dependent Variables box. Move “Gender (q047_r01)” to the Fixed Factors box. Then, click “Options”.

Statistical Analysis of Data17 of of 42 Move “Gender (q047_r01” to the Display Means box. Click “Descriptive statistics”, “Estimates of effect size”, and “Homogeneity tests” in the Display box. Click “Continue”

Statistical Analysis of Data18 of of 42

Statistical Analysis of Data19 of of 42 SPSS Output

Statistical Analysis of Data20 of of 42 SPSS Output: Main Analysis

Statistical Analysis of Data21 of of 42 SPSS Output: Univariate ANOVAs

Statistical Analysis of Data22 of of 42 Results Section A one-way MANOVA on the two dependent variables, the summed scales for the Social and Pragmatic dimensions, was significant for gender, Wilks’s  =.995, F(2,1340) = 3.289, p<.05. Table 1 contains the means and the standard deviations on the dependent variables for males and females. Analyses of variance (ANOVAs) on each dependent variable were conducted as follow-up tests to the MANOVA. The ANOVA on the Social Dimension summed scale scores was significant, F(1,1341) = 6.29, p.05.

Statistical Analysis of Data23 of of 42 Practice Problem: Effects of study strategies on learning A researcher investigates the effectiveness of different study strategies on learning. Thirty undergraduates are randomly assigned to one of three study conditions. All participants receive the same set of study questions, but each group receives different instructions about how to study. The write group is instructed to write responses to each question, the think group is instructed to think about answers to the questions, and the talk group is instructed to develop a talk that they could deliver centering on the answers to the questions. At the completion of the study session, all students take a quiz consisting of two types of questions: recall and application.

Statistical Analysis of Data24 of of 42 SubjectGroupApplicationRecall SubjectGroupApplicationRecall Data Set: Effects of study strategies on learning

Statistical Analysis of Data25 of of 42

Statistical Analysis of Data26 of of 42 Move “recall” and “applicat” to the Dependent Variables box. Move “Study Strategies” to the Fixed Factors box. Then, click “Options”.

Statistical Analysis of Data27 of of 42 Move “group” to the Display Means box. Click “Descriptive statistics”, “Estimates of effect size”, and “Homogeneity tests” in the Display box. Click “Continue”

Statistical Analysis of Data28 of of 42 Select “Post Hoc” from the Multivariate Screen Move “group” to the Post Hoc Test for box. Select “Bonferroni” Select “Games Howell” Click “Continue” Click “OK”

Statistical Analysis of Data29 of of 42 SPSS Output for MANOVA

Statistical Analysis of Data30 of of 42 SPSS Output for MANOVA

Statistical Analysis of Data31 of of 42 SPSS Output: Univariate ANOVAs

Statistical Analysis of Data32 of of 42 SPSS Output for Post-Hoc Tests

Statistical Analysis of Data33 of of 42 Results Section A one-way Multivariate analysis of variance (MANOVA) was conducted to determine the effect of the three types of study strategies (thinking, writing, and talking) on the two dependent variables, the recall and the application test scores. Significant differences were found among the three study strategies on the dependent measures, Wilks’s  =.42, F(4,52) = 7.03, p<.01. The multivariate  2 based on Wilks’s  was.35. Table 1 contains the means and the standard deviations on the dependent variables for the three groups.

Statistical Analysis of Data34 of of 42 Results Section - continued Univariate ANOVAs on each dependent variable were conducted as follow-up tests to the MANOVA. The ANOVA on the recall scores was significant, F(2,27) = 17.11, p<.01,  2 =.56, as was the ANOVA on the application scores, F(2,27) = 4.20, p =.026,  2 =.24. Post-hoc analyses to the univariate ANOVA for the recall and application scores consisted of conducting pair-wise comparisons to find which study strategy affected performance most strongly. With respect to the recall scores, the writing group produced significantly superior performance on the recall questions in comparison with either of the other two groups (ps.05). With respect to the application scores, the writing group produced significantly better performance on the application questions than the thinking group (p.05).

Statistical Analysis of Data35 of of 42 One More Step … Following up MANOVA with Discriminant Analysis The practice of conducting ANOVAs as follow-up tests to significant MANOVA has been criticized because univariate ANOVAs do not take into account the multivariate nature of MANOVA An alternative is to conduct follow-up analyses using discriminant function analysis (DFA). DFA can be used after MANOVA to see how the dependent variables discriminate the groups. DFA yields one or more uncorrelated linear combinations of dependent variables (variates) that maximize differences among the groups.

Statistical Analysis of Data36 of of 42 Discriminant Analysis: Step by Step

Statistical Analysis of Data37 of of 42 Discriminant Analysis: Step by Step Predictors

Statistical Analysis of Data38 of of 42 Discriminant Analysis: Step by Step Already in the MANOVA output Useful for gaining insight into the relationships between DVs for each group Produces the bs for each variate Finally, click on “Continue”, then “Classify” on the main dialog box for discriminant analysis

Statistical Analysis of Data39 of of 42 Discriminant Analysis: Step by Step Plots the variate scores for each participant groups according to the study strategy they were given Provides an overall gauge of how well the discriminant variates classify the actual participants

Statistical Analysis of Data40 of of 42 Discriminant Analysis: Step by Step Click on “Save” These scores can be useful because the variates that the analysis identifies may represent important underlying constructs

Statistical Analysis of Data41 of of 42 Discriminant Analysis Output Shows that only one of the variates is significant. Thus, the group differences shown by the MANOVA can be explained in terms of one underlying dimension

Statistical Analysis of Data42 of of 42 Discriminant Analysis Output These values are comparable to factor loadings. They represent the relative contribution of each DV to group separation. These coefficients tell us the relative contribution of each variable to the variates.

Statistical Analysis of Data43 of of 42 Results The MANOVA was followed up with discriminant analysis, which revealed two discriminant functions. The first explained 96.7% of the variance, canonical R 2 =.56, whereas the second explained only 3.3% of the variance, canonical R 2 =.04. In combination, these discriminant functions significantly differentiated the treatment groups, Λ = 0.42,  2 (4) = 22.90, p.05. The correlations between outcomes and the discriminant functions revealed that total correct on recall questions loaded almost exclusively on the first function (r =.99); total correct on application questions loaded more highly on the second function (r =.89) than on the first function (r =.47).