1. Department of Cognitive Science
Inferential Statistics II: Introduction to ANOVA
Michael J. Kalsher
Department of Cognitive Science
MGMT PSYCHOMETRICS © 2012, Michael Kalsher

2. ANOVA: A Framework Understand the basic principles of ANOVA
Why it is done?
What it tells us?
Theory of one-way independent ANOVA
Following up an ANOVA:
Planned Contrasts/Comparisons
Choosing Contrasts
Coding Contrasts
Post Hoc Tests
Writing up Results

3. Why ANOVA?
t tests are limited to situations in which there are only two levels of a single independent variable or two associated groups.
There are many instances in which we’d like to compare more than two levels. But … performing multiple t tests can inflate the Type I error rate.
The concept of “Omnibus test.”
Familywise error: 1 - (.95)n

4. What is ANOVA
Tests the null hypothesis that three or more means are roughly equal.
Produces a test statistic termed the F-ratio.
Systematic Variance (SSM)
Unsystematic Variance (SSR)
The F-ratio tells us only that the experimental manipulation has had an effect—not where the effect has occurred.
-- Planned comparisons (before you collect the data)
-- Post-hoc tests (after-the-fact “snooping”).
-- Both conducted to control the overall Type I error rate at 5%.
-- Guards against familywise error [1 - (.95)n ]
-- For a study with 3 conditions, Type 1 error rate = 14.3%!

5. ANOVA as Regression
Suppose we are interested in determining whether the drug Viagra is effective at making someone a better lover. A group of men are then randomly assigned to one of three groups: (1) Placebo (sugar pills); (2) Low-dose Viagra; or (3) High-dose Viagra. The dependent measure is an “objective” measure of libido. The data are below:

6. Outcomei = (model) + errori
ANOVA as Regression
To predict levels of libido from the different levels of Viagra, we can use the general equation:
Outcomei = (model) + errori

7. Model for the Placebo group (base category)
We are predicting the level of libido when both doses of Viagra are ignored. Thus, the predicted value will be the mean of the placebo group.
The intercept, b0, is always equal to the mean of the base category

8. Model for the High-dose group
b2 represents the difference between the means of the High-dose group and the Placebo (base) group.

9. Model for the Low-dose group
b1 represents the difference between the means of the Low-dose group and the Placebo (base) group.

10. Regression Output
Conclusion: Using group means to predict the outcome is significantly better than using the overall mean.
b1 = Low Dose
b2 = High Dose
Note:
b2 = the difference between the means of the high-dose group and the placebo group ( = 2.8).
b1 = the difference between the means of the low-dose group and the placebo group ( = 1)

11. Theory of ANOVA: Partitioning Variance
We compare the amount of variability explained by the Model (MSM), to the error in the model [individual differences] (MSR)
This ratio is called the F-ratio.
If the model explains a lot more variability than it can’t explain, then the experimental manipulation has had a significant effect on the outcome (DV).

12. Partitioning the Variance
Total sum of squares (SST)
Model sum of squares (SSM)
Residual sum of squares (SSR)

13. SST = based on differences between each data point and the grand mean
SSM = differences between predicted values (group means) and the grand mean
Grand Mean = 3.467
XPlacebo = 2.20
XLowDose = 3.20
XHighDose = 5.00
SSR = based on differences between person’s score and their group mean.
Basis for SST
Basis for SSR
Basis for SSM
Partitioning the Variance
Base Category

14. Total sum of squares (SST)
Compute the difference between each data point and the grand mean, square these differences, then add them together. 1
Grand Mean = 3.467
Grand SD = 1.767
Grand Variance = 3.124 2 3 4

15. Model sum of squares (SSM)
Calculate the difference between the mean of each group and the grand mean.
Square each of these differences
Multiply each result by the number of participants within that group.
Add the values for each group together.

16. Residual sum of squares (SSR)
SSR = SST - SSM … But, don’t rely on this relationship (e.g., might not work out if either SST or SSM is miscalculated)
SSR is the difference between what the model predicts and what was actually observed. Therefore it is calculated by by looking at the difference between the score obtained by a person and the mean of the group to which the person belongs.

17. Total sum of squares (SSR)

18. Mean Squares (MSM and MSR)
SSM = amount of variation explained by the model (exp. manipulation).
SSR = amount of variation due to extraneous factors.
These are “summed” scores and will therefore be influenced by the number of scores. To eliminate this bias we calculate the average sum of squares (mean squares) by dividing by the appropriate degrees of freedom.
Calculating Degrees of Freedom (for one-way independent groups ANOVA)
dftotal = N - 1 (number of all scores minus 1)
dfM / between = k - 1 (number of groups minus 1)
df R / within = N - k (number of all scores minus number of groups)

19. The F-ratio
F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).
Therefore, we again return to the basic conceptual model:
Test Statistic = systematic variance
unsystematic variance

20. Assumptions
The one-way independent groups ANOVA test requires the following statistical assumptions:
Random and independent sampling.
Data are from normally distributed populations Note: This test is robust against violation of this assumption if n > 30 for both groups.
Variances in these populations are roughly equal (Homogeneity of variance) Note: This test is robust against violation of this assumption if all group sizes are equal.

21. 1-Way Independent-Groups ANOVA
Characteristics:
Used when testing 3 or more
experimental groups.
Each participant contributes
only one score to the data.
Used to test the (null)
hypothesis that several
means are equal.
Example: Testing the idea that exposure to mobile phones will lead to brain tumors.
IV: Hrs./day exposure
DV: Size of brain tumor
Mobile Phone Use
(hours per day)

22. Steps in the Analysis
-- Calculate the Model (or between-groups) sum of squares (SSM) and the Residual (or within-groups) sum of squares (SSR).
-- The number of scores used to calculate SSM and SSR are different, so to make them comparable, we convert them to the average sums of square or “mean squares” (MS) by dividing by their appropriate degrees of freedom.
-- F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).

23. Degrees of Freedom (df): One-way Independent-Groups ANOVA
Formulas for the number of degrees of freedom in one-way independent-groups ANOVA differ for each SS term. The degrees of freedom (df) reflect the number of values free to vary in any sum of squares term.
Calculating Degrees of Freedom
dftotal = N - 1 (number of all scores minus 1)
dfbetween = k - 1 (number of groups minus 1)
df within = N - k (number of all scores minus number of groups)

24. SPSS Summary Tables 95% Confidence Interval N Mean Std. Deviation
Std. Error
Lower
Bound
Upper
Min.
Max. 1 2 3 4 5
Total 20
120
.0175
.5149
1.2614
3.0216
4.8878
4.7306
2.4056
.01213
.28419
.49218
.76556
.69625
.78163
.00271
.06355
.11005
.17118
.15569
.17478
.18500
.0119
.3819
1.0310
2.6633
4.5619
4.3648
2.0393
.0232
.6479
1.4917
3.3799
5.2137
5.0964
2.7720
.00
.48
1.77
3.04
2.70
.04
.94
2.34
4.31
6.05
6.14
Size of Tumor
95% Confidence Interval
Hrs. of Exposure
Sum of
Squares df
Mean
Square F
Sig.
Between Groups
Within Groups
Total
38.094 5
114
119
90.133
.334
.000

25. Levene’s Test: Testing the Homogeneity of Variance Assumption
If Levene’s test is significant (p<.05), then the variances are significantly different.
What to do if Levene’s test is significant?
Transform the data
Report the F value and the Levene’s test results and allow your audience to assess the accuracy of your results for themselves.
Levene
Statistic df1 df2 Sig.

26. Computing Effect Size (2 - eta squared and 2 - omega squared)
Analogous to r2, eta-squared reflects the proportion of DV variance explained by each IV in the sample data It tends to be upwardly biased (i.e., an overestimate).
2 = SSModel / SStotal
One 2 for each effect (i.e., 1 per IV plus 1 per interaction) and the 2s sum to 1.
SPSS reports partial eta squared (p2 = SSEffect / SSEffect+ SSError)
In 1-way ANOVA, eta-squared and partial eta-squared are the same.
An alternative, omega-squared (2), estimates the proportion of DV variance explained by the IVs in the population (see Field, 2005, p ).

27. Computing Effect Size (2 vs. 2 omega squared)
2 = SSM = =
SST
2 = SSM - (dfM)(MSR) = (5 x .334)
SST + MSR
= =
489.09
(called eta squared, 2 )
(called omega squared, 2 )

28. Reporting the Results
Levene’s test indicated that the assumption of homogeneity of variance had been violated, F(5, 114) = 10.25, p< Transforming the data did not rectify this problem and so F tests are reported nonetheless. The results show that using a mobile phone significantly affected the size of brain tumor found in participants, F(5, 114) = , p<.001, 2 = .92. Tumor size (measured in mm) increased with the duration of exposure. Specifically, tumor size for 0, 1, 2, 3, 4 or 5 hours/day of exposure was .02 mm, .51 mm, 1.26 mm, 3.02 mm, 4.89 mm, and 4.73 mm, respectively. The effect size indicated that the effect of phone use on tumor size was substantial. Games-Howell post hoc tests revealed significant differences between all groups (p<.001 for all tests) except between 4- and 5-hours (ns).

29. Computing a One-way Independent-groups ANOVA by hand

32. Critical Values for F

33. Computing Effect Size (2 vs 2 Omega squared)
2 = SSM = = .69
SST
2 = SSM - (dfM x MSR) = (2 x 4.61)
MSR + SST
= =
139.28

34. Computing the One-way Independent-groups ANOVA using SPSS

35. SPSS Variable View: Assign Value Labels

36. SPSS Data View: Type in the Data

37. SPSS Data Editor: Select Procedure

38. Specify IV and DV, then select “options”
Select “Descriptives” and “Homogeneity test”. If the variances are unequal, choose “Brown-Forsythe” or “Welch” tests. 38

39. Choosing Follow-up tests: Contrasts vs. Post-Hoc
Select “Contrasts” if you have apriori hypotheses.
Select “Post-Hoc” if you do not. 39

40. Planned Contrasts (if we have specific hypotheses)
The first contrast compares the combined “experimental drug groups” against the control (Saltwater) group.
The second contrast compares the the two experimental groups, Fenfluramine vs. Amphetamine. 40

41. Post-Hoc Tests (if we do NOT have specific hypotheses)
LSD (Least Significant Difference) does not control Type I error rate. Bonferroni tightly controls Type I error rate. Other post-hoc procedures have specific applications (see Field text). 41

42. SPSS ANOVA Output: Descriptives and ANOVA Summary Table

43. SPSS ANOVA Output: Planned Contrasts
Levene’s test was not significant (p>.05) and so we can assume equal variances.
In the Contrast Tests summary, use the “Assume equal variances” row values.
Contrast 1: weight gain among rats receiving the experimental drugs is significantly different from rats receiving saltwater.
Contrast 2: weight gain among rats taking fenfluramine and amphetamine are not significantly different.

44. Post-hoc tests After the Fact “Snooping”: Some Considerations
LSD does not control Type I error rate, whereas the Bonferroni test tightly controls the Type I error rate.
Given equal sample sizes and equal variances, use REGWQ or Tukey HSD, as both have good power and tight control over Type I error rate.
If sample sizes across groups are slightly different, use Gabriel’s procedure because it has greater power, but if sample sizes are very different use Hochberg’s GT2.
If there is any doubt that the population variances are equal, use the Games-Howell procedure, in place of, or in addition to other tests. 44

45. Post-Hoc tests: LSD vs Bonferroni

46. Reporting the Results
The results show that the amount of weight gained by the rats was significantly affected by the type of drug they received, F(2,9) = 10.10, p < .01, 2 = Specifically, weight gain was greatest for rats receiving Saltwater (M=10.0, SE=.82) and least for rats receiving Fenfluramine (M=3.25, SE=1.11). Weight gain for rats receiving Amphetamine was intermediate (M=5.75, SE=1.25).
If you used planned comparisons …
Planned comparisons revealed a significant difference in weight gain between the combined experimental groups and the saltwater control group, t(9)=-4.18, p<.05 (1-tailed). The difference in weight gain between the two experimental groups receiving fenfluramine and amphetamine, respectively, was not significant, t(9)= -1.65, p>.05 (1-tailed).
If you used post-hoc tests …
Bonferroni post hoc tests revealed a significant difference in weight gain between the Fenfluramine and Saltwater groups (p<.05). No other comparisons were significant (ps>.05).

47. Sample Problem: Your turn…