1. Research methods and statistics
Tests of Difference
Research methods and statistics

2. Tests of Difference

3. Learning outcomes
At the end of these session and with additional reading you will be able to:
describe the assumptions of ANOVA, ANCOVA, MANOVA
describe main effects and interactions
interpret the F ratio
decide when to use post hoc or planned comparisons

4. Analysis of Variance (ANOVA)
ANOVA allows analysis on two or more independent variables
e.g. ethnicity and gender
ANOVA allows analysis on two or more conditions
e.g. Black, White Asian
MANOVA - Multivariate analysis of variance (ANOVA for several dependent variables)

5. ANCOVA Analysis of co-variance
The analysis of continuous (ratio) variables that predict the dependent variable
These variables are called covariates
The analysis of covariates reduce the within-group error variance
Elimination of confounding variables

6. Assumptions of ANOVA, MANOVA, ANCOVA
The dependent variable comprises of interval or ratio data
The dependent variable is normally distributed
The variance in each condition are fairly similar (Homogeneity of variance)
Observations should be independent (not applicable in repeated measures ANOVA’s)

7. How are ANOVA’s, MANOVA;s, ANCOVA’s described
Independent variables are known as factors
Conditions are know as levels
e.g. gender is a factors with two levels male/female

8. Familywise error Why not just perform numerous t tests ?
Every time you perform a test there is a 5% chance of the results occurring due to sampling error
If you perform numerous test on the same data you will increase the likelihood of error:
1-(.95)n, where n is the no of tests required
e.g. 3 tests on previous example of ethnicity
1-(.95)³ = =.143 or 14%

9. Interactions e.g. gender and ethnicity on performance
By analysisng two or more factors we are able to investigate the interaction between the factors on the DV
e.g. gender and ethnicity on performance
using the first example this would be described as a
2 x 3 ANOVA
(where 2 represents gender with 2 levels and 3 represents ethnicity with 3 levels)

10. Main effects and interaction
A one way ANOVA where there is only one factor would produce:
a main effect of A
A two way ANOVA where there are two factors would produce:
a main effect of B
a two interaction between A and B

11. What are ANOVA’s looking for
ANOVA’s are assessing whether the variance between a condition is larger than the variance within conditions. Only then can we suggest that a factor is having an effect on the DV.

12. The F ratio
The variance brought about by other nuisance factors such as individual differences is often referred to as the error variance
An ANOVA calculates the ratio of variance due to the manipulation of the IV and the error variance:
this is know as the F ratio

13. The F ratio II
If the error variance is small then the F ratio will be greater than 1
If the error variance is large then the F ratio will be less than 1

14. Post Hoc or planned comparisons
The F ratio tells us only whether there is a difference between means in a factor, but it does not tell us exactly where it lies
It is therefore necessary to conduct further analysis to find out which groups differ. However we must remember that we do not want to inflate the familywise error
This can be achieved in two ways
planned comparisons
post hoc comparisons

15. Post Hoc or planned comparisons
The difference between planned and post hoc comparisons are akin to one/two tailed hypothesis
Planned comparison should be used when you have specific hypothesis that you want to test
Post hoc comparisons should be performed when you have no specific hypothesis

16. Post hoc comparisons
Bonferroni and Tukeys both control the type 1 error rate very well
Bonferroni is best used when the number of comparisons are small
Tukey is best used when there are a large number of comparisons

17. Planned comparisons
Factors are broken down into smaller chunks of variance
e.g. high/low/placebo effect
contrast 1 will be between low and high against the placebo
contrast 2 will be low against high

18. Planned comparisons Comparisons must then be weighted
each chunk of variance must be given a weight
each chunk of variance must be positive or negative
the sum of weights should always add up to zero

19. Example Chunk 1 High/low Chunk 2 placebo contrast 1 1 2 weight
positive
negative
sign
high
low
placebo -2

20. Reporting results
F (df) = F value, P< or P > than .05