1. Experiments with More Than Two Groups
Ryan M. Denney, Ph.D.
The University of Southern Mississippi
PSY 361

2. Multiple Groups Design
A two-group design can tell you whether your IV has an effect.
A multiple-group design is appropriate when you find the answer to your basic question and wish to go further.
A multiple-group design compares three or more levels or amounts of an IV.
A multiple-group design can have a control group and two or more experimental groups (but does not have to have a control group).
We can compare three, four, five, or even more differing levels or amounts of an IV (when interested in impact of an IV)
Multiple group designs sometimes compare several groups at only ONE level of the IV (when interested in how groups differ)

3. Multiple Groups Design
Independent groups
Groups of participants that are formed by random assignment.
Random assignment serves as an important control procedure.
You must take into account the large number of participants you will need to make random assignment feasible and to fill the multiple groups.
Correlated samples (nonrandom assignment to groups)
Matched sets
Participants are matched on a variable that will affect their performance on the DV (matching variable).
Then sets of participants are created who are essentially the same on the matching variable.
You must consider the difficulty of finding three (or more) participants to match on the extraneous variable you choose.

4. Multiple Groups Design
Correlated samples (con’t)
Repeated measures
Each participant must participate in all of the treatment conditions.
That is, each participant must be measured at least three times.
Natural sets
Analogous to using natural pairs except that sets must include more than two research participants.
Many animal researchers use littermates as natural sets (siblings, people belonging to a social group).

5. Variations on Multiple Groups Design
Comparing different amounts/levels of an IV
If we already know that a particular IV has an effect, then we can use a multiple-group design to help us define the limits of that effect.
Placebo effect
An experimental effect that is due to expectation or suggestion rather than the IV.
Controlling for the “power of suggestion”
E.g., caffeine
Ex post Facto Research
Deals with “measured” rather than “manipulated” IV’s.

6. Analyzing Multiple Group Experiments
Multiple-groups designs are measured with the analysis of variance (ANOVA).
The ANOVA procedure used to analyze a multiple- group design with one IV is known as a one-way ANOVA. (2-way ANOVA is used to analyze differences in groups along 2 IVs)
A one-way ANOVA for independent groups is known as a completely randomized ANOVA.
A one-way ANOVA for correlated groups is known as a repeated-measures ANOVA.

7. ANOVA Between-groups variability
Variability in DV scores that is due to the effects of the IV.
Error variability (within-groups variability)
Variability in DV scores that is due to factors other than the IV (individual differences, measurement error, and extraneous variation).
An ANOVA compares the ratio of between-groups variability to within-groups variability.

8. ANOVA
Source Table: a table displaying the results of an ANOVA and showing the “source” of different types of variation
Sum of Squares
Sum of the squared deviations around the mean
Represents the amount of variability in the DV attributable to each source (between groups, within groups)
Mean Squares
The averaged variability of each source
Calculated by dividing each source’s sum of squares by its degrees of freedom: N-1 (number of groups or participants minus one)
Used to put sums of squares on equal footing—correcting for the fact that only a few groups can contribute to between-group variability and many participants can contribute to within-group variability

9. ANOVA Source Table Betwn grps 1141.74 2 570.88 4.71 .02
SOURCE SUM OF DF MEAN F PROB
SQUARES SQUARES
Betwn grps
Within grps
Total
N= 24
Groups= 3

10. ANOVA
When the IV has a significant effect on the DV, the F ratio will be large.
When the IV has no effect or only a small effect, the F ratio will be small (near 1).

11. ANOVA Possible Distributions of Variability in an Experiment.
A depicts a large F ratio.
B depicts an F ratio of 1.

12. ANOVA
To discern where the significance lies in a multiple-group experiment, we must do additional statistical tests known as post hoc comparisons (also known as follow-up tests).
Post hoc comparisons
Statistical comparisons made between group means after finding a significant F ratio.
Which groups differed significantly from each other?
Tukey’s HSD (Honestly Significant Difference): a common post hoc test used to perform pairwise comparisons of group means.