1. Chapter Eleven: Designing, Conducting, Analyzing, and Interpreting Experiments with More Than Two Groups
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

2. Objectives Review project assignment
How do we design multiple group studies?
How do we analyze multiple group studies?
The logic of analysis of variance
Review exam
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

3. Experimental Design: Adding to the Basic Building Block
The two-group design is the basic building block.
Researchers typically want to move beyond two-group designs so they can ask more complicated and interesting questions.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

4. Example
What is the effect of background noise during study time on recall?
Two basic groups?
More than two?
Which is the control group?
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

5. What is the effect of delay on recall
Start with basic two group experiment?
What additional groups could you add?
What is the control group?
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

6. The Multiple-Group Design
How Many Groups?
This question marks the difference between the multiple-group design and the two-group design.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

7. The Multiple-Group Design
How Many Groups?
This question marks the difference between the multiple-group design and the two-group design.
A multiple-group design compares three or more levels or amounts of an IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

8. The Multiple-Group Design
How Many Groups?
This question marks the difference between the multiple-group design and the two-group design.
A multiple-group design compares three or more levels or amounts of an IV.
A multiple-group design can have a control group and two or more experimental groups.
We can compare three, four, five, or even more differing levels or amounts of an IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

9. The Multiple-Group Design
How Many Groups?
This question marks the difference between the multiple-group design and the two-group design.
A multiple-group design compares three or more levels or amounts of an IV.
A multiple-group design can have a control group and two or more experimental groups.
We can compare three, four, five, or even more differing levels or amounts of an IV.
A multiple-group design does not have to have a control group.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

10. The Multiple-Group Design
Assigning Participants to Groups
After we decide to conduct a multiple-group experiment, we must decide about assignment or research participants to groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

11. The Multiple-Group Design
Assigning Participants to Groups
After we decide to conduct a multiple-group experiment, we must decide about assignment or research participants to groups.
We may choose between independent groups or correlated groups.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

12. The Multiple-Group Design
Independent Groups
Groups of participants that are formed by random assignment.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

13. The Multiple-Group Design
Independent Groups
Groups of participants that are formed by random assignment.
Correlated Samples (Nonrandom Assignment to Groups)
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

14. The Multiple-Group Design
Independent Groups
Groups of participants that are formed by random assignment.
Correlated Samples (Nonrandom Assignment to Groups)
Matched sets
Participants are matched on a variable that will affect their performance on the DV (matching variable).
Then sets of participants are created who are essentially the same on the matching variable.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

15. The Multiple-Group Design
Independent Groups
Groups of participants that are formed by random assignment.
Correlated Samples (Nonrandom Assignment to Groups)
Matched sets
Natural sets
Analogous to using natural pairs except that sets must include more than two research participants.
Many animal researchers use littermates as natural sets.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

16. The Multiple-Group Design
Independent Groups
Groups of participants that are formed by random assignment.
Correlated Samples (Nonrandom Assignments to Groups)
Matched sets
Natural sets
Analogous to using natural pairs except that sets must include more than two research participants.
Many animal researchers use littermates as natural sets.
Repeated measures
Each participant must participate in all of the treatment conditions.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

17. Comparing Multiple-Group and Two-Group Designs
All you have to do to change your two-group design into a multiple-group design is add another level (or more) to your IV.
A two-group design can tell you whether your IV has an effect.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

18. Comparing Multiple-Group and Two-Group Designs
A two-group design can tell you whether your IV has an effect.
If you find no answer in a library search, then you should consider conducting a two-group (presence vs. absence) study.
A multiple-group design is appropriate when you find the answer to your basic question and wish to go further.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

19. Variations on the Multiple-Group Design
Comparing Different Amounts of an IV
If we already know that a particular IV has an effect, then we can use a multiple-group design to help us define the limits of that effect.
In this type of experiment, we often add an important control in order to account for a possible placebo effect.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

20. Variations on the Multiple-Group Design
Comparing Different Amounts of an IV
Dealing with measured IVs
Ex post facto research deals with measured rather than manipulated IVs.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

21. Analyzing Multiple-Group Designs
Multiple-Groups designs are measured with the analysis of variance (ANOVA).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

22. Analyzing Multiple-Group Designs
Multiple-Groups designs are measured with the analysis of variance (ANOVA).
The ANOVA procedure used to analyze a multiple-group design with one IV is known as a one-way ANOVA.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

23. Analyzing Multiple-Group Designs
Multiple-Groups designs are measured with the analysis of variance (ANOVA).
The ANOVA procedure used to analyze a multiple-group design with one IV is known as a one-way ANOVA.
A one-way ANOVA for independent groups is known as a completely randomized ANOVA.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

24. Analyzing Multiple-Group Designs
Multiple-Groups designs are measured with the analysis of variance (ANOVA).
The ANOVA procedure used to analyze a multiple-group design with one IV is known as a one-way ANOVA.
A one-way ANOVA for independent groups is known as a completely randomized ANOVA.
A one-way ANOVA for correlated groups is known as a repeated-measures ANOVA.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

25. ANOVA
The null hypothesis: The independent variable had no effect – any differences between treatment levels are due to chance
The alternative hypothesis: The independent variable effected at least one of the groups
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

26. Rationale of ANOVA Between-Groups Variability
Variability in DV scores that is due to the effects of the IV.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

27. Rationale of ANOVA Between-Groups Variability
Variability in DV scores that is due to the effects of the IV.
Error Variability (Within-Groups Variability)
Variability in DV scores that is due to factors other than the IV (individual differences, measurement error, and extraneous variation).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

28. Rationale of ANOVA
The notion that has evolved for the ANOVA is that we are comparing the ratio of between-groups variability to within-groups variability.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

29. Rationale of ANOVA
The notion that has evolved for the ANOVA is that we are comparing the ratio of between-groups variability to within-groups variability.
F = between-groups variability
within-groups variability
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

30. Rationale of ANOVA
When the IV has a significant effect on the DV, the F ratio will be large.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

31. Rationale of ANOVA
When the IV has a significant effect on the DV, the F ratio will be large.
When the IV has no effect or only a small effect, the F ratio will be small (near 1).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

32. Hypothetical experiment
Three groups – A, B, and C
Each with 5 participants
Do the groups differ significantly
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

33. Understanding within-group and between-group variance
Group B
Group C 1
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

34. Answers Group Means = 1, 1, and 1 Within Group Variance = 0
Between Group Variance = 0
F is not significant
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

35. Understanding within-group and between-group variance
Group B
Group C 1 3
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

36. Answers Group Means = 1, 1 and 3 Between Group Variance = 6.67
Within Group Variance = 0
F is infinitely large
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

37. Understanding within-group and between-group variance
Group B
Group C 1 3 2
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

38. Answers Between Group Variance = 6.67 Within Group Variance = .17
F = 40 significant
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

39. Understanding within-group and between-group variance
Group B
Group C 3 1 2
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

40. Answers Group Means = 1, 1, and 3 Between Group Variance = 6.67
Within Group Variance = .33
F significant
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

41. Understanding within-group and between-group variance
Group B
Group C 1 2 3 4 5
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

42. Answers Group Means = 1, 1, and 3 Between Group Variance = 6.67
Within Group Variance = 1.17
F significant = 5.71
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

43. Interpretation: Making Sense of Your Statistics
Adding a third group (or more) creates an interesting statistical problem for us so that we often need to compute an extra statistical test to explore significant findings.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

44. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Source table
A table that contains the results of ANOVA. Source refers to the source of the different types of variation.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

45. Hypothetical experiment
In the following study, participants were randomly assigned to receive humanistic therapy, psychodynamic therapy, or behavioral therapy for their depression. Six months after therapy ended, participants’ levels of depression were rated on a Likert scale from 1 (not at all depressed) to 15 (extremely depressed).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

46. Sample source table
GROUP N MEAN STD DEV STD ERROR
1 (humanistic)
2 (psychodynamic)
3 (behavioral)
ONE-WAY ANOVA: THERAPY by DEPRESSION
SOURCE SS DF MS F RATIO PROB.
BETWEEN GROUPS
WITHIN GROUPS
TOTAL
POST HOC TEST: Tukey-HSD with significance level .05
* Indicates significant differences shown for pairings
G G G
r r r
p p p
1 2 3
Mean THERAPY
5.40 Grp 1
7.50 Grp 2
5.60 Grp 3
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

47. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Source table
A table that contains the results of ANOVA. Source refers to the source of the different types of variation.
Sum of squares
The amount of variability in the DV attributable to each source.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

48. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Mean square
The “averaged” variability for each source = Same as variance
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

49. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Mean square
The “averaged” variability for each source.
The mean square is computed by dividing each source’s sum of squares by its degrees of freedom.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

50. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Mean square
F = mean square between groups
mean square within groups
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

51. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Mean square
Variance
A single number that represents the total amount of variation in a distribution.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

52. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Mean square
Variance
A single number that represents the total amount of variation in a distribution.
The square of the standard deviation.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

53. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
To discern where the significance lies in a multiple-group experiment, we must do additional statistical tests known as post hoc comparisons (also known as follow-up tests).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

54. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Post hoc comparisons
Statistical comparisons made between group means after finding a significant F ratio.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

55. Hypothetical Experiment
A researcher was interested in determining the effectiveness of humanistic therapy for alleviating depression. He administered a depression inventory to a sample of clients with depression on three occasions: before therapy started, two months after therapy began, and four months into therapy. Depression scores could range from 1 (not at all depressed) to 15 (extremely depressed).
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

56. Sample source table GROUP N MEAN STD DEV STD ERROR
1 (Time 1)
2 (Time 2)
3 (Time 3)
ONE-WAY ANOVA: DEPRESSION by TIME (CORR SAMP)
SUM OF MEAN
SOURCE SQUARES DF SQUARES F RATIO PROB.
TIME
SUBJECTS
WITHIN CELLS
TOTAL
POST HOC TEST: Tukey-HSD with significance level .05
* Indicates significant differences shown for pairings
G G G
r r r
p p p
1 2 3
Mean TIME
Grp * *
9.40 Grp 2
7.30 Grp 3
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

57. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Translating Statistics into Words (text example):
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

58. Interpreting Computer Statistical Output
One-way ANOVA for Independent Samples
Translating Statistics into Words (text example):
The effect of different clothing on salesclerks’ response time was significant F(2, 21) = 4.71, p = The proportion of variance accounted for by the clothing effect was Tukey tests indicated (p < .05) that clerks waiting on customers dressed in sloppy clothes (M = 63.25, SD = 11.73) responded more slowly than clerks waiting on customers in dressy (M = 48.38, SD = 9.46) or casual clothes (M = 48.88, SD = 9.55). The response times of clerks waiting on customers in dressy and casual clothes did not differ from each other.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education

59. Interpreting Computer Statistical Output
Translating Statistics into Words (text example):
The effect of three different clothing styles on clerks’ response times was significant, F(2, 14) = 19.71, p < The proportion of variance accounted for by the clothing effect was Tukey tests showed (p < .01) that clerks took longer to respond to customers dressed in sloppy clothes (M = 63.25, SD = 11.73) than to either customers in dressy clothes (M = 48.38, SD = 9.46) or customers in casual clothes (M = 48.88, SD = 9.55). Response times did not differ between the clerks waiting on customers in dressy or casual clothing.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education