1. Analysis of Complex Designs
Steps for Data Analysis:
Check the data for errors and outliers.
Summarize the results using descriptive statistics.
The factorial design tells you how many means you need to analyze.
2 x 2 design has
2 main effects each with 2 marginal means each
4 conditions, so 4 cell means
Graph the cell means “means for each condition”
Confirm what the data reveal.
Use inferential statistics to determine if means are different simply because of chance (error variation), or whether the independent variables caused the means to differ.

2. Table 8.4 Guidelines for Analysis of Two-Factor Experiment
Is the A X B interaction effect statistically significant?
No Yes
Are the main effects Are the simple main
significant? effects significant?
No Yes No Yes
Compare Compare
Stop two means Stop two means

3. Analysis of Complex Designs Effort to Obtain a Confession
Guidelines for the analysis of a complex design
First, determine whether the interaction effect between the independent variables is statistically significant. See fig 8.1 below
If so, identify the source of the interaction effect by examining the simple main effects
Pair-wise differences between the cell means “conditions” at each level of the IV

4. Analysis of Complex Designs Effort to Obtain a Confession
Guidelines for the analysis of a complex design
Simple main effect of Suspect Status
At Guilty expectation level 5.64 to is significant
At Innocent expectation level 5.56 to not significant
Simple main effect of Interrogator Expectation
For Actual Guilty level comparing to 5.56
For Actual Innocence level comparing to 5.85
Note: do not need to use both of these

5. Analysis Plan with No Interaction Effect
If the analysis indicates the interaction effect is not statistically significant. The next step is to determine whether the main effects of the variables are statistically significant
Significant main effects can be determined by comparing means for each level of the IV or by using confidence intervals to compare those means

6. Table 8.4 Guidelines for Analysis of Two-Factor Experiment
Is the A X B interaction effect statistically significant?
No Yes
Are the main effects Are the simple main
significant? effects significant?
No Yes No Yes
Compare Compare
Stop two means Stop two means

7. Analysis Plan with No Interaction Effect
Interrogator Expectation
Guilt (3.62) compared to Innocent (2.60)
This main effect of Interrogator Expectation maybe statistically significant
Suspect Status
Actual Guilt (3.04) compared to Actual Innocence (3.18)
This main effect of Suspect Status probably not statistically significant
Means for Interrogator Expectation:

8. Analysis Plan with an Interaction Effect
Kaiser 2006 social identity threat research using the emotional Stroop task
when words are presented subliminally has a significant interaction
Using a 2 x 3 design
Social identity (random groups design)
Threat: male partner with sexist views
Safety: male partner without sexist views
Word type (repeated measures design)
Social identity threat; sexist words
Illness threatening; disease words
Nonthreatening; household object words
Response time to identify the color of the word

9. Analysis Plan with an Interaction Effect
Kaiser 2006 social identity threat research using the emotional Stroop task
when words are presented subliminally there was a significant interaction between Independent Variables
In a 2 X 3 design the interaction can be at just one level of an I.V.
Because there is an interaction identify the source of the interaction effect by examining the simple main effects

10. Analysis Plan with an Interaction Effect
Guidelines for the analysis of a complex design
Compare simple main effects at each level of the I.V.
Do not need to use all possible simple main effects
Threat vs Safety at the Social-identity Threatening level vs 603.9
Threat vs Safety at the Illness Threatening level vs 615.0
Threat vs Safety at the Non-Threatening level vs 614.5
Social-identity vs illness vs non for the threatening level vs vs are significantly different
Follow up on significant differences by examining the simple main effect
When a simple main effect with three or more levels is statistically significant, conduct comparisons of means two at a time.
598.9 vs 577.7, vs 583.9, vs 583.9
Social-identity vs illness vs non for the Safety level vs vs are not significantly different
Do not need to follow up

11. Analysis Plan with No Interaction Effect
For example Kaiser when words can be processed consciously
Mean response times
No Interaction between I.V.
Analyze main effects
Social Identity
613.6 vs 631.4
Word type is significant
637.5 vs 617.6, 610.8
Follow up by comparing means
Two at a time

12. Interaction Effects and Theory Testing
Theory will predict interaction of variables
Need a complex design to test the theory
Kaiser et al. (2006) social identity theory
tested hypotheses about attention to prejudice cues in the environment
From prior research
when individuals’ social identity is threatened
they are consciously aware of cues in their environment relating to potential prejudice
extend this research
Do threatened individuals pay attention to prejudice cues nonconsciously, without awareness?
Complex designs can resolve differences between studies
Combine I.V. from contradictory studies in a complex design

13. Interaction Effects and External Validity
Interaction effect is not present
Generalize findings across conditions of experiment
Interaction effect is present
Sets limits on generalizing a finding
Conditions of experiment identify the limits

14. Interaction Effects and External Validity
The presence of a statistically significant interaction effect sets limits on the external validity of a finding.
For example, in the Kassin et al. (2003) study
Interaction of interrogator expectation and suspect status
Combination of
Guilty level of interrogator expectation
actual innocence level of the suspect status
produces the highest score on effort to obtain confessions
So we cannot generalize findings for interrogator expectation
It has an influence when there is actual innocence
It does not when there is actual guilt

15. Interaction Effects and External Validity
Because Kassin et al. (2003) observed a statistically significant interaction effect for Interrogator expectations and Suspect Status
Interrogator expectations depends on the suspect status of an individual
There are limits to external validity

16. Interaction Effects and External Validity
When an interaction effect is not statistically significant in an experiment, the researcher can generalize the findings across the conditions of the experiment.
To understand this, we can look at Kassin et al. (2003) Number of Guilt-Presumptive Questions variable
Main effect of interrogator expectation with guilty greater then innocent

17. Interaction Effects and External Validity
More guilt-presumptive questions for guilty compared to innocent
This was true for actual guilt and actual innocent therefore, the effect of suspect status generalized across the levels of interrogator expectations

18. Interaction Effects and External Validity
Floor and Ceiling Effects
Sometimes an interaction effect can be statistically significant “by mistake.”
This occurs when:
the means for one or more condition reach the highest possible score (ceiling effect)
the lowest possible score (floor effect).
When floor or ceiling effects occur, an interaction effect is uninterpretable.

19. Ceiling Effect: Example
This graphs shows an interaction effect between Test Difficulty (easy, hard) and Study Hours (10, 15).
Hours of study had an effect only in the hard-test condition, not in the easy-test condition.
How do we interpret this interaction effect when we know the highest possible score on the tests is 50?

20. Ceiling Effect
If we have enough “room” in our dependent variable to assess the effect of the independent variables, the interaction effect disappears.
This graph shows two main effects: A main effect of Study Hours and a main effect of Test Difficulty.

21. Interaction Effects and Natural Groups Designs
Using complex designs, researchers can test causal inferences for natural groups variables.
Recall that we can’t make causal inferences with natural groups variables.
Natural groups variables are correlational.
So, how can we make causal inferences using a complex design?

22. Interaction Effects and Natural Groups Designs
Steps for making causal inferences about natural groups variables in a complex design:
State your theory. Why are the groups different? What is the theoretical process?
Identify a relevant independent variable. This IV should influence the likelihood that the theorized process will occur (e.g., relationship maintenance).
Look for an interaction effect. In order to make a causal inference, the natural groups variable and manipulated variable should produce a statistically significant interaction effect in the predicted direction.
This interaction effect allows us to make causal inferences about why individuals differ — that is, we begin to understand why people differ.

23. Interaction Effects and Natural Groups Designs
We can make causal inferences about natural groups when we test a theory for why the natural groups differ.
For example, we can theorize that musicians and nonmusicians different in musical performance because of the way that these groups organize melodies
By manipulating type of musical structure
Test for interaction