1. Outline 1. Definition of Complex Designs 2. Some important terms
3. Advantages of complex designs
Testing theories
Resolving contradictions
Establishing the external validity of a result
4. Analysis in the presence of an interaction
5. Analysis when there is no interaction
6. Natural Groups designs
7. Ceiling effects

2. Definition of Complex Design
A complex design is one in which more than one variable is manipulated at the same time.
‘Complex’ here does not mean ‘difficult to understand.’

3. Some important terms Factorial design
The most useful kind of complex design is the factorial experiment, in which each variable is manipulated at all levels of each other variable.

4. The basic 2 X 2 factorial design
A1B1
A1B2
A2B1
A2B2 A1 B1 B2 A2
The basic 2 X 2 factorial design

5. The basic 2 X 2 factorial design
Motor - Short
Abstract - Short
Motor - Long
Abstract - Long
Training duration
Short Long
Motor
Abstract
Task
The basic 2 X 2 factorial design

6. Some Important Terms Factorial design Main effect
The effect of one variable in a multi-variable design, ignoring all other variables

7. The basic 2 X 2 factorial design
A1B1
A1B2
A2B1
A2B2 A1 B1 B2 A2 B1 B2
Comparing these two means gives us the main effect of B A1 A2
Comparing these two means gives us the main effect of A

8. Some Important Terms Factorial design Main effect Simple main effect
The effect of one variable in a multi-variable design, observed at one level of a second variable.

9. A1 A2 B1
A1B1
A2B1 B2
A1B2
A2B2
Here, A1B1 – A1B2 gives the SME of B at A1
When we ask about the effect of A, we’re asking:
Is there a difference in performance quality when subjects operate under condition A1 and when they operate under condition A2?
SME = simple main effect

10. A1 A2 B1
A1B1
A2B1 B2
A1B2
A2B2
Here, A2B1 – A2B2 gives the SME of B at A2
When we ask about the effect of A, we’re asking:
Is there a difference in performance quality when subjects operate under condition A1 and when they operate under condition A2?
SME = simple main effect

11. A1 A2 B1 B2 Here, A1B1 - A2B1 gives the SME of A at B1 A1B1 A2B1 A1B2
When we ask about the effect of A, we’re asking:
Is there a difference in performance quality when subjects operate under condition A1 and when they operate under condition A2?
SME = simple main effect

12. A1 A2 B1 B2 A1B1 A2B1 Here, A1B2 – A2B2 gives the SME of A at B2 A1B2
When we ask about the effect of A, we’re asking:
Is there a difference in performance quality when subjects operate under condition A1 and when they operate under condition A2?
SME = simple main effect

13. Some important terms Factorial design Main effect Simple main effect
Interaction
an interaction occurs when the effect of one variable varies at levels of another variable.
thus, when there is an interaction between A and B, the SME of A will vary across levels of B (and vice versa).
Vice-versa: the SME of B will vary across levels of A

14. 400
425
500
575 A2 B1 B2 A1 25 75
These numbers show observations on some dimension (such as reaction time in milliseconds)
SME of B at A1
SME of B at A2
Note – table shows arbitrary units of measurement
The SME of B is much smaller for A1 than for A2 – that’s an interaction of variables A and B

15. No Coffee Cereal Coffee No Cereal 100 60 50 40 10
SME of Coffee is larger with Cereal than without
SME of Cereal with
Coffee
Note – table shows arbitrary units of measurement
SME of Cereal
Without Coffee
The SME of Cereal is larger with Coffee than without.
DV = a measure of mood quality

16. Interaction – an example
Godden & Baddeley (1975)
Wanted to test context-dependent learning hypothesis
Divers learned a list of words, then recalled the list.
Each step could be either on land or under the water.
Context-dependent learning hypothesis: You remember something better if learning context is re-instantiated at test.

17. Learning On deck Recall In pool
13.5
8.4
8.6
11.4
On deck
In pool
Recall
Learning
DV = # words recalled out of 15
Is it better to learn on deck or in the pool? It depends upon whether you will have to recall on deck or in the pool.
Godden & Baddeley (1975)

18. Some important terms Factorial design Main effect Simple main effect
Interaction
Analytical comparisons
Tests that determine what is producing a main effect
E.g., is B1 different from B2? Is it different from B3?

19. Some important terms Factorial designs Main effect Simple main effect
Interaction
Analytical comparisons
Simple comparisons
tests that determine what is producing a simple main effect
E.g., is B1 different from B2 at level A1? Is B2 different from B3 at A2?

20. Some important terms Analytical comparisons:
Tests that determine what is producing a main effect
Simple comparisons:
tests that determine what is producing a simple main effect

21. Advantages of complex designs
Testing theories
Complex Designs allow tests that are:
more powerful
more economical, and
less likely to be correct by chance
Most human behavior is caused by several variables at once.

22. Advantage: Testing theories
More powerful
Variability in your data is either random (E) or associated with a systematic source (T)
In a factorial design, associating some variance with the interaction reduces the random error.
Total variability, S2, is divided into two parts: systematic and random error variability. If you use a factorial design, there are three systematic sources of variance: main effect of A, main effect of B, and the interaction of A and B. Since S2 is fixed in any given study, if some variability is associated with the interaction, then the residual error (the random error variability remaining after you remove all the systematic variability) must be smaller.
A systematic source

23. Advantage: Testing theories
More powerful
More economical
Better use made of subjects’ time – test several hypotheses at once.

24. Advantage: Testing theories
More powerful
More economical
Less likely to be correct by chance
More complex predictions are less likely to be correct by chance, since there are more ways they can go wrong.
Recall Stanovich’s example of the person knocking on the door: probability of correctly guessing goes down as prediction gets more detailed.

25. Advantages of complex designs
Testing theories
More powerful
More economical
Less likely to be correct by chance
Resolving contradictions
Recall Stanovich’s example of the person knocking on the door: probability of correctly guessing goes down as prediction gets more detailed.

26. Advantages of complex designs
Testing theories
Resolving contradictions
Results from different labs sometimes conflict because different researchers unwittingly choose different levels of variables they are not manipulating.
If those variables can be identified, they can be manipulated in a new study with a factorial design.

27. Arousal High 40 Difficulty 50 Low80 60 50
High
Low
Difficulty
Arousal
DV = accuracy (% correct)
If one lab used a difficult task and another used an easy task, researchers would draw opposite conclusions about the effect of arousal.

28. Advantages of Complex Designs
Testing theories
Resolving contradictions
Establishing external validity of a result
When no interaction is found, it’s safer to generalize effects of each variable across levels of the other variable.
But don’t generalize the effect of A beyond the levels of B used in the experiment.

29. Advantages of complex designs
Don’t generalize effect of A beyond levels of B.
E.g., if A = stimulus quality and B = stimulus size
Levels of B = 2, 4 and 10 cm in our experiment
We find no interaction
We can generalize the effect of A to 7 cm stimuli, but not to 20 cm stimuli.

30.
Clear
Degraded
We don’t know what’s going on in this region – so we shouldn’t say anything about it 7 20

31. Analysis when interaction occurs
Once we detect an interaction, the next step is to ‘decompose’ the interaction.
That is, compare SMEs of A at levels of B (or vice versa).
Which SMEs we examine should be dictated by theory.
To understand differences between SMEs, use simple comparisons.

32. Analysis when no interaction occurs
When a variable A does not interact with other variables in the design, you analyze the main effects of A.
As before, use simple comparisons to test for differences between pairs of means for levels of A.
Again, simple comparisons are not necessary if A only has two levels.

33. Does A interact with B? No
Main effect of A?
Finished
Yes
Simple comparisons
SME of A at B1?
SME of A at B2?
More than 2 means?
Main effect of B?

34. Complex design example
Pratkanis et al. (JPSP 1988)
The ‘sleeper effect’
The passage of time improves the effect of a persuasive message
This occurs only if message is accompanied by a discounting cue – a cue that causes you to distrust the persuasive message

35. Pratkanis et al. (1988) Persuasive message:
“Dr. Smith’s research shows that orange juice consumption can reduce cholesterol.”
Discounting cue:
“This research was funded by Tropicana.”

36. Pratkanis et al. (1988) Why does sleeper effect occur?
One model: it’s caused by dissociation – over time, link in memory between persuasive message and discounting cue gets weaker.
Pratkanis et al. tested this idea

37. Pratkanis et al. (1988) Basic paradigm:
People are given a persuasive message about an object or product + a discounting cue
Later, they are asked to rate the object or product

38. Pratkanis et al. (1988)
Pratkanis et al. used two independent variables
Delay
Was opinion rating given immediately or six weeks later?

39. Pratkanis et al. (1988)
Pratkanis et al. used two independent variables
Delay
Order
Was discounting cue presented before or after persuasive message during original session?

40. Pratkanis et al. (1988)
This is the sleeper effect – found when we look at only the variable delay
Message is rated more persuasive (higher score) after delay of 6 weeks
0 6 wks 15 10 5 -5

41. Pratkanis et al. (1988)
There’s no main effect of the variable order (discounting cue given before or after persuasive message during original session)
Before After 15 10 5 -5

42. Pratkanis et al. (1988)
This interaction shows that we get the sleeper effect only when the cue is presented after the persuasive message
Dissociation model can’t explain this
0 6 wks 15 10 5 -5
cue before message
cue after message
Dissociation should occur at same rate regardless of when cue is presented (before or after persuasive message during original session).

43. Pratkanis et al. (1988)
The design of this experiment allowed Pratkanis et al. to test the interaction hypothesis
The interaction observed – sleeper effect occurred only when discounting cue came after persuasive message – is strong evidence against the dissociation theory of the sleeper effect.

44. Natural groups designs
Designs in which experimenter does not assign subjects to groups
Groups are naturally occurring
It is very risky to draw conclusions about why such groups differ in performance on some task.

45. Natural groups designs
For example: people who are mentally active into their later years are less likely than people who are not mentally active to suffer Alzheimer’s Type Dementia (ATD).
Why?
Having a healthy brain makes you active?
Being active gives you a healthy brain?
Of course, there could be some third variable which is causing variation in mental activity and in brain health, which would explain their correlation.

46. Natural groups designs
A natural groups design is really a correlational study, not an experiment!
Thus, in the ATD case, severity of the disease is correlated with mental activity.
Dividing the subjects into two groups (With and Without ATD) doesn’t change this.
But you can still make an argument for cause…

47. Natural groups designs
Halpern & Bower (1982)
Studied memory for musical notation
People with musical training recall notation better than people without musical training.
Is this because of the training?
Or are people with better memories drawn to musical training?

48. Halpern & Bower example
Theory: musical training gives musicians the ability to “chunk” notation.
A chunk is a unit formed from several smaller pieces, on the basis of knowledge.
Examples of “chunks:”
BMW
CBC
IBM
NHL
SOA
ISI
JND

49. Halpern & Bower example
Halpern & Bower compared natural groups: people with and without musical training
used two sets of musical notation:
one with structure (so notation stimuli could be chunked)
one without structure

50. Halpern & Bower example
Note that this design allows us to test the prediction of an interaction:
Group by structure %
Structured
Unstructured
Musicians
Non-musicians

51. Halpern & Bower example
Result: Musicians’ recall superiority was greater for musical notation stimuli that had structure (so could be chunked).
Conclusion: musical training gave musicians better memory.

52. Halpern & Bower example
Reasoning: other accounts don’t explain the importance of structure in producing the musicians’ advantage.
Caveat: This is a sensible argument – but it is just an argument. H & B can invite us to share their conclusion, but we don’t have to.

53. CAUTION! Ceiling & Floor Effects
Some interactions are spurious. They can be produced by “ceiling” or “floor” effects.
When performance reaches a theoretical maximum (e.g., 100%) or minimum (e.g., 0%) at one level of one treatment condition, subjects cannot get any better (or worse) at other levels of that condition.

54. A1 A A3
100 B1 B2
Why do these lines have different slopes? We cannot say. Might be a real interaction of A and B. Might be a ceiling effect.

55. CAUTION! Ceiling & Floor Effects
An interaction produced by running up against a ceiling or floor cannot be interpreted.
Only solution is to run the study again, trying to eliminate the ceiling or floor effect (e.g., make the stimuli harder to perceive).

56. Complex Designs – Review
A complex design is one in which more than one variable is manipulated at the same time.
In factorial designs, each IV is manipulated at all levels of the other IVs.
A significant F is followed by tests of simple main effects and simple comparisons

57. Complex Designs – Review
Complex designs allow us to:
Test theories, using precise hypotheses.
Explain contradictory findings across labs.
Establish external validity (or its limits).

58. Complex Designs – Review
Interactions help us:
Decide whether a variable is relevant to investigation of some topic.
Test theories about why natural groups differ in performance on some task.