1. Chapter 9 Internal Validity
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

2. Overview Problems with two-group designs
Problems with pretest-posttest designs
Internal versus external validity
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

3. Two-Group Designs The selection problem
Unsuccessful approaches to the selection problem
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

4. Sources of Selection Bias
Self-assignment to group produces selection bias
Researcher assignment to group produces selection bias
Arbitrary assignment to group produces selection bias
Choosing groups based on their differences results in having groups that are different
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

5. Matching: A Valiant, but Unsuccessful Strategy for Getting Identical Groups
The impossibility of perfectly matching individual participants: Identical participants do not exist
The difficulty of matching groups on every variable: There are too many variables
Two difficulties with matching groups on every relevant variable
Too many relevant variables
We don’t know what the relevant variables are
Problems with matching on pretest scores*
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

6. Problems with Matching on Pretest Scores
Selection by maturation interactions: participants growing In different ways
Regression
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

7. Conclusions about Two-Group Designs
Selection is a big problem
Trying to solve the selection problem may result in having new problems:
Selection by maturation interactions
Regression
Even if selection, selection by maturation, and regression aren’t problems, mortality can ruin the internal validity of a study.
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

8. Problems with the Pretest-Posttest Design
Even without the treatment, participants may change from pretest to posttest*
Even if participants don’t change, their scores may change from pretest to posttest*
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

9. Three Reasons Participants May Change Between Pretest and Posttest
Maturation: Natural physiological changes within the participant may change the participant’s behavior
History: The treatment may not be the only thing in the participant’s environment that changes
Testing: Measuring participants changes participants
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

10. How Measurement changes May Cause Scores to Change Between Pretest and Posttest
Instrumentation: Changes in how participants are measured
Regression revisited: Extreme pretest scores may lead to less extreme posttest scores
Mortality: Changes in how many participants are measured
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

11. Conclusions about Establishing Internal Validity
Trying to keep everything except the treatment constant is impossible.
Only option is to rule out extraneous variables.
As you will see in Chapter 10, one way to automatically rule out extraneous variables is to use random assignment.
Without random assignment, have to identify extraneous variables and then try to rule them out.
Campbell and Stanley’s 8 threats to validity provides a convenient way to identify extraneous variables.
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

12. The Relationship between Internal and External Validity
Attempts to rule out threats to internal validity may hurt external validity
Reducing selection threat at the expense of not studying a heterogeneous group of participants
Reducing history threat at expense of not studying participants in a naturalistic setting
Internal and external validity are not completely incompatible
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley

13. Concluding Remarks
Be cautious about accepting cause-effect statements--if the study is not an experiment, the study’s internal validity is probably threatened by at least one of Campbell and Stanley’s 8 threats to validity.
Experimental designs, as you will soon see, automatically rule out the 8 threats to internal validity.
PowerPoint presentation to accompany Research Design Explained 6th edition;
©2007 Mark Mitchell & Janina Jolley