2 Chapter Outline • Sources of invalidity • Threats to internal validity • Threats to external validity• Controlling threats to internal validity• Controlling threats to external validity• Types of designs
3 Experimental Research Tries to Establish Cause and Effect • Selection of a good theoretical framework• Application of appropriate experimental design• Use of correct statistical model and analysis• Proper selection and control of independentvariables• Appropriate selection and measurement ofdependent variables• Correct interpretation of results
4 Three Criteria for Cause and Effect 1. The cause must precede the effect in time.2. The cause and effect must be correlated witheach other.3. The correlation between cause and effect cannotbe explained by another variable.If the condition is necessary and sufficient toproduce the effect, then it is the cause.
5 Distinguishing Between Types of Validity • Internal validity: Did the treatments (IV) causethe change in the outcome (dv)?• External validity: To what populations, settings,or treatments can the outcome be generalized?• Is there a trade-off between internal and externalvalidity?• Can a series of studies address the trade-off?
6 Threats to Internal Validity • History: Events that are not part of treatment• Maturation: Events due to passage of time• Testing: Effects of more than one testadministration• Instrumentation: Change in calibration ofmeasurements• Statistical regression: Selection based onextreme score(continued)
7 Threats to Internal Validity • Selection biases: Nonrandom participantselection• Experimental mortality: Differential loss ofparticipants• Selection–maturation interaction: Passage oftime influencing groups differently• Expectancy: Influence of experimenters onparticipants
8 Threats to External Validity • Reactive or interactive effects of testing: Pretestmay make participants sensitive to treatment.• Reactive effects of experimental arrangements:Setting constraints may influence generalizability.• Multiple-treatment interference: One treatmentmay influence the next treatment.
9 Controlling Threats to Internal Validity • Randomization - Real randomization- Matched pairs (not matched groups)- Randomizing treatments or counterbalancing• Placebos• Blind setups(continued)
10 Controlling Threats to Internal Validity • Double-blind setups • Reactive effects of testing: Eliminate pretest.• Instrumentation- Calibration and test reliability- Halo effects• Experimental mortality: Keeping participants
11 Controlling Threats to External Validity • Selecting from larger populations- Participants- Treatments- Situations• Ecological validity: Does the setting capture theessence of the real world?
12 Types of Designs: Preexperimental Designs One-shot studiesT OOne-group pretest-posttestO1 T O2 Statistical analysis?Static group comparisonT O1Statistical analysis?O2
13 Types of Designs: True Experimental Designs Randomized-groups designR T O1 Statistical analysis?R O2Extending the levels—randomized-groups designR T1 O1R T2 O2 Statistical analysis?R O3(continued)
14 Types of Designs: True Experimental Designs (continued)
15 Types of Designs: True Experimental Designs Pretest-posttest randomized-groupsR O1 T O2R O3 O4 Statistical analysis?Extending the design on the RM factorR O1 T O2 T O3 Statistical analysis? R O4 O5 O6(continued)
16 Types of Designs: True Experimental Designs Solomon four-group design—purposeR O1 T O2R O3 O4R T O5R O6Statistical analysis (factorial ANOVA)No treatment TreatmentPretested O4 O2Unpretested O6 O5(continued)
17 Quasi-Experimental Designs: Time Series Campbell and Stanley 1963.
19 Quasi-Experimental Designs: Ex Post Facto This is one of the preexperimental designs, but withthe treatment not under the control of the experimenter.T O1Statistical analysis?O2
20 Quasi-Experimental Designs: Single Participant Identify participant and follow over time.• Does the treatment produce the same effecteach time?• Are treatment effects cumulative, or doesparticipant return to baseline?• Does participant’s response become less variableover treatment times?(continued)
21 Quasi-Experimental Designs: Single Participant • Is participant’s magnitude of response sensitiveto multiple treatment applications?• Do varying intensities, frequencies, and lengthsof treatment produce varying responses?