Topics Appropriate to Experiments Projects with limited and well-defined concepts. Projects that are exploratory rather than descriptive. Studies of small group interaction.
Components of Experiments Independent and dependent variables Pretesting and posttesting Experimental and control groups – Stimulus and no stimulus
Experimental and Control Groups Must be as similar as possible. Control or Comparison group represents what the experimental group would have been like had it not been exposed to the stimulus.
Experiment
Two & Half Men Short clip making fun of the experimental design in testing drugs
Selecting Subjects Probability sampling Randomization Matching
Randomization and Matching May not know which variables will be relevant for matching process. Most statistics used to analyze results assume randomization. Randomization only makes sense if you have a large pool of subjects.
Internal Validity Refers to the possibility that the conclusions drawn from experimental results may not accurately reflect what went on in the experiment itself. Did something other than the experimental stimulus affect the dependent variable?
Sources of Internal Invalidity Historical events may occur during the course of the experiment. Maturation of the subjects. Testing and retesting can influence behavior. Instrumentation
Sources of Internal Invalidity Selection biases. Experimental mortality - subjects drop out of the study before it's completed.
External Validity Refers to the possibility that conclusions drawn from experimental results may not be generalizable to the “real” world. Is the experimental setting unrealistic? Is pre-testing influencing the subjects (i.e. cueing them in on what the researcher wants)
Limiting External Invalidity Solomon Four Group Design - Four groups of subjects, assigned randomly: Groups 1 and 2 are the control and experimental group. Group 3 does not have the pre-test. Group 4 is only posttested.
Solomon Four-Group Design – G1:Pretest-stimulus-posttest – G2:Pretest-posttest – G3:Stimulus-posttest – G4:posttest
Solomon Four-Group Design
Example Champney and Edleman “Assessing Student Learning Outcomes in United States Government Courses” Available on WebCampus (not required reading).
Posttest-only Control Group Design Includes Groups 3 and 4 of the Solomon design. With proper randomization, only these groups are needed to control problems of internal invalidity and the interaction between testing and stimulus. Commonly done in the social sciences
Web-based Experiments Increasingly, researchers are using the World Wide Web to conduct experiments. Because representative samples are not essential in some experiments, researchers use volunteers who respond to invitations online.
Experimental Method Strengths: Isolation of the experimental variable over time. Experiments can be replicated several times using different groups of subjects.
Experimental Method Weaknesses: Artificiality of laboratory setting. Social processes that occur in a lab might not occur in a more natural social setting.
Example of Experiments in Political Science Mendelberg, Tali Executing Hortons: Racial Crime in the 1988 Presidential Campaign. The Public Opinion Quarterly 61(1, Spring): Available on WebCampus (not required reading).
Methodology Experiment 77 white students at U of M Median age was 18 Treatment group shown the ad, control group was not OLS regression (not a T-test)
Findings Students shown the Horton ad were more likely to have negative views on race and racial policies Students shown the Horton ad did not have different views on crime
The End Read Levin and Fox Ch. 7 – Will go over homework assignment Read Matland, Richard E “Putting Scandinavian Equality to the Test: An Experimental Evaluation of Gender Stereotyping of Political Candidates in a Sample of Norwegian Voters.” British Journal of Political Science 24, 2: Available on WebCampus