Chapter 8 Experimental Design The nature of an experimental design Pre-experimental models True experimental models Threats to validity

The Nature of Experimental Design Changes in the independent variable (IV) result in (cause) changes in the dependent variable (DV); causation may be improperly attributed. True experiments can address all three parts of causality

Methodology Stimulus (or treatment): the independent variable in experiments; X Observations: the dependent variables represented in experimental models; O Procedures

A True Experiment A true experiment requires two things: Random assignment Control group

After Only Design An experimental design in which a single measurement of the observation is taken, after the application of the stimulus; measurement of the DV is taken after the application of the IV

Random Assignment Distribute available subjects into groups in some systematic way in an attempt to create equivalent groups

Control Group The stimulus (treatment) group is subjected to some experience (the independent variable) The control group has a similar experience to the stimulus group, but no exposure to the independent variable

2 Primary Classes of Experiments Pre-experimental models True experimental models

Pre-Experimental Models Case study Simple pre-post Static group comparison In the models: X = the IV, stimulus, treatment O = the DV, usually a measurement

Case Study X O Most commonly used in examining naturally occurring events or phenomena; something happens (X), we measure its impact (by observation, O) Sometimes referred to as firehouse research

Simple Pre-post (without control) O1 X O2 We anticipate that an event will occur Observations (O1 and O2) are made before and after the event (stimulus), X

Static Group Comparison X O1 O2 The researcher separates a single group (O) into two parts (O1 and O2) based on some criteria. Differences are assessed based on exposure to the stimulus, X. O1 experiences the stimulus, O2 does not.

True Experimental Models The researcher has control over who will get the stimulus There is random assignment to experimental groups and a control group

True Experimental Models After only design Classic experimental model Solomon 4-groups design

After Only Design R X O1 R O2 Two lines indicates two groups; the R indicates random assignment The first group has the treatment (X) and an observation/measurement (O1) The second group has no treatment, just an observation (O2), making it the control group

Classic Experimental Model Sometimes, we want to be certain that the groups are equivalent in levels of the dependent variable A pretest at time 1 (T1) and a posttest at time 2 (T2) allow us to confirm that random assignment created equivalent groups However, every measurement potentially changes the phenomenon we are trying to observe

Classic Experimental Model T1 T2 R O1 X O2 R O3 O4 Each group is measured, observed (O1, O2, O3, O4) before (T1) and after (T2) the experimental group receives treatment (X)

Solomon 4-Group Design After only design and classical experimental design combined Four randomly assigned groups, two stimulus groups, two control groups, six observations (two pretests and four posttests) Rarely used

Solomon 4-Group Design T1 T2 R O1 X O2 R O3 O4 R X O5 R O6 Combines every possible configuration of after only design and classical experimental design

Threats to Validity Potential main effects (direct impact of one variable on another variable): History Maturation Testing or reactivity

Threats to Validity History: becomes a threat to the validity of our design when something occurs in the world outside of our study that has an impact on our subjects Can affect performance on posttest

Threats to Validity Maturation: results from changes in the subjects, unrelated to the study, which will have impact on the results of the study

Threats to Validity Testing or reactivity: the act of observing / measuring changes the very phenomenon we are studying Demand characteristic (the Hawthorne effect): a situation where the subject is unconsciously trying to help the researcher

Threats to Validity The previous threats to validity are main effects Sensitivity is a more subtle threat to validity Interactions: occur when two or more variables create changes in the dependent variable A pretest may interact with the stimulus to create an effect that appears to bolster the impact of the stimulus

Threats to Validity Self selection: rather than a subject being selected into a sample, an individual decides whether to participate or not Negatively impacts random assignment

Threats to Validity Regression to the mean: multiple measures of the same thing can randomly give multiple results; a second measure may be closer to the “true” average