Chapter 11 Quasi-Experimental Designs ♣ ♣ Introduction   Nonequivalent Comparison Group Design   Time-Series Design   Regression Discontinuity Design    Back to Brief Contents

11.0 Introduction Quasi-experimental design  Not meet all the requirements necessary for controlling the influence of EV (e.g.) random assignment  Not eliminate all threats to internal validity Threats to internal validity are ruled out by Tab 11.1 Tab 11.1  Identification and study of the threats  By including design elements as pretests or other control groups  By coherent pattern matching—making a complex prediction that few rival hypotheses can explain  Back to Chapter Contents

11.1 Nonequivalent Comparison Group Design pre-test treatment posttest Exper. Gp Y 1 X Y 2 Control Gp Y 1 Y 2 Fig 11.1 Fig 11.1 This is the most common quasi-experimental design Threats frequently reveal themselves in the outcome Example Fig Fig Possible biases Tab 11.2 Tab 11.2  Back to Chapter Contents

11.1 Nonequivalent Comparison Group Design Outcomes with Rival Hypotheses  Outcome I: First Increasing Treatment Effect  Outcome II: Increasing Treatment and Control Groups  Outcome III: Second Increasing Treatment Effect  Outcome IV: Crossover Effect Causal Inference from the Nonequivalent Comparison Group Design  Back to Chapter Contents

Outcome Patterns in the Nonequivalent Comparison Group Design I: Increasing treatment effect Fig 11.4 Fig 11.4 experimental control pretest posttest Threat: Selection-Maturation, Selection-History Eliminate Selection-Maturation : Matching Tab 11.3 Tab 11.3 Statistical Regression Techniques

II: Increasing treatment and control groups Fig 11.5Fig 11.5 experimental control pretest posttest  Selection-maturation is a threat

III: Increasing treatment effect Fig 11.6 Fig 11.6  Statistical regression is a threat  Selection-history is a threat control experimental pretest posttest

IV: Cross-over effect Fig 11.7 Fig 11.7  No possible rival hypotheses (history?) experimental control pretest posttest

11.1 Nonequivalent Comparison Group Design Outcomes with Rival Hypotheses Causal Inference from the Nonequivalent Comparison Group Design  Well designed and executed: ≈ randomized design  Well designed P assigned to group: not self-select Reduce pretest difference: matching  Back to Chapter Contents

11.2 Time-Series Design Interrupted Time-Series design Fig 11.8 Fig 11.8 Pre-response Treatment Post-response Y 1 Y 1 Y 1 Y 1 X Y 2 Y 2 Y 2 Y 2  Treatment effect revealed by a different pattern of pre and posttest responses  Back to Chapter Contents

11.2 Time-Series Design Interrupted Time-Series design  Lawler & Hackman (1969) IV: employee participation in develop and implementation of incentive plan DV: absenteeism Fig 11.9 Fig 11.9  Vernon, Bedford, & Wyatt (1924) IV: introduce a rest period DV: productivity (output) Fig Fig  One-group pretest-posttest Fig Fig  Back to Chapter Contents

11.2 Time-Series Design Interrupted Time-Series design  Possible patterns Fig Fig  Statistical method Bayesian moving average model Tryon (1982), Crosbie (1993)  Threat: History (end)  Back to Chapter Contents

11.3 Regression Discontinuity Design Used to determine if the special treatment some individuals receive has any effect Characteristics of the design Fig Fig  All individuals are pretested  Individuals who score above some cutoff score receive the treatment  All individuals are posttested  Discontinuity in the regression line indicates a treatment effect Fig Fig  Back to Chapter Contents

11.3 Regression Discontinuity Design Design Criteria that Must be Adhered to Tab 11.4 Tab 11.4  Assignment must be based on the cutoff score  Assignment cannot be a nominal variable as gender, or drug user or nonuser  Cutoff score should be at (or near) the mean  Experimenter should control group assignment  Relationship (linear, curvilinear, etc.) should be known  Participants must be from the same population  Back to Chapter Contents

11.3 Regression Discontinuity Design Primary Threat in the Regression Discontinuity Design  Contemporaneous history effect — This is possible but not plausible  Attrition: any design  This is the more powerful of the quasi-experimental designs (end)  Back to Chapter Contents