CJ490: Research Methods in Criminal Justice UNIT #4 SEMINAR Professor Jeffrey Hauck

Chapter 5: 2 Experimental and Quasi-Experimental Designs

Introduction An experiment is “a process of observation, to be carried out in a situation expressly brought about for that purpose” Experiments involve: Taking action Observing the consequences of that action Especially suited for hypothesis testing 3

The Classical Experiment Involves three major pairs of components: Independent and dependent variables Pretesting and posttesting Experimental and control groups 4

Independent Variables The independent variable takes the form of a dichotomous stimulus that is either present or absent It varies (i.e., is independent) in our experimental process IV = cause, DV = effect 5

Dependent Variables The outcome, the effect we expect to see Depends on the IV Might be physical conditions, social behavior, attitudes, feelings, or beliefs KC: several including crime, fear of crime, perceptions of the police, traffic accidents, etc. 6

Pretesting and Posttesting Subjects are initially measured in terms of the DV prior to association with the IV (pretested) Then, they are exposed to the IV Then, they are remeasured in terms of the DV (posttested) Differences noted between the measurements on the DV are attributed to influence of IV 7

Experimental and Control Groups Experimental group – exposed to whatever treatment, policy, initiative we are testing Control group – very similar to experimental group, except that they are NOT exposed If we see a difference, we want to make sure it is due to the IV, and not to a difference between the two groups 8

Hawthorne effect Pointed to the necessity of control groups IV: improved working conditions (better lighting) DV: improvement in employee satisfaction and productivity Workers were responding more to the attention than to the improved working conditions 9

Placebo We often don’t want people to know if they are receiving treatment or not We expose our control group to a “dummy” IV just so we are treating everyone the same Medical research: participants don’t know what they are taking Ensures that changes in DV actually result from IV and are not psychologically based 10

Double-Blind Experiment Experimenters may be more likely to “observe” improvements among those who received drug In a DB experiment, neither the subjects nor the experimenters know which is the experimental group and which is the control group 11

Selecting Subjects First, must decide on target population – the group to which the results of your experiment will apply Second, must decide how to select particular members from that group for your experiment Cardinal rule – ensure that E and C groups are as similar as possible Randomization purposes towards this 12

Experiments and Causal Inference Experimental design ensures: Cause precedes effect via taking posttest Empirical correlation exists via comparing pretest to posttest No spurious 3 rd variable influencing correlation via posttest comparison between experimental and control groups, and via randomization 13

Internal Validity Threats Conclusions drawn from experimental results may not reflect what went on in experiment 1. History – external events may occur during the course of the experiment 2. Maturation – people constantly are growing 3. Testing – the process of testing and retesting 14

Internal Validity Threats 4. Instrumentation – Changes in the measurement process 5. Statistical regression – Extreme scores regress to the mean 6. Selection bias – the way in which subjects are chosen 7. Experimental mortality – subjects may drop out prior to completion of experiment 15

Internal Validity Threats 8. Causal time order – ambiguity about order of stimulus and DV – which caused which? 9. Diffusion/imitation of treatment – when E and C groups communicate, E group may pass on elements to C 10. Compensatory treatment – C group is deprived of something considered to be of value 16

Internal Validity Threats 11. Compensatory Rivalry – C group deprived of the stimulus may try to compensate by working harder 12. Demoralization – feelings of deprivation result in C group giving up 17

Construct Validity Threats Concerned with generalizing from experiment to actual causal processes in the real world Link construct and measures to theory Clearly indicate what constructs are represented by what measures Decide how much treatment is required to produce change in DV 18

External Validity Threats Significant for experiments conducted under carefully controlled conditions rather than more natural conditions But, this reduces internal validity threats! A conundrum! Suggestion – explanatory studies -> internal validity; applied studies -> external validity 19

Statistical Conclusion Validity Threats Becomes an issue when findings are based on small samples More cases allows you to reliably detect small differences; less cases result in detection of only large differences 20

Quasi-Experimental Designs When randomization isn’t possible for legal or ethical reasons Renders them subject to IV threats quasi = “to a certain degree” Two categories: non-equivalent-groups designs and time series designs 21

Nonequivalent-Groups Designs When we cannot randomize, we cannot assume equivalency; hence the name We take steps to make groups as comparable as possible Match subjects in E and C groups using important variables likely related to DV under study Aggregate matching – comparable average characteristics 22

Cohort Designs cohort – group of subjects who enter or leave an institution at the same time Necessary to ensure that two cohorts being examined against one another are actually comparable 23

Time-Series Designs Examine a series of observations over time Interrupted – observations compared before and after some intervention Instrumentation threat to internal validity is likely because changes in measurements may occur over a long period of time Often use measures produced by CJ organizations 24

Variable-Oriented Research A large number of variables are studied for a small number of cases or subjects Case-study design – centers on an in-depth examination of one or a few cases on many dimensions Boston Gun Project 25

QUESTIONS?