Research Design: The Experimental Model and Its Variations CHAPTER THREE Research Design: The Experimental Model and Its Variations

Research Design Research design is the plan, the blueprint, or a schematic for a study – it flows from the problem formulation stage. The who, what, where, when, why, and how of an investigation. The goal is how best to address the hypothesis.

Causality The purpose of scientific investigation is: to isolate, define, and explain the relationship between key variables in order to predict and understand the underlying nature of reality. (who, what, where, when, and how) Causation lies at the basis of reality.

Questions How would causation apply to the variables: foot patrol and crime? Does showing a relationship (correlation) between two variables imply or demonstrate causation?

Steps for Resolving the Causality Problem Demonstration of a relationship between variables (covariance). In other words, is there a predictable relationship in the value of one variable to the value of another variable. Specifying or indicating the time sequence or time order of the relationship. Which variable will be “X” (predictor) and which variable will be “Y” (outcome). Eliminate rival causal factors to avoid a spurious relationship (a false relationship) or, avoid the elimination of other variables that could conceivably explain away the original relationship.

Classic Experimental Design E O1 X O2 E O1 O2 E = equivalence O = observation X = treatment 1,2 = time Examples: Candid Camera, Scared Straight, Community Policing (Text, pages: 91-93).

Experimental Design Terms: X = treatment (independent variable) Y = outcome (dependent variable) Z = any rival causal factors O = observation (some measurement or assessment of the dependent variable) E = equivalence (randomization or matching) 1,2 = number of times

Rival Causal Factors Factors other than “X” (the treatment) that may be responsible for the relationship. Validity refers to accuracy or correctness in research, i.e., internal and external. Internal Validity is concerned with a variable other than X that may have produced the change in Y. External Validity is concerned with what other variables may limit one’s ability to generalize the findings in a study to larger groups or populations.

Internal Validity History: Refers to other specific events that may have occurred over the time of the study that may have produced the results. Example: a new program introduced such as “Crime Watch” or urban renewal. Maturation: Biological or psychological changes in respondents during the course of study that are not due to the treatment variable. Example: age. Testing: Pretest bias, bias and foreknowledge introduced to respondents as a result of having been pretested.

Internal Validity (cont’d) Instrumentation: Changing the measurement instrument from the beginning or the first period of evaluation to the second or final evaluation. Ex.: the method of recording citizen complaints changes. Statistical Regression: the tendency of groups selected for study on the basis of high or low scores to regress or move toward the mean or the average on second testing. Scores become more normal upon retest. Ex.: First test may have been atypical.

Internal Validity (cont’d) Selection Bias: Choosing nonequivalent groups for testing. Ex: Selecting all volunteer prisoners or all model prisoners. Experimental Mortality: Expected loss of subjects in the sample group over a period of time. Ex: Following recidivism cohorts over long periods of time. Selection-Maturation Interaction: Selection bias coupled with issues that emphasize biological or psychological changes during the course of the study or over time. Ex.: Age and Recidivism

External Validity Testing Effects: Exposure to pretests by respondents negates the generalizability of the results to larger populations that have not been pretested. Example: Pretest of community attitudes toward the police prior to a foot patrol experiment followed up by a posttest regarding community attitudes toward the police after the experiment. Selection Bias: Specific studies that are based on a specific group may not be comparable to a larger group that does not have the same specific characteristics. Ex.: Local drug use studies.

External Validity (Cont’d) Reactivity: Awareness of being studied tends to produce atypical or unnatural behavior on the parts of subjects. Example: Hawthorne Effect Multiple-Treatment Interferences: Occurs when more than one treatment or predictor variable is used on the same subjects. Example: addition of foot patrol plus foot patrol officers were also unarmed and wore different uniforms.

Question What might be a rival causal factor which may affect the relationship between our predictor variable foot patrol (X), and our outcome variable crime (Y)?

Related Rival Causes (be able discuss the following): Hawthorne Effect Halo Effect Post Hoc Error Placebo Effect Diffusion of Treatment Compensatory Equalization of Treatment Local History

Experimental Design Terms: X = treatment (independent variable) Y = outcome (dependent variable) Z = any rival causal factors O = observation (some measurement or assessment of the dependent variable) E = equivalence (randomization or matching) 1,2 = number of times

Classic Experimental Design E O1 X O2 E O1 O2 E = equivalence O = observation X = treatment 1,2 = time Examples: Candid Camera, Scared Straight, Community Policing (Text, pages: 91-93).

THREE ELEMENTS OF THE CLASSIC EXPERIMENTAL DESIGN 1) Equivalence is the assignment to comparison groups in a manner in which the subjects are alike in all major respects, i.e., randomization and matching. (E) Randomization is the random assignment of subjects where all individuals have an equal probability to being assigned to a particular group. Matching is the selecting of subjects for comparison groups based on key characteristics so that the group is similar in respect to these characteristics.

Three Elements (Cont’d) 2) The classic experiment consists of pretests and posttests. A pretest is an observation prior to exposure to treatment (O1). A posttest is an observation and measurement after treatment (O2).

Three Elements (Cont’d) 3) The classic experiment consists of experimental and control groups. The experimental group is exposed to treatment (X). The control group is not exposed to the treatment.

Solomon Four-Group Design E O1 X O2 E O1 O2 E X O2 E 02 Eliminates testing effects and reactivity (awareness of being studied)

Cross-sectional and Longitudinal Design Often referred to as cross-sectional (one group at one time); longitudinal (one group over time), i.e., time-series, cohort studies, panel studies, and trend studies. Time series: measuring a single variable at successive points in time (E = matching) Interrupted time-series: measurement before and after treatment for an equivalent period of time Trend studies: analyze different sample of the same population longitudinally Cohort studies: Analyze subgroups over time. Panel studies: Analyze the same group over time.

Time Series Design Interrupted Time-Series Designs O O O X O O O Multiple Time-Series Designs O O O O O O Examples: Problem Oriented Policing (Spelman, et al.), Monahan and Walker’s Study of mental health centers, and Shneider and Smykla’s “War and Capital Punishment” study (Text, pages: 102-103).

Advantages and Disadvantage of Experiments Advantages: 1) Control of rival factors (internal validity); 2) Quick and inexpensive; 3) Manageability; and, 4) Can be applied to natural settings. Disadvantages: 1) Artificiality (hinders generalizability); and, 2) Difficult to apply experiments to human subjects and situations in criminal justice (i.e., ethical issues and experimenter effects). Disadvantages in criminal justice subject matter often outweigh the advantages.

Questions What type of design was the Kansas City Gun Experiment? What were the major findings of the project? What is a rival factor of concern when evaluating shock incarceration programs? Why are time-series designs particularly useful in criminal justice studies?