Chapter 9 Expanding on Experimental Designs: Repeated Measures and Quasi-Experiments

Repeated Measures Designs Advantages to Repeated Measures Designs Increasing efficiency in data collection Increasing validity of data Reducing error in measurements Finding enough participants

Repeated Measures Designs Repeated measures designs are sometimes not optimal? Questions How big is 9? How big is 221? Is 9 larger than 221?

Repeated Measures Designs Method Independent Groups Design: Some participants responded to the question How big is 9 on a scale of 1 to 10? Other participants responded to the question How big is 221 on a scale of 1 to 10?

Repeated Measures Designs Result: The mean rating of 9 was higher than the mean rating of 221. Source: Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243-249. Copyright American Psychological Association. Used with permission.

Repeated Measures Designs Conclusion Participants compared 9 to small numbers, so it seemed large, relatively speaking. Participants compared 221 to very large numbers, so it seemed small, relatively speaking. A repeated measures design would avoid this type of result.

Repeated Measures Designs Limitations to Repeated Measures Designs Some research does not lend itself to repeated measures, like measuring young and old adults. Studies using subject (participant) variables, like political affiliation, cannot make use of repeated measures because a person can realistically be in only one condition. Sequence and order effects can be a problem if early treatments affect later treatments.

Repeated Measures Designs Overcoming Sequence or Order Effects Counterbalancing –Changing the order of conditions from one participant to avoid the problems of sequence and order effects Complete counterbalancing—using all possible orders across participants (e.g., for conditions A, B, and C: ABC, ACB, BAC, BCA, CAB, CBA) Partial counterbalancing—using a subset of all possible orders across participants

Repeated Measures Designs Additional Problems with Repeated Measures Designs Transfer—participants change because they learn from one situation the next. Symmetric Transfer—participants’ behaviors change equally with repeated measurements, no matter what treatments come first Asymmetric transfer—participants’ behaviors change differently depending on what treatments come first

Data Analysis with Repeated Measures Designs When we compare two groups in a repeated measures design, we typically use a dependent measures t test (also called repeated measures t test or correlated groups t test) With multiple groups or multiple IVs, we typically use the Analysis of Variance

Quasi-Experimental Designs When researchers create groups to compare but cannot randomly assign participants to those groups, the design is called quasi-experimental. Quasi-experimental designs look like true experiments, but there is no random assignment or true manipulation of the IV. Without random assignment and manipulation of the IV, we cannot determine cause-effect relationships.

Quasi-Experimental Designs Threats to Internal Validity Due to Participants Selection threat—Groups may differ in important ways before the study even starts. Maturation threat—Short- or long-term changes in participants may change their behaviors (e.g., boredom, physical development) Attrition—People may drop out of the study; those who do may differ from those who remain History threat—Some minor or major event may occur during a study that changes the participants’ behaviors.

Quasi-Experimental Designs Threats to Internal Validity Due to Measurement Instrumentation threat—Changes in the way the DV is measured due to changes in apparatus or in the way observers code data. Testing threat—Changes in participants’ behaviors because they have undergone testing. Statistical regression threat—Changes in participants’ scores on repeated testing because extreme scorers were included and their subsequent scores move toward the mean

Controversy: Sex and Death Question: Are sex and death are psychologically related, which Terror Management Theory suggests.

Controversy: Sex and Death Method Researchers measured people to see if they were low or high in neuroticism (a quasi-experimental variable, subject variable). They primed participants to think either of the romantic or of the physical aspects of sex (a true independent variable). The study involved a 2 x 2 design, with one quasi-experimental IV (level of neuroticism) and one true IV (type of prime)

Controversy: Sex and Death Participants then completed word stems (e.g., COFF_ _ or SK_LL) to see if participants completed them with death-related words (COFFIN versus COFFEE or SKULL versus SKILL) Result: There was an interaction between the neuroticism level and type of prime. People high in neuroticism were more likely to generate death-related words when physically primed compared to people low in neuroticism and less likely to generate death-related words when romantically primed.

Controversy: Sex and Death Conclusion Sex and death are related among those high in neuroticism.

Types of Quasi-Experimental Designs One-Group Pretest-Posttest Design—Quasi-experimental design in which a single group is tested, administered a treatment, then tested again. Single group of participants Observation Treatment Observation

Types of Quasi-Experimental Designs Static-Group Comparison Design—Quasi-experimental design in which two groups of nonrandomly assigned participants differing in some pre-existing way are compared, with one receiving a treatment and the other no treatment Nonrandom placement in Group 1 Treatment Observation Nonrandom placement in Group 2 No Treatment Observation

Types of Quasi-Experimental Designs Nonequivalent Control Group Design—Quasi-experimental design in which two groups with pre-existing differences are compared in a repeated measures situation. Nonrandom placement in Group 1 Observation Treatment Observation Nonrandom placement in Group 2 Observation No Treatment Observation

Types of Quasi-Experimental Designs Potential problem with Nonequivalent Control Group Design Participants may have pre-existing differences such that they could show different patterns across time that you might not spot with simply two measurements.

Types of Quasi-Experimental Designs Time Series Design—Quasi-experimental design in which a researcher makes multiple observations over time, trying to discern patterns of behavior Group of participants Observation Observation Observation

Types of Quasi-Experimental Designs Interrupted Time Series Design—Quasi-experimental design in which a researcher makes multiple observations over time, with a treatment at some point along the measurement continuum; there can be many observations before or after a treatment Group Observation Observation Treatment Observation Observation

Types of Quasi-Experimental Designs Replicated Interrupted Time Series Design—Quasi-experimental design in which a researcher identifies two groups and makes multiple observations over time, with a treatment at different times for the two groups; there can be many observations before or after a treatment Group 1 Observation Treatment Observation Observation Observation Group 2 Observation Observation Observation Treatment Observation

Replicated Interrupted Time Series Design Question: Does antisocial behavior in the media lead to antisocial behavior in life? In the early 1950s, Congress passed a law barring the creation of new television stations in cities that did not already have them. Crime dramas were popular even in the early days of TV. So some cities had access to TV crime shows showing larceny, but other cities didn’t.

Replicated Interrupted Time Series Design Method: Collect data on the incidence of antisocial behavior (e.g., larceny) when cities did or did not have access to television that portrays antisocial behavior. The researchers used a replicated interrupted time series design.

Replicated Interrupted Time Series Design Result: When cities did not have access to crime shows on TV, the crime rate was lower compared to cities that had such access. When TV was later allowed in those cities, the crime rate rose. Arrows indicate year of arrival of TV in the prefreeze and postfreeze cities. Source: Hennigan, K. M., Del Rosario, M. L., Heath, L. Cook, T. D., Wharton, J. D., & Calder, B. M. (1982). Impact of the introduction of television on crime in the United States: Empirical findings and theoretical implications. Journal of Personality and Social Psychology, 42, 461-577. Copyright American Psychological Association. Used with permission.

Type of Quasi-Experimental Designs Conclusion: The arrival of media that portrays antisocial behavior is associated with more crime. The fact that crime rates rose twice, after each introduction of television, suggests that antisocial behavior like larceny (i.e., theft) results from exposure to that behavior on TV. The design is quasi-experimental, so we cannot determine causation with absolute confidence, although this design is very suggestive.