# Quasi-Experimental Design

## Presentation on theme: "Quasi-Experimental Design"— Presentation transcript:

Quasi-Experimental Design
Jung Eun (Jessie) Hong Feb. 23, 2009

Outlines Experimental Design Types of Experimental Designs
Definition Process A Key Point Types of Experimental Designs Quasi-Experimental Design Designs Strengths Weaknesses Examples in Geography Ongoing Debate

Definition of Experimental Design
A structured, organized method To determine whether some program or treatment causes some outcome or outcomes to occur. If X, then Y Because there may be lots of reasons, other than the program, for why you observed the outcome, If not X, then not Y needs to be addressed, too

Process of Experimental Design
To show that there is a casual relationship, Two “equivalent” groups The program or treatment group gets the program The comparison or control group does not The groups are treated the same in all other respects Differences in outcomes between two groups must be due to “the program”

A Key Point of Experimental Design
How do we create two groups that are “equivalent”? Assign people randomly from a common pool of people into the two groups The experiment relies on the idea of “random assignment” to obtain two similar groups. A key to the success of the experiment Assume that two groups are “probabilistically equivalent”

Types of Designs Is random assignment used?
No Yes Randomized or True experiment Is there a control group or multiple measures? No Yes Non-experiment: A research design in which a researcher observes a phenomenon without manipulating the independent variables(s) Non-experiment Quasi-experiment

Quasi-Experimental Design
Similar to the experimental design, but lacks the key ingredient, “random assignment” Easily and more frequently implemented Extensively used in the social sciences A useful method for measuring social variables Two classic quasi-experimental designs The Nonequivalent Groups Design The Regression-Discontinuity Design

The Nonequivalent Groups Design
The most frequently used in social research Try to select groups that are as similar as possible to compare the treated one with the comparison one e.g. two comparable classrooms or schools Cannot be sure whether the groups are comparable The groups may be different prior to the study Any prior differences between the groups may affect the outcome of the study Require a pretest and posttest

The Regression-Discontinuity Design
A useful method for determining whether a program of treatment is effective Participants are assigned to program or comparison groups based on a cutoff score on a pretest e.g. Evaluating new learning method to children who obtained low scores at the previous test. Cutoff score = 50 The treatment group: children who obtained 0 to 50 The comparison group: children who obtained 51 to 100 The program (treatment) can be given to those most in need

The Regression-Discontinuity Design
With no treatment effect All points to the left of the cutoff (treatment group) have been raised by 10 points on the posttest. The dashed line shows what we expect the treated group’s regression line to look like if the program had no effect. With Ten point treatment effect

The Regression-Discontinuity Design
The program effect is suggested when we observe a discontinuity in the regression lines at the cutoff point. Also, we can now see how this design got its name.

Strengths of Quasi-Experimental Design
Useful in generating results for general trends in social sciences Difficult pre-selection and randomization of groups Easily integrated with individual case studies Generated results can reinforce the findings in a case study Allow statistical analysis to take place Enable to reduce the time and resources required for experimentation Not required extensive pre-screening and randomization

Weaknesses of Quasi-Experimental Design
Without proper randomization, statistical tests can be meaningless Do not explain any pre-existing factors and influences outside of the experiment The researcher needs to control additional factors that may have affected the results Some form of pre-testing or random selection may be necessary to explain statistical results thoroughly

Quasi-experiments vs. Non-experiments to address similar questions
Both designs are applicable when the subjects are not able to be randomized Some variables cannot ethically be randomized e.g. Studying the effect of maternal alcohol use when the mother is pregnant Quasi-experiments Non-experiments Strengths Enable to compare with other groups Enable to focus on one variable Weaknesses Unexpected factors might affect the results Interpretations might be improper

Baker and White (2003) The Effects of GIS on Students’ Attitudes, Self-efficacy, and Achievement in Middle School Science Classrooms Conducted the Nonequivalent quasi-experimental design Two eighth grade teachers, across ten classrooms Total 192 eighth grade students participated Treatment group: used a Web-based GIS application Control group: used paper maps Treatment Group Control Group Instructor 1 51 36 Instructor 2 42 63