Presentation is loading. Please wait.

Presentation is loading. Please wait.

Quasi-Experimental Design Jung Eun (Jessie) Hong Feb. 23, 2009.

Similar presentations


Presentation on theme: "Quasi-Experimental Design Jung Eun (Jessie) Hong Feb. 23, 2009."— Presentation transcript:

1 Quasi-Experimental Design Jung Eun (Jessie) Hong Feb. 23, 2009

2 Outlines Experimental Design  Definition  Process  A Key Point Types of Experimental Designs Quasi-Experimental Design  Designs  Strengths  Weaknesses  Examples in Geography  Ongoing Debate

3 Definition of Experimental Design A structured, organized method  To determine whether some program or treatment causes some outcome or outcomes to occur. If X, then Y  Because there may be lots of reasons, other than the program, for why you observed the outcome, If not X, then not Y needs to be addressed, too

4 Process of Experimental Design To show that there is a casual relationship,  Two “equivalent” groups The program or treatment group gets the program The comparison or control group does not The groups are treated the same in all other respects  Differences in outcomes between two groups must be due to “the program”

5 A Key Point of Experimental Design How do we create two groups that are “equivalent”?  Assign people randomly from a common pool of people into the two groups The experiment relies on the idea of “random assignment” to obtain two similar groups.  A key to the success of the experiment Assume that two groups are “probabilistically equivalent”

6 Types of Designs Randomized or True experiment Yes No Is random assignment used? Is there a control group or multiple measures? Yes No Quasi-experiment Non-experiment

7 Quasi-Experimental Design Similar to the experimental design, but lacks the key ingredient, “random assignment” Easily and more frequently implemented Extensively used in the social sciences  A useful method for measuring social variables Two classic quasi-experimental designs  The Nonequivalent Groups Design  The Regression-Discontinuity Design

8 The Nonequivalent Groups Design The most frequently used in social research Try to select groups that are as similar as possible to compare the treated one with the comparison one  e.g. two comparable classrooms or schools  Cannot be sure whether the groups are comparable The groups may be different prior to the study  Any prior differences between the groups may affect the outcome of the study  Require a pretest and posttest

9

10 The Regression-Discontinuity Design A useful method for determining whether a program of treatment is effective Participants are assigned to program or comparison groups based on a cutoff score on a pretest  e.g. Evaluating new learning method to children who obtained low scores at the previous test. Cutoff score = 50 The treatment group: children who obtained 0 to 50 The comparison group: children who obtained 51 to 100 The program (treatment) can be given to those most in need

11 The Regression-Discontinuity Design With no treatment effect With Ten point treatment effect

12 The Regression-Discontinuity Design Discontinuity

13 Strengths of Quasi-Experimental Design Useful in generating results for general trends in social sciences  Difficult pre-selection and randomization of groups Easily integrated with individual case studies  Generated results can reinforce the findings in a case study  Allow statistical analysis to take place Enable to reduce the time and resources required for experimentation  Not required extensive pre-screening and randomization

14 Weaknesses of Quasi-Experimental Design Without proper randomization, statistical tests can be meaningless  Do not explain any pre-existing factors and influences outside of the experiment The researcher needs to control additional factors that may have affected the results Some form of pre-testing or random selection may be necessary to explain statistical results thoroughly

15 Quasi-experiments vs. Non-experiments to address similar questions Both designs are applicable when the subjects are not able to be randomized  Some variables cannot ethically be randomized e.g. Studying the effect of maternal alcohol use when the mother is pregnant Quasi-experimentsNon-experiments Strengths Enable to compare with other groups Enable to focus on one variable Weaknesses Unexpected factors might affect the results Interpretations might be improper

16 Example of Quasi-Experimental Design in Geography Baker and White (2003)  The Effects of GIS on Students’ Attitudes, Self-efficacy, and Achievement in Middle School Science Classrooms  Conducted the Nonequivalent quasi-experimental design Two eighth grade teachers, across ten classrooms Total 192 eighth grade students participated Treatment group: used a Web-based GIS application Control group: used paper maps Treatment Group Control Group Instructor Instructor 24263

17 Example of Quasi-Experimental Design in Geography Impossible to randomly assign each student to a GIS or paper mapping conditions Randomly assigned whole classes to two conditions Different instructors affected the results differently  Instructor effect played a substantial role in student attitudes and self-efficacy

18 Ongoing Debate Whether true experiments or quasi-experiments represents the superior design Supporters of true experiments  Difficult to isolate the program effects using quasi- experiments  Quasi-experimental results are biased and sensitive to minor changes  Not sure about whether quasi-experimental designs can adequately control selection bias Hard to determine better design  True experiments are impossible and impractical in some cases

19 Any questions???


Download ppt "Quasi-Experimental Design Jung Eun (Jessie) Hong Feb. 23, 2009."

Similar presentations


Ads by Google