1. Research Design Part II: Cross-sectional and Quasi-Experimental Designs
Frankfort-Nachmias & Nachmias (Chapter 6 – Cross-Sectional and Quasi-Experimental Designs)
Gerring (Chapter 8)
Campbell and Stanley, “Experimental and Quasi-Experimental Designs for Research.” (remainder)
King, Keohane and Verba (Chapter 5, sections )
Applications
Stack, S. and Gunlach, J. (1992) “The Effect of Country Music on Suicide.” Social Forces 71: 211–18.
Lawrence S. Rothenberg; Mitchell S. Sanders, “Severing the Electoral Connection: Shirking in the Contemporary Congress.” American Journal of Political Science, Vol. 44, No. 2. (Apr., 2000), pp (Difference-in-difference)  

2. Writing a Literature Review
Typical format for research article
Introduction
Literature Review
Theory
Research Design
Empirical Analysis and Results
Conclusion

3. Writing a Literature Review
Purpose of literature review
Inform reader of prior relevant work
Persuade reader that your work is important (justify your research)

4. Possible Justifications for Your Research
New question, new theory
New question, existing theory
Old question, new theory
Old question, conflicting theories
Old question, conflicting findings
Old question, new methods
Old question, new data

5. Literature Review Don’ts
Don’t (just) provide a chronological listing of article summaries
Don’t provide every detail of every study

6. Literature Review Do’s
Organize your discussion of the literature in a way that reflects and supports the justification for your research
Provide more detail for seminal studies, less detail (or simply a citation) for less cited studies
For questions that have been studied extensively, it is not necessary to cite every study
End your literature review with a summary and critique that justifies your research

7. Literature Review Assignment
Approx pages, double-spaced
Due November 2nd

8. How to Identify the Relevant Literature
Use electronic databases and keyword searches (Google Scholar, JSTOR)
Prioritize:
Articles
Articles published in highly-ranked journals
Recently published articles
Seminal articles
Number of articles: 10-20?

9. Quasi-experimental and Cross-Sectional Designs
What are they?
Quasi-experimental – study of more than one sample (often over period of time)
Cross-sectional – Analysis of a single sample (lacks random assignment, temporal variation, and manipulation); but includes comparison groups
Pre-experimental – Cross-sectional, with no comparison group; causal inference impossible

10. Quasi-experimental and Cross-Sectional Designs
Why?
Property-disposition relationship
vs.
Stimulus-response relationship

11. Pre-Experimental Designs
One-Shot Case Study
One group
No variation in independent variable
X O

12. Pre-Experimental Designs
Example:
Dependent variable: Americans’ support for campaign finance reform
Independent variable: Watergate scandal
Data: 1976 survey of American adults; examine mean level of support

13. Cross-Sectional Designs
Static-Group Comparison Design
Two groups – observed at one time
Allows variation in the independent variable
X O1 O2

14. Elaboration of Static Group Comparison Design
Correlational / Cross-Sectional Designs
X1 O1
X2 O2
X3 O3
X4 O4
Xi Oi
Problems with correlational/cross-sectional designs?

15. Example: Wine and Health
Hypothesis: Drinking wine causes individuals to be healthier (esp. heart)
Existing studies: compared the health of wine drinkers to the health of those who do not drink wine:
Research design
X1 (Wine drinkers) O1 (Health)
X2 (Non-drinkers) O2 (Health)

16. Spurious Results?

17. Controlling for Affluence
Research design:
X1 (Affluent Wine drinkers) O1 (Health)
X2 (Affluent Non-drinkers) O2 (Health)
X3 (Poor Wine drinkers) O3 (Health)
X4 (Poor Non-drinkers) O4 (Health)

18. Another Example (?)

19. Quasi-Experimental Designs
Contrasted Groups Design
Multiple groups, based on some categorical variable
Observed at one point in time (similar to cross-sectional) O1 O2 O3 O4 Oi
Problems with contrasted groups designs?

20. Quasi-Experimental Designs
One-Group Pretest-Posttest Design
One group
Allows variation in the independent variable
O1 X O2

21. Elaborations of the One-Group Pretest-Posttest Design
Time Series Designs
simple vs. extended
O1…Ok X Ok+1…Om
k = # of pretest observations
m = total # observations
Or (“equivalent time samples design”)
X1 O1 X2 O2 X3 O3 … Xm Om

23. Example: Murray’s “Poverty-Spending Paradox” (Schram 1991)

24. Example: Murray’s “Poverty-Spending Paradox” (Schram 1991)

25. Nonequivalent Control Group Design
No random assignment
O1 X O2
O O4 or
O1 X1 O2
O3 X2 O4

26. Control Series Designs
Addition of second (control) group to time series design (CS: “multiple time series design”)
O1 O2 O3 X1 O4 O5 O6……
O7 O8 O9 X2 O10 O11 O12……

27. Panel Designs Repeated observations of the same units over time
Also goes by:
Pooled time-series design
Pooled cross-sectional time-series design

28. The Effect of Sanctioning in the TANF Program

29. The Effect of Sanctioning in the TANF Program

32. Donahue and Levitt
Hypothesis: The (legal) availability of abortion in a state is negatively related to the crime rate (many years later).
O1 O2 O3 X1(early legalization) O4 O5 O6…
O7 O8 O9 X2 (legalize 1973) O10 O11 O12…

34. Gerring’s Criteria for Research Design
Plentitude (N)
Boundedness (relevant cases)
Comparability (descriptive/causal)
Independence
Representativeness
Variation (X, Y, X&Y)
Analytic Utility (of the sample)
Replicability
Mechanism
Causal Comparison

35. KKV – Overcoming Common Problems
Omitted Variable Bias
Inclusion of Irrelevant Variables
Endogeneity
Assigning Values of the IV’s
Controlling the Research Situation

36. Does Country Music Cause Suicide?

37. Stack & Gundlach
Hypothesis: There is a positive relationship between exposure to country music and suicide rates
Research design:
X1 (no country music) O1 (suicide rate)
X2 (1 station) O2 (suicide rate)
X3 (2 stations) O3 (suicide rate)
X4 (3 stations) O4 (suicide rate)
Xi ( etc.) Oi (suicide rate)

38. Stack & Gundlach
Findings: 51% of the variation in urban white suicide rates can be explained by variation in airtime devoted to country music
Internal Validity?

45. O1A RetiredA O4A
O2B SW-OfficeB O5B
O3C ReturnedC O6C

47. Regression Discontinuity Designs
Example: What is the effect of an award on later achievement? (from CS)

48. Regression Discontinuity Designs
Example: What is the effect of an award on later achievement? (from CS)
Inferential challenge: Award recipients are likely to do well anyway, even without the award, because criteria for receiving award also predict future success

49. A regression discontinuity design is appropriate for any research design in which the assignment of the treatment is determined by a continuous variable that is also related to the outcome of interest.

50. Ludwig and Miller, 2007

51. Head Start
Head Start is a federal program that provides comprehensive education, health, nutrition, and parent involvement services to low-income children and their families.
Head Start began in 1965
Local nonprofits/education agencies must apply to federal government to be a Head Start provider

52. Head Start
Research Question: What is the effect of Head Start on health/well-being of former participants?
What problems might we encounter if we were to conduct the following study:
X O (where X = Head Start participation) O

53. Ludwig and Miller, 2007
OEO and application assistance to poor counties in 1965
“Treatment” group – 300 poorest counties that received head start application assistance
“Control” group – 228 poor counties that did not