Presentation on theme: "INTRODUCTION TO RESEARCH DESIGN Jonas Pontusson 04.11.2011."— Presentation transcript:
INTRODUCTION TO RESEARCH DESIGN Jonas Pontusson 04.11.2011
what is this about? common challenges that we all face in developing and executing a research project: research questions theory building (note: focus on causal arguments) testing/evaluating theories through empirical analysis practical approach: discussion of specific examples (optional readings), mostly in the area of “comparative” but more broadly relevant key theme: negotiating trade-offs no prepared discussion of the following: “methodology” in a technical sense philosophy of science (epistemology) strictly “practical” matters (grant applications, field research, etc.) bottom line: what I think you ought to think about before embarking on your dissertation research (cumulated wisdom)
about me US-trained “comparativist” started out as qualitative work, turning quantitative quantitative macro-comparative work: incorporating insights from “qualitative tradition” recent efforts to engage “multi-level analyses:” combining individual-level survey data and macro-level variables
readings for general discussion King, Keohane and Verba (KKV), Designing Social Inquiry (DSI) (1994) bridging qualitative/quantitative divide: different data and methods, but common logic of inference telling qualitative researcher how to improve by adopting standards of qualitative social science APSR symposium: Laitin, Caporaso, Collier, Rogowski, Tarrow Brady, Collier and Seawright, eds., Rethinking Social Inquiry (2004)
KKV basic vision of how “science” work: elaborate a (casual) theory generate empirical implications (the more the merrier) test the hypotheses advice to qualitative researchers: specify clear what you are trying to explain (DV)! make sure that your measures are valid! maximize “leverage,” minimize “bias”! maximize variation on X (N), never select on the DV! estimate uncertainty!
today’s agenda 1.research questions: where do we get them from? why are some more interesting than others? 2.link between theory and research 3. analytical leverage (research design) 4. causal mechanisms 5. combining different research designs (“mixed methods”)
warning 48 slides to follow I conceived this presentation before I saw your projects: much of it is not directly relevant…
Topic #1: types of research questions open-ended questions: why Y? what are effects of X? not so interesting… testing (reasonably) well-specified theories: is the theory “true”? potentially interesting… puzzles usually interesting, at least a stepping stone…
theory-testing interest depends on (a) status of theory and (b) empirical results interesting to confirm a controversial (or counter-intuitive) theory interesting to invalidate conventional wisdom not so interesting to confirm conventional wisdom testing multiple (competing) theories is almost always interesting! critical importance of engaging with alternative explanations: who are you arguing against? why are they wrong? articulation of your own theory empirics: case selection, control variables
puzzles theory-based puzzles: theory X (“conventional wisdom”) would lead us to expect Y (outcome), but this is not what we observe… why? note: single observation can serve this purpose if the theory is sufficiently well-specified (cf. Rogowski) “real-world” puzzles: why do observe Y in case A, but not case B? note: such puzzles are only interesting if the cases are similar, i.e., share (values on) many Xs (MSSD)
framing/marketing research questions broader (theoretical) implications of narrow research questions turning topics into research questions of broad appeal... key question: what is the intellectual community that you are seeking to join? (political science?) in any case: reading broadly is crucial to effective framing!
example of puzzle-driven research project Evan Lieberman, Race and Regionalism in the Politics of Taxation (2003) Research question: Narrow version: Why has South Africa relied more heavily on income taxation than Brazil (to finance similar levels of public spending)? Broad version: What determines reliance on income taxation (relative to other revenue sources)? Point of departure: level of economic development is not an adequate explanation…
Lieberman’s historical-institutionalist account definition of “national political community” at the moment of founding as key => path dependency Brazil: federal race-inclusionary state undermines upper-class willingness to pay taxes S.A.: centralized race-exclusionary state promotes upper-class willingness to pay taxes
does the argument travel? key principle (KKV): an inductive argument cannot be tested with cases that were used to generate the argument final chapter of Lieberman’s book: cross-section regression analysis of 71 countries (cases) DV=income+property taxation in % of GDP (1990-94)
Lieberman’s process 1. scatterplot to identify exceptions to conventional explanation 2. generate new explanation by probing the experience of exceptional cases 3. test the new theory against data for many countries
Topic #2: types of explanation and their implications for research design types of explanations: frequentist vs. system-level explanations partialist vs. holistic explanationist explanations determinist vs. probabilistic explanations 2 types of research design (for now) MSSD vs MDSD (Przeworski and Teune, The Logic of Comparative Social Inquiry (1970); cf. John Stuart Mill, A System of Logic (1843)
MSSD vs. MDSD MSSD (Mill “method of difference”/matched-case comparisons) cases selected on DV (different values/outcomes) inductive approach: looking for variables (Xs) that can explain the outcome the more similar the cases, the more confident we are that (a particular) X is the cause of Y MDSD (Mill “method of agreement”/akin to regression) cases selected on IV (different values) deductive approach: prior expectation that X influences Y the more different the cases, the more confident we are that X is the cause of Y
Przeworski and Teune: MDSD better than MSSD multi-level questions: why are French workers less likely to vote Left than Swedish workers? frequentist explanation: there is some individual-level variable (e.g., religiosity) that takes on different values in the two countries (more religious workers in France) system-level explanation: the relationship between class and voting (also religion and voting) is different in the two countries MSSD immediately jumps to system-level; working in the MDSD mode, we can and should add systemic factors as necessary
Przeworski and Teune: substitute variables for proper names! macro level: continuous variables (e.g., proportionality) typological variables (e.g., PR vs SMD)
partialist vs. holistic explanations “holistic” approach (MSSD): complete explanation of Y e.g.: Immergut on health-care systems “partialist” explanation (MDSD): goal is to establish that a causal relationship exists, not to explain (all of) Y e.g., Rogowski, Commerce and Coalitions (1989)
deterministic vs. probabilistic explanations arguments for maximizing N and against selecting on the DV are premised on probabilistic theories and/or theories about marginal effects deterministic theories (events as outcomes?): (1) X is a necessary cause (condition) of Y => only cases where Y occurs are relevant (2) X is a sufficient cause (condition) of Y => only cases where X occurs are relevant
Geddes debate Geddes, “How the Cases You Choose Affect the Answers that You Get, Political Analysis (1991) critique of literature that argues (based on the experience of EA and LA NICs) that political repression promotes econ
(1) “indeterminate research design:” no inferences can be drawn
retorts 1). the theory is about necessary conditions: only upper-left space is relevant (Jordan?) 2). scope restriction: the theory is only meant to apply to EA (and LA) => Przeworki and Teune: develop a theory that transforms “EA” into a variable (or several variables)! 3). the theory is not about growth per se, but a particular kind of growth: capital-intensive industrialization (Jordan isn’t relevant) 4). there is a relationship, but it is not linear! some but not too much repression promotes industrialization
bottom line what is the scope of the theory you are testing? what kind of causal claims does it entail?
Topic #3: analytical leverage experimental and quasi-experimental designs individual variation (political behavior) variation across “macro-social units” (nations, regions/cantons) variation across policy arenas temporal variation (sequencing) prominent hybrids (in comparative): pooled cross-section time-series analysis two-level pooling of micro and macro variables
an example of qualitative case comparison Immergut, “The Rules of the Game” in Steinmo, Thelen and Longstreth, eds., Structuring Politics (1992) what is to be explained? health systems ca. 1975: Sweden: de facto national health service (i.e., socialized medicine), doctors as government employees, uniform fees Switzerland: private system, doctors free to set fees, role of government limited to subsidies to low-income people France as “half-way house:” public insurance and private provision, regulation of fees => outcomes of political struggles 1945-65
Sweden vs. Switzerland federal vs. centralized dominant party vs. permanent coalition overrepresentation of rural areas (CH) referenda as means to veto federal legislation (CH) but the cases are different on many other dimensions… how similar is similar enough?
key leverage: temporal variation in the French case Fourth Republic: weak executive party fragmentation lack of party discipline parliament as source of veto power for special interests Fifth Republic: strong executive: directly elected president, decree powers completion of healthcare reform in early 1960s
pooled CSTS analysis method of choice for quantitatively-inclined comparative political economists working on OECD countries only 18-20 countries (due to data limitations, also scope of theories) ambition to test conditional arguments theorized interaction effects (contrast QCA)
debate: Comparative Social Research, v. 24 (2007) Shalev’s critique: we fundamentally care about cross-national variation: adding cross- temporal variation does buy us more leverage the determinants of cross-sectional and cross-temporal variation are different the interaction effects involve multiple variables: something like QCA is better suited for this don’t get rid of the proper names: they contain lots of useful information!
my principles we ought to theorize about specific interaction effects, not simply infer interaction effects from the data we ought to aspire to an explanatory framework that can explain change over time as well as cross-sectional variation
addressing causal complexity within an CSTS framework are causal dynamics different within different clusters of countries? varieties of capitalism (LMEs vs. CMEs) welfare-state regimes (Esping-Andersen) PR vs. majoritarian systems do causal dynamics change over time? Kwon and Pontusson, Globalization, Labour Power and Partisan Politics Revisited,” Socio-Economic Review 8(2010):251-281.
Kwon-Pontusson theory (adaptation of Garrett): partisan conflict increases with globalization when unions are strong partisan conflict declines with globalization when unions are weak 3-way interaction of government partisanship, globalization and union strength: impossible to estimate (multicollinearity) second-best solutions…
more systematic approaches individual-level analysis of preferences process-tracing (qualitative analysis): what did actors think/say/do? sequencing (history) as key to causal inference
example of individual-level analysis Iversen and Soskice, “An Asset Theory of Social Policy Preferences,” APSR 95(2001):874-893. theory about skill specificity and demand for social insurance: specific skills provide for well-paying jobs, but finding an equivalent job is difficult => demand for insurance individual-level analysis: skill specificity associated with support for social spending the analysis doesn’t prove that insurance motives are at work, but do we really need that? from a macro-comparative point of view, this analysis speaks to causal mechanisms…
Topic #5: combining different research designs (or modes of argumentation) example 1: Lieberman (see above) example 2: Ansell, “University Challenges,” World Politics 60(2008):189- 230. 1. formal model as device to illustrate theory, generate specific hypotheses 2. quantitative analysis (pooled CSTS) 3. “stylized” case studies to illustrate (demonstrate?) causal mechanisms
Ansell higher education policy: enrollment levels and structure of funding 1.formal model: preferences of income groups => partisan preferences (tipping point) 2.quantitative analysis: pooled CSTS, OECD countries, 1980-97 3.case studies Britain, Germany and Sweden: causal mechanisms note: new research question, but building on existing theoretical framework
critique untested assumptions about policy preferences by income untested assumptions about links between income groups and parties case studies are pretty “stylized:” don’t really add a lot of new information
summing up trade-offs in matters of research design: for general discussion research as cumulative process: you build on your previous work: new research questions you build on (and challenge) the work of others (warning: previous work might be wrong, not only theoretically, but also descriptively)
additional readings on “process tracing:” Alexander George and Andrew Bennet, Case Studies and Theory Development (2005) on ‘temporality:’ Paul Pierson, Politics in Time: History, Institutions, and Social Analysis (2004)