Research design: comparative and case studies HEM 4112 – Research methods I Martina Vukasovic

Overview of session Case studies Comparative studies Causal effect and causal mechanism Methods of comparison Indeterminate research design Literature: Bryman, chapter 2; notes from the lecture; suggested advanced literature

Case study (1) Intensive study of one single community, group, individual, organization, person, event... the context is important the specific case should not be considered as a sample where N=1 Nomothetic/theory-centred (a) vs. idiographic/case-centred (b): are you interested in something that somehow corresponds to a wider set of phenomena (a) for example: a case study of organizational change (in three flagship universities in the former Yugoslavia) or are you interested in what is unique about that particular case (b) (this classification departs a bit from Bryman!)

Case study (2) Case study – research design, not research methodology Both qualitative and quantitative (and mixed) methods can be used in a case study depends on the research questions asked and on the goal of the case study But more case studies employ qualitative methods, since the study is supposed to be intensive and the context matters Focus on how and why, not just what

Case study (3) Lijphart (1971)*: Bryman: Nomothetic vs. idiographic Types of case studies (1) Lijphart (1971)*: atheoretical interpretative hypothesis-generating theory-confirming or theory-infirming deviant Bryman: critical unique revelatory exemplifying Nomothetic vs. idiographic if nomothetic: most-likely or least-likely cases of... * Lijphart, A. (1971) Comparative Politics and the Comparative Method. In The American Political Science Review, vol. 65, n. 3, pp. 682-693

Case study (4) Types of case studies (2) Atheoretical – no theory used, the goal is to provide a thick description Interpretative – the goal is also to describe, but using theory Hypothesis-generating – starts with a very vague idea about hypothesis and ends up with one or more hypotheses that should be tested in future research; similar to revelatory Theory-confirming (a) or theory-infirming (b) – tries to see if an already established theory “works” on a previously not analysed case; can result in a confirmation of a theory (a) or its modification (b) Deviant – a case that “is not behaving” as it is supposed to, the theory does not “work” and the modifications necessary to make it work are too big, can result in rejection of old and building of new theory, similar to critical case Atheortical – can you describe without any link to theory? Interpretative – theory is taken as a given, the goal is not theory testing

Case study (5) Types of case studies (3) Critical – a case in which a hypothesis does not hold, similar to deviant cases Unique – interesting because of peculiarities of the case/context Revelatory – a context/phenomena which was not accessible for studying before, could be used for hypothesis building Exemplifying – similar to a nomothetic case study, interesting because it sheds more light on some phenomena

Case study (6) Types of case studies (4) Most likely vs. least likely important for assessing whether or not (and how much) you can generalize from a case study Most likely case of... A case that is most likely to be in line with a theoretical perspective If it does  ... So what? If it does not  ..... Yippieeee! Least likely case of... A case that is least likely to be in line with a theoretical perspective If it does not  ... So what? If it does  ..... Yippieeee! Generalizing from a case study primarily depends on whether or not that is your goal, i.e. do you want to use the case study to make a contribution to the theoretical perspectives of a certain phenomenon What does this imply for ontological considerations in a given case study?

Case study (7) Types of case studies (5) Most likely vs. least likely, an example: “Assumption”: flagship universities as least likely cases of organizational change in higher education institutions Hypothesis: change in universities in the post-Communist countries is determined by the pressures coming from the European level (e.g. Bologna Process) Assumption + hypothesis  if hypothesis is confirmed for flagship universities – if flagship universities change due to the pressure coming from the European level – there is sufficient ground to claim that it will be confirmed for other higher education institutions Flagship universities (public, oldest, biggest universities in their respective systems) are least likely cases of organizational change (in comparison to other higher education institutions in their respective systems) because: they are most bottom heavy - they have a more complex structure, are bigger and their academic staff is more prone to resist change they can act as the so-called veto players, i.e. they are better position within their respective HE systems to block or counteract change pressures coming from the national (and/or international) level

Case study (8) Types of case studies (6) How to determine what type of case study are you doing? Should you? Is this essential? What are the previous classifications good for? The types are ideal types – a particular study can possibly fit in more than one type The types are more useful for understanding and explaining to others the different aspects of your study Some choices imply a particular ontological or epistemological position Back to research problem and research questions Ask yourself: how much do you use/rely on theory, why is that particular case interesting for you, what are your aims ... ? How to determine what type of case study you are doing? Should you determine that? What are the previously discussed classifications good for?

Comparative study (1) A comparison of two or more instances of (the same) phenomena “two or more instances” similar to “two or more cases” or “two or more contexts” Cases need to be comparable you need to be able to argue convincingly that the phenomena in all instances is the same or at least very similar you need to demonstrate that you used the same research approach in terms of type of data collected and methods of analysis used Strongly linked to theory

Comparative study (2) - comparability Concepts - definitions and operationalisations participation rate in HE may be defined as gross-enrolment ratio (GER), as net-enrolment ratio (NER) or in some other way Data – what is measured/observed and how? in country A GER is calculated only for universities, while in country B GER includes also students studying in the non-university sector Context - are the phenomena/process that you want to study really the same in all contexts? in country A governance of HE includes a federal, state and local level of governance (e.g. USA), while in country B it includes a supranational, national and regional level of HE (e.g. Spain) Gross-enrolment ratio (GER) – ratio between the total number of students regardless of their age in a higher education system and the total number of people that belong to a certain age group that is expected to participate in higher education (e.g. 18-24, this is different for each higher education system) Net-enrolment ratio (NER) – ratio between the total number of students of higher education age (e.g. 18-24) in a higher education system and the total number of people that belong to the same age group (18-24) HE in Spain is also influenced by European level processes (supranational level) and also regional authorities have specific competencies over higher education as well (e.g. Catalonia and Basque country enjoy significant independence with respect to the central authorities, i.e. Spanish government)

Questions to ponder Recall the distinction between objectivist and constructivist perspective as well as insider and outside perspectives (day 1) Where would you put comparative studies? Why?

Causal effect and causal mechanism (1) Comparative studies and some case studies may be constructed in terms “suspected” causes (Xs) and effects (Ys) Quantitative approach: X – independent variable, Y – dependent variable Causality Cause precedes effect There is a regular connection between cause and effect A process that embodies this connection can be identified

Causal effect and causal mechanism (2) The use of “cause” and “effect” during analysis is, strictly speaking, premature Hence “suspected” cause and sometimes “expected” effect Essentially two steps in the analysis: compare across cases to see the patterns in Xs and Y  causal effect of X and Y analyse within cases to see what steps (events, processes) link X and Y  a causal mechanism

Causal effect and causal mechanism (3) What links X and Y? How does X lead to Y? Why? A series of intermediary “events” (Is): X I1  I2  I3  ....  I10  ....  Y Usually done through a technique called process-tracing A number of sources of information can be used in process-tracing (interviews, documents, statistical data, observation...) An X and a Y can be linked with different causal mechanisms in different cases, i.e. the causal mechanism may depend on the context, BUT that does not change the causal effect

Causal effect and causal mechanism (4) Case X1 – European pressures for change X2 – national pressures for change X3 – internal/ institutional pressures for change Y – organisational change University A University B University C Often a useful tool in comparative studies – a table presenting cases, suspected causes and an effect There can be more than one effect, but the choice of Y needs to be in line with the research aim and research questions. Furthermore, the analysis of the cases and the identification of the causes is more straightforwards when only one effect is presented.

Causal effect and causal mechanism (4) Case X1 – European pressures for change X2 – national pressures for change X3 – internal/ institutional pressures for change Y – organisational change University A + University B University C - Organisational change took place in universities A and B, but not in C, according to the definition of the concept of organisational change, its operationalisation and available data

Causal effect and causal mechanism (4) Case X1 – European pressures for change X2 – national pressures for change X3 – internal/ institutional pressures for change Y – organisational change University A + University B University C - The three universities all experience change pressures from their national levels. For example, in all three countries the government has a plan to reform higher education and is implementing it.

Causal effect and causal mechanism (4) Case X1 – European pressures for change X2 – national pressures for change X3 – internal/ institutional pressures for change Y – organisational change University A + University B University C - Universities A and B experience also pressures for change from the European level. University C does not, because, for example, the country in which it is located does not participate in the Bologna Process

Causal effect and causal mechanism (4) Case X1 – European pressures for change X2 – national pressures for change X3 – internal/ institutional pressures for change Y – organisational change University A + - University B University C Universities A and B experience also pressures for change from the European level. University C does not, because, for example, the country in which it is located does not participate in the Bologna Process

Causal effect and causal mechanism (5) Words of caution (1): The table presents a simplified perspective, it should not be used as the only tool But can point towards interesting patterns Often not possible to provide a clear + or – Sometimes a better description would be e.g. strong, intermediate, weak Lead to development of QCA and fuzzy sets analysis* *Ragin, C.C. (1987) The Comparative Methods: Moving beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press

Causal effect and causal mechanism (6) Words of caution (2) Some “suspected” causes may be overlooked when designing research Some causes may “act together” – consider whether there is some theoretical grounding in “merging” these causes together A particular research design may not give clear cut indications of what are the suspects for causes

Task to work in groups Compare across cases in the examples for regularities between Xs and Y Which X you think is a good suspect for a cause of the effect Y? Case X1 X2 X3 Y A + - B C There are no internal/institutional pressures for change, i.e. there is no pressure for change coming from within the university.

Comparison methods Classical methods: MoA and MoD Comparisons of pairs of cases Method of Agreement (MoA) Two cases have the same “score” (+ or -) on the effect (Y) and different “scores” for all causes (Xs) except one Method of Difference (MoD) Two cases have different “scores” on the effect (Y) and similar “scores” for all the causes (Xs) except for one Indirect method of difference (Ragin 1987) Combination of MoA and MoD

Task to work in groups Which method(s) did you use for Task 1? Use MoA, MoD or a combination for the examples in the handout Solve the other 4 problems

Indeterminate research design for comparative studies (1) Case X1 X2 X3 Y A + - B C D Suspects for causes: lack of X2 alone OR presence of X3 alone OR lack of X2 combined with presence of X3?

Indeterminate research design for comparative studies (2) How to deal with them? Add more cases But that complicates the study Add more “suspects” for causes But that also complicates the study Check what the theory suggests Dive into within-case analysis, i.e. analyse the causal mechanisms or Conclude that the design was indeterminate and write that this is a good topic for further research 

Concluding remarks... Do not fill intimidated by –isms, -ologies etc. Research primarily requires: Discipline and systematic approach to data collection and data analysis Logical reasoning when developing inferences (think as a detective!) Strong argumentation of inferences Good writing skills