1. GS/PPAL 6200 3.00 Section N Research Methods and Information Systems February 3, 2015 Professor Brenda Spotton Visano Office: 130 McLaughlin Voice Mail: (416) 736-2100 ext. 20470 E-mail: spotton@yorku.caspotton@yorku.ca

2. Agenda Review of last class Sample size in research – how big is big enough? Small-N debate Medium-N and mixed methods

3. Review of Last Class Presentation by David Northrup and Hugh McCague from the Institute for Social Research/York University “Ice is cold” - How do we “know” this? Common "ways of knowing“ include… – Casual empirical observation, Tradition, An authority said so, Religion, Inductively (from specifics to the general), Deductively (from the general to specifics) “Knowing”: Mill’s Methods of Agreement, Difference, Agreement & Difference, Concomitant Variation Observations: What’s specific? Cases versus variables Observations: What’s general? How many observations are sufficient? Sampling Types and Strategies for Selection

4. Sampling Issues (Why Sample?) Sample Types: Representative (typical of category), Prototypical (expected to become typical), Deviant (an exception to the norm), Crucial (tests in the least favorable conditions), Archetypical (creates a category) Sample Selection: Probabilistic (random, stratified random, systematic random) or Non-probabilistic (convenience, purposive, expert or snowball) Sample Size: How many is enough?

5. Sample Size Is the Research Question privileging depth or breadth – internal or external validity? Is the researcher trying to describe a phenomenon in detail (smaller sample), develop a hypothesis about it, or to test it (larger sample)? Should the sample be weighted to reflect population differences? Link to Qualitative versus Quantitative Methods of Data Collection and Analysis If the researcher uses a smaller sample, can they make “big” conclusions…

6. Small-N and Big Conclusions The Lieberson – Savolainen Debate Why emphasize the “probabilistic” nature of research? “The value of Mill’s methods is in their capacity to eliminate a limited set of alternative causal statements.” More generally, what is the value of conducting small-N research? What are its limitations? (What’s the tradeoff between large and small sample studies?)

7. Medium – N Analyses …and Big Conclusions? Paul, Clarke, Grill, Savitsky (2013) Recall the Checklist of Questions: – What is their research question? – What is their sample and how would you characterize it (type, selection, size)? – How did they collect their data? – How do they analyze it?

8. Bivariate Relationships Factor (A) present/absent – Case outcome (x: win, y: loss) (see Table 2, p.226) A BC –> x =5, y = 0 BC –> x =3, y =22 Implicitly then by Mill’s Method of Difference, there is a probability that the absence of A (COIN force credibility) influences counter insurgency “losses”

9. And Mill’s methods? “It could be that one or more of the three critical factors are not really present in the exceptional case but they were evaluated as present based on a superficial reading of the history, which a detailed review exposes as incorrect.” GIVEN: Superficial reading of history, conclude ABC occur together with xyz AND: Exceptional case (for one factor not present): BC occur together with xyz Can we conclude anything by any of Mill’s methods?

10. And Mill’s methods? (cont’d) “Or it could be that the three critical factors are very much present, but a detailed exploration reveals…the impact of the three factors is thwarted by the presence of a fourth factor,…” GIVEN: Superficial reading of history, conclude ABC occur together with xyz AND: exceptional case reveals “thwarting factor”: ABCD does NOT occur with xyz Can we conclude anything by any of Mill’s methods?

11. Discovery and Proof What causes Counter-insurgency success? What causes major civil uprisings (revolutions)? “Mill’s method of agreement is…worthless as a method of discovery and fallacious as a canon of proof. Its true value is in its function to eliminate alternative explanations…. No factor can explain an outcome satisfactorily that is not common to all occurrences of that outcome.”