Why do we need mixed- methods? Should we differentiate integration versus mixed-methods? Alan Bryman, Management Centre, University of Leicester, UK.

The current situation Exciting times Paradigm wars Epistemological arguments Pragmatism Washing machines and questionnaires! Not addressing epistemological issues today

My Research on Multi-Strategy Research Funded by Economic & Social Research Council – Research Methods Programme 2 main strands today: 1.Traditional content analysis of journal articles using mixed-methods research 2.Interviews with social researchers whove used it

Research methods Wanted to map general characteristics of mixed-methods research – content analysis Wanted perspectives of mixed-methods researchers + contingencies involved – semi-structured interviews

Content analysis Searched Social Sciences Citation Index for: quantitative and qualitative; triangulation; multi(-)method; mixed method in titles and abstracts of English language articles 5 disciplines: sociology; social psychology; organizational behaviour; human and cultural geography; media and cultural studies articles

Content analysis Articles foregrounded mixed-methods Emphasis on mixed-methods in terms of data collection and analysis Major focus on rationales for mixed- methods research

Classifying Forms of Mixed- Methods Research Distinction between rationale and practice Rationale = stated purpose(s) of integrating quantitative and qualitative research Practice = actual use(s) made of integrating quantitative and qualitative research Used both Greene et al. scheme and my own grounded scheme

Classifying Forms of Mixed- Methods Research Greene et al. (triangulation; complementarity; development; initiation; expansion) Parsimonious but only 2 rationales coded (primary & secondary) Developed alternative scheme

Alternative Scheme Triangulation Offset Completeness Process Different research questions Explanation Unexpected results Instrument development Sampling Credibility Context Illustration Utility Confirm & discover Diversity of views Enhancement Other/unclear/not stated

Highlights of Findings: Rationale No rationale in 27% of all articles Main categories in terms of rationale: Enhancement32% Completeness13% Sampling13% Triangulation 13%

Highlights of Findings: Practice Main categories in terms of practice: Enhancement52% (rationale 32%) Triangulation 35% (rationale 13%) Completeness29% (rationale 13%) Illustration23% (rationale 2%) Sampling19% (rationale 13%)

Highlights of Findings: Rationale and Practice Rationale and practice not always in line Rationale often not reflected in how multi- strategy research actually used Practice often doesnt chime with rationales given Examples from contingency table analysis

Highlights of Findings: Rationale and Practice – The Case of Triangulation Of the 29 articles citing triangulation as a rationale, 19 used it that way, i.e. one-third of articles citing triangulation as rationale didnt use multi-strategy research that way or didnt report doing so. Other prominent uses of articles citing triangulation were: enhance (13); completeness (10); and illustration (8).

Highlights of Findings: Rationale and Practice – The Case of Triangulation Other way around 80 articles used a triangulation approach but only 19 of them gave it as a rationale, i.e. three-quarters of articles using triangulation didnt cite it as a rationale Suggests triangulation hard to resist when opportunity arises

Highlights of Findings: Rationale and Practice – The Case of Completeness Completeness was a rationale for 31 articles and 84% of them used it that way But when practice is examined, 61% of all articles using a completeness approach didnt specify it as a rationale

Highlights of Findings: Rationale and Practice – The Case of Enhancement 73 articles specified enhance as rationale, a quarter of them didnt use multi-strategy research this way 121 articles used multi-strategy research this way, but over half of them hadnt specified it as a rationale Several other examples of mismatches

Highlights of Findings: Rationale and Practice Often mismatch between rationale and practice Mixed-methods research a moveable feast

Minority Strategy: The Gatling Gun Approach

Gatling Gun Strategy 4 or more rationales: 6 articles in terms of rationale 33 articles in terms of practice

Themes from Semi-Structured Interviews Similar to content analysis Mixed-methods research increasingly expected Concern for many Research questions important Particularistic versus universalistic discourses Not due to confusion – lack of guidelines about mixed- methods issues; textbook account too simple; ambivalence about role of research questions; lack of prescription Uncertainty

What do we mean by mixing, etc.? Mixing vs. integration Use of verbs What does bringing together of quantitative and qualitative research entail? Mixed-methods research or multi- methodology/multi-method research?

Is Integration Occurring? Content analysis findings Content analysis findings Genuine integration – 18% of articles Parallel presentation – 47% of articles Looked for evidence of findings being brought together comprehensive picture interweaving both

Is Integration Occurring? Semi-structured interviews Most expressed concern. Main themes: Different audiences Greater faith in one; also familiarity Design issues Time-lines differ Skill specialisms One more striking or interesting Objectivist vs. constructionist accounts Journal publication issues

Bryman Goes Reflexive Mine was a mixed-methods project Justified using both content analysis and qualitative interviewing quite well Outcomes consistent with rationales Mixed-methods research linked to my research questions Integration of data not adequately achieved

Back to the Title

Why do we need mixed- methods? We often dont need it But difficult to decide when we do Good deal of uncertainty about when to use a mixed-methods approach

Should we differentiate integration vs. mixed-methods? Dont get preoccupied with the verbs Interviewees saw problem Mixing findings like a conversation Forging an overall account Lack of templates

Should we differentiate integration vs. mixed-methods? Tended to emphasize doing mixed-methods research Need more attention to representation of mixed-methods findings in publications Quality criteria