1. Mapping the Landscape of Mixed Methods Research
by John W. Creswell, Ph.D.,
Department of Educational Psychology,
University of Nebraska-Lincoln
Founding Co-editor, Journal of Mixed Methods Research
University of Manitoba, Canada
March 26, 2010
© Please do not duplicate or use these slides without the express permission of the author.

2. Positioning Myself View research as set of interactive
components; not always linear
Focus on rigorous data collection and analysis
Applied research methodologist
Often work with individuals in a step-by-step process
First generation mixed methods researcher
I am still learning how to discuss/think about this topic

4. How Do We Understand the Social World and Social Life?
Quantitative methods
Trends, graphs, statistics
Cause-and-effect (factors that influence)
Comparison of groups
Frames study using theory
Qualitative
methods
Individual stories
Different perspectives
Complexity of situation
Builds up to a theory
Quantitative data
Close-ended scales
Attitudinal/behavioral scales
Behavioral checklists
Census, attendance records
Qualitative data
Open-ended responses
Semi-structured interviews
Semi-structured observations
Records/documents
Videotapes
Images such as pictures or drawings

5. The Al Gore Documentary on Global Warming

6. Stories and Trends
Whitechuck
Glacier -
1973
Whitechuck
Glacier -
2006

7. The Story of Mixed Methods Research
How mixed methods
began and initially developed
has become defined
grew in philosophical ways
developed technically
became a “movement”
has come under criticism
continues to evolve as a methodology

8. HOW MIXED METHODS RESEACH DEVELOPED AND IS DEFINED

9. How, when, and why did mixed methods begin?
Evolved over last twenty years
Quantitative research to qualitative to mixed methods
When?
Beginnings in
Jennifer Greene – USA- University of Illinois - evaluator
John Hunter and Allen Brewer – USA -Northwestern and Boston College - sociologists
Alan Bryman - England – management
John Creswell – USA – education – my chapter on combining, my teaching of proposal development class – lst generation
Fielding – sociologist in UK
Why?
Legitimacy of qualitative research
Increased interest in having two strands – quan and qual – in studies
Individuals looking for new methodologies

10. What emerged? Quantitative Data Qualitative Data Mixed Methods Method
Methodology
Paradigm
Perspective
Using mixed
methods in other designs

11. Six core characteristics of mixed methods research
The collection of both qualitative and quantitative data (open- and closed-ended) in response to research questions
The analysis of both qualitative and quantitative data
Persuasive and rigorous procedures for the qualitative and quantitative methods
The integration of these two data sources (merging, connecting, embedding)
The use of a specific mixed methods design that involves a concurrent or sequential integration (and equal or unequal emphases)
An approach to research that has a philosophical foundation

12. “Mixing” the two types of data… timing, implementation, priority
Converge data:
Qual
Results
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data

13. Typical Scenarios:
Surveys and focus groups data are merged and compared.
Survey is first completed and then focus groups used as a follow-up to explain the quantitative results.
Focus groups are first conducted. Information learned is then used to construct an instrument for a follow-up with a random sample.
An experiment is conducted and qualitative data are collected before the experiment begins and after the experiment has concluded.
A longitudinal study is underway with multiple studies organized to address a single overarching research objective. The studies are both quantitative and qualitative .

14. A primer on quantitative research
Researcher makes decisions before study (variables, hypotheses, instruments)
Researcher reduced the inquiry to a small number of variables to study and a large number of people
Researcher asks specific closed-ended questions
Researcher analyzes numbers
Researcher places importance on reliability, validity, generalizability, replicability, control, and lack of bias
Researcher writes the report using a standard structure

15. A primer on qualitative research
Researcher makes decisions based on views of participants
Researcher opens the inquiry up to understand the complexity of the situation
Researcher asks open-ended questions – single phenomenon
Researcher analyzes words and images
Researcher places emphasis on individual meaning, context, and self-reflexivity
Researcher writes the report using a flexible, literary structure

16. Reasons for “mixing”
The insufficient argument – either quantitative or qualitative may be insufficient by itself
Multiple angles argument – quantitative and qualitative approaches provide different “pictures”
The more-evidence-the-better argument – combined quantitative and qualitative provides more evidence
Community of practice argument – mixed methods may be the preferred approach within a scholarly community
Eager-to-learn argument – it is the latest methodology
“Its intuitive” argument – it mirrors “real life” 16

17. PHILOSOPHICAL DEVELOPMENTS

18. Empirical observation and measurement Theory verification
Four paradigm perspectives
Postpositivism
Determination
Reductionism
Empirical observation and measurement
Theory verification
Constructivism
Understanding
Multiple participant meanings
Social and historical construction
Theory generation
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented 18 18

19. What are the paradigm stances in mixed methods?
Paradigms cannot be mixed – hence mixed methods is untenable (“incompatibility” thesis)
Multiple paradigms can be used – but they need to be explicit and honored – and they lead to new insights (“dialectic” – Greene, 2007)
Multiple paradigms can be used and they relate to design and stages in the research (Creswell and Plano Clark, 2007)
There is one paradigm for mixed methods (pragmatism or transformative – Tashakkori and Teddlie, Mertens)
The paradigm to use is based on the researcher’s community of scholars (Morgan)

20. The fit between paradigm and designs stance (Creswell & Plano Clark, 2007)
Postpositivist Paradigm
Constructivist Paradigm
Focus Group
Interviews
Survey 20

21. THE DEVELOPMENT OF TECHNIQUES

22. Technique Developments
Types of designs; diagrams, detailed procedures, notation
Scripts for purpose statements
Mixed methods research questions
Analysis strategies for merging data
Point of interface strategies for sequential data

23. Concurrent Mixed Methods Designs
Types of Mixed Methods Designs
Concurrent Mixed Methods Designs
Convergent Parallel Design
QUAN Data & Results
Interpretation
QUAL Data & Results
Embedded Design
QUAN Pre-test Data & Results
QUAN Post-test Data & Results
Intervention
qual Process
Interpretation 23

24. Before-intervention qual After-intervention qual
Sequential Designs Mixed Methods Designs
Explanatory Design
QUAN Data & Results
Interpretation
qual Data & Results
Following up
Exploratory Design
QUAL Data & Results
quan Data & Results
Interpretation
Building to
Sequential Embedded Design
Before-intervention qual
QUAN Intervention
Trial
After-intervention qual
Interpretation 24

25. Emancipatory Theory Design
A Framework
(Participatory action research, community health research, critical theory, case study, narrative study, experimental study, etc. )
QUAL Data & Results
quan Data & Results
Interpretation
Building to OR
QUAN Data & Results
Interpretation
QUAL Data & Results

26. Multiphase (or Multi-project) Design
QUAN Data & Results
Interpretation
QUAL Data & Results
QUAL Data & Results
quan Data & Results
Interpretation
Building to

27. Detailed diagram 27 27

29. Write a Purpose Statement for a Convergent Design
Intent
This mixed methods study will address _________________ (overall content-aim of the study). A convergent mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results).
Design
Data
Purpose
Rationale

30. A new type of research question: A mixed methods question
Three ways to write this question:
Methodologically-focused:
To what extent do the qualitative results confirm the quantitative results?
Content-focused:
How do the interviews with adolescent boys support the quantitative results that their self-esteem changes during the middle school years?
Hybrid of quantitative and qualitative elements:
What results emerge from comparing the exploratory qualitative data about boy’s self-esteem with outcome quantitative instrument data measured on a self-esteem instrument?

31. Example of Joint Display (Li, Marquart, & Zercher, 2000)

32. Example of Joint Display (Lee & Greene, 2007)

33. Point of interface strategies in sequential designs
Exploratory Sequential Design = developing and testing an
instrument
Phase I
Qualitative-
Exploring a topic
Phase II
Developing an
instrument
Phase III
Administering
the instrument
Quotes=items
Codes =variables
Themes=scales
Testing
psychometric
properties

34. THE “MOVEMENT” OF MIXED METHODS RESEARCH

35. Describing mixed methods as a “movement.” “The emergence of mixed
methods as a third
methodological movement
in the social and behavioral
sciences began during the
1980’s.” (p. 697)
-Tashakkori &
Teddlie (2003) 35 35

36. Tracking empirical studies…
Selected Mixed Methods Studies in Trauma Research (Creswell & Zhang, in progress)
Authors
(Year)
Journal
Topic
Quan Data
Qual Data
Integration
Impact Factor
Jones, Kafetsios
(2005)
Clinical Child Psychology and Psychiatry
Political Violence and Psychological Well-being
HTQ
and
HSCL-25
Open-ended questions and observations
Quan results are explored in a qual followup
4.434
Monneuse et al.
(2008)
Journal of the Surgeons
Attitudes about Injury among High School Students
Questionnaire
Open-ended questions
3.101
Evans
et al.
(2007)
Journal of Traumatic Stress
Intrusive Memories Related to Violent Crime
Semi-structured interview
Quan results are explored in a qual follow-up and vice versa
1.884
Yassi
(2004)
Nursing Research
Factors Associated with Staff Injuries
Telephone survey
Interview and focus group
Qual findings help to develop or select quan instruments
1.748
Feeney, Ylvisaker
(2003)
Journal of Head Trauma Rehabilitation
Context-Sensitive Behavioral Supports for Young Children with TBI
MAS
ABCI
Interview of open-ended questions
1.643
Banyard, Williams
Child Abuse & Neglect
Women’s Voice on Recovery from Child Sexual Abuse
Structured Interview
In-depth, open-ended interview
1.506
Igreja
(2006)
The Journal of Nervous and Mental Disease
Women’s Posttraumatic Suffering After the War
SRQ
In-depth interview
Comparison of the data sources
1.496
Arditti
Journal of Loss and Trauma
Family Visiting at a Local Jail
Quan results are explored in a qual followup and vice versa
0.528
Gaskell
Developmental Neurorehabilitation
Evaluating Rehabilitative Activity Holidays for Burn-injured Children
SDQ and Self-perception Profile for Children
Qual questionnaire
Comparing the data sources NA 36

37. Noting
books
available… 37 37

38. Seeing the development of mixed methods conferences…
Programme
2008
Cambridge University, UK
Seeing the development of
mixed methods conferences… 38

39. Noting journals devoted to mixed methods research…
Founding Editors: John W. Creswell
and Abbas Tashakkori
Current Editors: Max Bergman
(Switzerland), Donna Mertens (USA) 39 39

40. Additional mixed methods journals…
Field Methods
Quality and Quantity
Multiple Approaches to Research (on-line)
Other journals that publish mixed methods studies
(e.g., International Journal of Social Research Methodology)
Special issue journals 40

41. CONTROVERSIES

42. Eleven key controversies (Creswell, in press)
How should mixed methods be defined?
Does the use of “quantitative” and “qualitative” in mixed methods create an artificial and unnecessary binary distinction?
What is driving the interest in mixed methods?
Is mixed methods a “new” approach?
Can we “mix” paradigms in mixed methods?
Does mixed methods privilege post-positivism?
Is there a dominant, meta-narrative discourse emerging in mixed methods?
Should there be a bilingual nomenclature for mixed methods?
Have we arrived at an understanding of the types of research designs in mixed methods?
Is mixed methods claiming other designs as “their own” ?
What is the value added by mixed methods beyond conducting a quantitative or a qualitative study?
Source: Creswell, J. W. (in press). Controversies in mixed methods. In N. K. Denzin and Y. S. Lincoln (Eds.) SAGE handbook of qualitative research (4th ed.). Thousand Oaks, CA: Sage.

43. Some Writing on the Controversies
Is there a post-positivist leaning to mixed methods?
Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10,
Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11(3),
Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive frameworks. Contributed paper for the Mixed Methods Conference, Cambridge, University, July 10-12, 2006.
Is there a dominant discourse in mixed methods research?
Freshwater, D. (2007). Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research, 1(2),

44. Some questions I have heard at University of Manitoba
I am collecting both quantitative and qualitative data
my study, but am not calling it “mixed methods.” Should I?
What skills are essential for conducting a mixed methods
study?
I have the qualitative skills, but really I am not competent
in quantitative research, can I conduct a mixed methods
You have not highlighted the importance of research
questions that drive good research. Shouldn’t a person
know their question before they choose their method?
Most journals in my field have a 3000 word limit. How can
I publish my mixed methods study with such limitations?

45. What will hold mixed methods back?
Time/resources to conduct studies
Lack of understanding the methodology
Difficult to change research behaviors
Disagreements within the mixed methods community

46. What will push mixed methods forward?
Discipline expansion - unique discipline adoption
International expansion and cooperation - little “Atlantic gap,” little North American gap
Increased graduate student interests
Refinement and expansion of techniques
More advocates (and more critics)
The need for an empirical study of the value added by mixed methods to qualitative or quantitative

47. Summary Mixed methods is a recent methodological development
There are different philosophical stances
We know the basic feature of this approach
Many techniques for conducting mixed methods research have developed
Mixed methods has grown into an interdisciplinary, world-wide movement
Yet, critics have emerged, and challenges exist
The future is positive, but there is much work left to be done 47

48. Additional readings Books:
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications.

49. Additional readings Articles and Chapters:
Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp ). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3),
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1,
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40,

50. Mapping the Landscape of Mixed Methods Research
by John W. Creswell, Ph.D.,
Department of Educational Psychology,
University of Nebraska-Lincoln
Founding Co-editor, Journal of Mixed Methods Research
University of Manitoba, Canada
March 26, 2010
© Please do not duplicate or use these slides without the express permission of the author.