1. An Introduction to Mixed Methods Research
by John W. Creswell, Ph.D.
University of Nebraska-Lincoln
SSP, University of Nebraska-Lincoln
March 9, 2007
© Please do not duplicate or use these slides without the express permission of the author.

2. Office of Qualitative and Mixed Methods Research (OQMMR), Educational Psychology, UNL
Purpose: To provide consultation on qualitative and mixed methods research, help scholars develop proposals for funding, and help conduct and evaluate funded projects.
5 Ph.D.s: Creswell, Plano Clark, Lu, Green, Shope; 2 RA’s
Began 5 years ago
Current projects in health sciences, physics, language arts, family/child research, educational assessment, veterans affairs
Funding sources: NIH, NSF, Dept of Veterans Affairs, National Department of Education, Kellogg Foundation, Esperance Family Foundation, NE Dept of Education
23 funded projects since opening; 28 journal publications and 5 books or book chapters
Editorial Office of the Journal of Mixed Methods Research
Self-supporting research office

3. Topics that I address in my mixed methods classes and workshops:
Core idea of mixed methods research
Assumptions about research needed
When to use mixed methods research
Basic components
Mixed methods literature
Philosophical assumptions
Mixed methods designs
Criteria for choosing a design
Procedural challenges in using designs
Standards for evaluation
Future research
About 1989, researchers began talking about how to best use quantitative and qualitative data together to address research problems.
The mixed methods movement was born.
One development has been four primary designs for mixed methods studies.
This presentation tells the story of mixed methods research, how it can be defined, and the procedures for conducting it.

4. Understanding the core idea of mixed methods research
Quantitative Data
Qualitative Data
Methodology
(called Mixed
Methods Research)

5. Learning the assumptions about research needed to conduct this form of inquiry
Qualitative is legitimate
Knowledge of qualitative research.
Methodology evolves
Diversity in methodology
Times when quantitative and qualitative inadequate

6. Identifying situations in which mixed methods research is needed
You are measuring a concept on an instrument. You have a sense that scores are not telling you the entire story. If you just asked a few people about the concept you might obtain a better understanding…mixed methods research provides a more complete understanding of the research problem than either quantitative or qualitative alone.

7. Identifying situations in which mixed methods research is needed
You look over the instruments available to study a concept. They were developed from a different sample/population than the one you are studying. You consider that you will need to develop an instrument before you can administer it to your sample… Mixed methods is a methodology for developing better, more context specific instruments.
You have gathered data about a factors that predict a concept on several instruments. Although you have general information about the importance of predictors, you can only guess as to what explains why the results occurred…Mixed methods helps to explain results (or how mechanisms work) in causal models.

8. Identifying situations in which mixed methods research is needed
You are conducting an intervention study. You have an intervention that was developed by other researchers. You are not certain that it will work with the sample you are studying…Mixed methods is a way to explore first to determine if an intervention will work.
We want to evaluate the performance of an organization. This calls for understanding the expected outcomes of the organization (needs assessment), designing some instruments to measure those outcomes, and then helping to explain why the outcomes occurred…Mixed methods is an approach to tie together several steps in an evaluation process.

9. Learning the basic components involved in mixed methods research
Mixed methods research is a methodology for conducting research that involves collecting, analyzing, and integrating quantitative and qualitative research in a single study or a longitudinal program of inquiry.
The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.

10. It is a research methodology
The entire process of research
Philosophical assumptions of inquiry
Research questions
Data collection
Data analysis
Data interpretation
Data reporting

11. It involves quantitative research
A research approach that involves:
Variables, hypotheses, questions
Instruments, closed-ended questions, reliability, validity
Statistical analysis
Generalizability, replicability, control, and lack of bias
A research approach that involves:
Posing variable-oriented research questions/hypotheses
Using close-ended questions on instruments with scores that are reliable and valid
Employing statistical analysis – descriptive/inferential
Having scientific rigor through generalizability, replicability, validity, control, and lack of bias

12. It involves qualitative research
An inquiry approach which includes:
Central phenomenon
Broad, general questions
Views of participants
Reciprocity and respect
Description and themes
Interpretation
Personal reflexivity
Flexible structure
Meaning or advocate for groups/individuals
An inquiry approach in which the researcher:
The central phenomenon is one key concept the researcher wants to explore
Asks participants broad, general questions which are open-ended and allow for in-depth responses
Collects detailed views of participants in the form of words or images
Collects data with reciprocity and respect in mind
Analyzes and codes the data for description and themes
Interprets the meaning of the information drawing on personal reflections and past research
Writes a report that includes personal reflexivity and a flexible structure
Emphasizes ideas such as the social construction of meaning, advocacy, and collaboration
(adapted from Creswell, 2002, p. 58)

13. It involves collecting both quantitative and qualitative data
Quantitative data
Instruments
Checklists
Records
Qualitative data
Interviews
Observations
Documents
Audio-visual materials
Quantitative data
Close-ended scales
Attitudinal/behavioral scales
Behavioral checklists
Census, attendance records
Qualitative data
Open-ended responses
Semi-structured interviews
Semi-structured observations
Records/documents
Videotapes
Images such as pictures or drawings

14. It involves quantitative and qualitative data analysis
Quantitative analysis
Use statistical analysis,
For description
For comparing groups
For relating variables
Qualitative analysis
Use text and images,
For coding
For theme development
For relating themes
Quantitative analysis
Numeric data
Descriptive trend analysis
Hypothesis testing, effect size, interval estimates
Qualitative analysis:
Text/image data
Coding
Themes
Description
Interrelated themes to build a picture of the phenomenon.

15. It involves mixing the data
Converge data:
Qual
Results
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data

16. It may consist of a single study or multiple studies
Qual
Quan
Results
Multiple Studies:
Quan Qual Qual Quan
Study 1
Study 2
Study 3
Study 4

17. Recognizing the body of mixed methods literature
1988, 1989 – books, articles by sociologists, evaluators
Accelerated interest in 1990s
14 books
Journals
Conference papers
Methodological journal article publications
Empirical journal article publications
US and international interest

20. Editors: John W. Creswell
and Abbas Tashakkori
Managing Editor: Vicki L.
Plano Clark

21. Topics addressed in this body of literature
Paradigm use
Research designs
Threats to rigor of the designs
Sampling procedures
Data analysis approaches
Writing study aims, research questions
Notation system
Interdisciplinary team research
Writing mixed methods studies
Evaluating mixed methods studies

22. Examining the philosophical assumptions behind mixed methods research
Worldview or philosophy (e.g., attitudes and beliefs about knowledge, such as constructivism, post-positivism)
Theoretical lens (e.g., feminist, racial)
Methodological approach (e.g., experiemental, survey, ethnography, mixed methods)
Methods of data collection (e.g. interviews, focus groups)
Worldview or philosophy – these refer to beliefs or issues regarding epistemology
These are theories such as ones from the social sciences, feminist theory , racial theory
This is the research approach that is used to conduct the study such as experimental research, survey, ethnography, grounded theory
Examples of methods of data collection include interviews, checklists, instruments, interviews, mixed methods
Adapted from Crotty M. The foundations of social research: Meaning and perspective in the research process.
London: Sage, 1998.

23. Learning about aspects of a worldview or paradigm
How we see reality in this world (ontology)
How we know what we know (epistemology)
How we view the role of values (axiology)
How we conduct the procedures of research (methodology)
How we view the use of language in research (rhetorical)

24. Understanding four typical worldviews
Postpositivism
Determination
Reductionism
Empirical observation and measurement
Theory verification
Constructivism
Understanding
Multiple participant meanings
Social and historical construction
Theory generation
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented

25. Understanding the different worldview stances in mixed methods research:
Different stances:
Single worldview – pragmatism, transformative-emancipatory (advocacy/participatory)
Multiple worldviews made explicit
Worldviews change depending on type of design
There are many worldviews that can be used
Worldview are important, and we need to make them explicit and honor them (Greene & Caracelli, 1997)
In research you can use multiple worldviews (Creswell, 2003)
If you want to use only one worldview, then it is pragmatism for mixed methods (Tashakkori & Teddlie, 2003)

26. Learning about the types of mixed methods designs typically used in research
I. Triangulation Mixed Methods Design
II. Embedded Mixed Methods Design
QUAN
Data and Results
Pre-test +
QUAL
Data and
Results
Post-test
Interpretation
Qual Process

27. III. Explanatory Mixed Methods Design
IV. Exploratory Mixed Methods Design
QUAN
Data and
Results
QUAL
qual
Data and Results
quan
Follow-up
Building

28. Learning about the concurrent Triangulation Design
One-phase project
Concurrent data collection
Combine results
Intent is to merge two separate “strands” of data
Collecting both quantitative and qualitative data
Collecting these data at the same time in the research procedure
Analyzing the quantitative and qualitative data separately
Comparing or combining the results of the quantitative and qualitative analyses
Example: collect survey data (quantitative) and collect individual interviews (qualitative) and then compare the results

30. Learning about the Embedded Design
Supportive role of one type of data or both forms of data (mixed methods within an experiment, case study, narrative study, correlational design)
Concurrent or sequential data collection
One phase or two phases
Intent of enhancing the design
Collecting both quantitative and qualitative data
Collecting these data at the same time in the research procedure
Analyzing the quantitative and qualitative data separately
Comparing or combining the results of the quantitative and qualitative analyses
Example: collect survey data (quantitative) and collect individual interviews (qualitative) and then compare the results

31. and analysis of qualitative (before, during, after trial)
Embedded Research Design
Experiment
Intervention
Quan
Data collection
Pre-test
Quan
Data collection
Post-test
Process – collection
and analysis of qualitative
data
(before, during, after trial)

32. Experimental Methodology
Embedded Design within an Experiment
Flow of the experiment
Experimental Methodology
QUAN QUAN
Pre- Intervention Post- & Follow-up Follow-up
Measure Measures
Overall results
and interpretation
qual
before intervention
qual
after intervention
Procedures:
One-on-
one semi-structured
interviews
Thematic analysis
Products:
Transcripts
Developed intervention
treatment
Procedures:
Three groups: control group, compliance
Intervention group, alliance intervention
group – group comparisons
Outcome measures: 1) attitudes toward
medication 2) adherence to treatment
3) avoidance of relapse
DAI measure completed 3 times (pre, post,
& follow up)
Products:
Numerical item scores
Change scores
Test statistics
Procedures:
One-on-one semi-
structured interviews –
exiting the trial,
participants from two
experimental conditions;
Thematic analysis
Products:
Transcripts
Themes and quotes
Procedures:
Discuss treatment
effectiveness
Discuss themes in
context of interventions
and outcomes
Products:
Discussion

33. Learning about the Explanatory Sequential Design?
Sequential data collection
Two-phase project
Quantitative phase first; qualitative, second
One phase builds on other phase
Intent: to explain results or to select participants to better understand results
Viewing the study as a two-phase project
Collecting quantitative data first followed by collecting qualitative data second
Typically, a greater emphasis is placed on the quantitative data in the study
Example: You first conduct a survey and then follow up with a few individuals through interviews

34. An Explanatory Sequential Design
Quant
itative
Quantitative
Case Selection
Qualitative
Interpretation –
based on quan
and QUAL results
Data Collection
(quan)
Data Analysis
(quan)
Data Analysis
(QUAL) +
Qualitative
Data Collection
Quantitative Data*
Quantitative Analysis
Case Selection
Qualitative Analysis
Interpretation
Selected 5 cases
maximally varying
Identified critical
months in which
smoking varied
Number of cigarettes
Graphic plot of CES -
Descri
ption of each
Why did changes in
CES - D6
D6 scores over time
case
smoking occur?
for each participant
Identification of life
Qualitative Data*
Graphic plot of
events occurring
Semi -
structured
cigarettes/day values
during critical
over time for each
months where
interviews, audio
recorded and
participant
smoking increased or
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40
Cross -
case thematic
participants
analysis
Source: Creswell, Plano Clark, Shope, McVea. (in progress)

35. Learning about the Exploratory Sequential Design
Sequential data collection
Two-phase project
Qualitative phase first; quantitative, second
One phase builds on other phase
Intent: to first explore in order to develop an instrument, to identify categories, taxonomy for follow up
Viewing the study as a two-phase project
Qualitative data collection precedes quantitative data collection
Typically, greater emphasis is placed on the qualitative data in the study
Example: You collect qualitative diary entries, analyze the data for themes, and then develop an instrument based on the themes to measure attitudes on a quantitative survey administered to a large sample.

36. Exploratory Sequential Design
Phase I Qualitative Research - Year 1
Unstructured Interviews -
50 participants
8 observations at the site
16 documents
Qualitative Data Collection
Qualitative Data Analysis
Text Analysis: Using QSR N6
Development of codes and themes
for each site
Qualitative Findings
Phase II Quantitative Research - Year 2
Create approximately a 80-item
instrument plus demographics
Quantitative Instrument Development
Administer survey to 500 individuals
Determine factor structure of items and
conduct reliability analysis for scales
Quantitative Test of the Instrument
Quantitative Results
Determine how groups differ
using ANOVA test
Exploratory Sequential Design

37. Identifying criteria for choosing a design
Intent for conducting mixed methods research
Concurrent (each strand stands alone then brought together) or sequential (one strand builds on the other)
Emphasis or priority given to one strand
Nature of research questions
Resources available
Stakeholders in field
Timing or Order of Use. Do you plan to collect your quantitative and qualitative data at the same time or in sequence over time?
Priority. What emphasis will be given to the qualitative and quantitative arms of your study?
Mixing. When do you plan to mix the data?
Research objective. What design best addresses your research problem?
Resources. What design is most workable given your time and resources?

38. Understanding the procedural challenges in using the designs
Contradictory findings
Data integration
Sample selection
Sample size
Introducing bias
Time
IRB support
Study same/different participants in phases
Sample sizes in phases
Contradictory findings
How to analyze the data
How to merge quantitative/qualitative data
What criteria to use to select participants (second phase)
How to treat validity
Practical factors:
Time
Resources
Skills
Convincing audiences

39. Recognizing standards for evaluating the “quality” of a mixed methods study
Collection of both quantitative and qualitative data (in response to quantitative and qualitative questions)
Mixing of the two forms of data
Rigor of quantitative and qualitative approaches
Contribution to the mixed methods literature
Use of mixed methods terms
Did study use appropriate terminology for the title and design?
Did it provide clear research objective for using mixed methods research?
Does it contain a mixed methods purpose statement?
Does it identify types of qual and quan data to be collected and qual and quan data analysis steps?
Does it include a visual/procedural diagram of methods with timeline?
Does it use rigorous procedures for the quantitative data collection and analysis
Does it address validity considerations

40. Needed research on mixed methods
Visual diagrams and notation system
Understanding types of designs, procedures (e.g., mixed methods questions), and reasons for mixed methods
Understanding issues (threats to rigor) arising in designs
Emerging language of methodology
Adaptation to different fields of study
Worldview perspectives
Inferences in mixed methods research
Skills needed
Funding source requirements and guidelines
International perspectives
Writing mixed methods studies
Collaborative research and team research

41. Additional Readings
Books:
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Greene, J. C., & Caracelli, V. J. (Eds.) (1997). Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Evaluation, Vol San Francisco, CA: Jossey-Bass Publishers.
Mertens, D. M. (2004). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications.

42. Additional Readings Articles and Chapters:
Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp ). Thousand Oaks, CA: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3),
Morgan, D. L. (1998). Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative Health Research, 8 (3),
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40,

43. An Introduction to Mixed Methods Research
by John W. Creswell, Ph.D.
University of Nebraska-Lincoln
SSP, University of Nebraska-Lincoln
March 9, 2007
© Please do not duplicate or use these slides without the express permission of the author.