1. John W. Creswell, Ph.D. Professor of Educational Psychology, University of Nebraska-Lincoln, USA Founding Co-Editor, Journal of Mixed Methods Research © Please do not duplicate or use these slides without the express permission of the author. Qualitative and Mixed Methods Research Presentation at UKZN, 15 October 2008

2. Abstract Provide an introduction to qualitative and mixed methods research design Discuss how I would design a qualitative study and a mixed methods study

3. Some of my writings… Published by Sage Publications, Pearson Education (Merrill Educ.)

4. Qualitative Research

5. How would I position myself among the many qualitative camps? Philosophers Interpretive lens Methodologists and Design Specific methodologists (Asset-based research) Some combination

6. Qualitative research is an inquiry approach useful for exploring and understanding a central phenomenon. To learn about this phenomenon, the inquirer asks participants broad, general questions, collects the detailed views of participants in the form of words or images, and analyzes the information for description and themes. From this data, the researcher interprets the meaning of the information, drawing on personal reflections and past research. The final structure of the final report is flexible, and it displays the researcher’s biases and thoughts. My definition as a methodologist…

8. Trends in Qualitative Research Fragmentation into “camps” Increased interest in narrative research Increased use of computer software programs More interest in discourse analysis internationally than in the US At the margins – performance ethnography Ever-shifting paradigms New journals as outlets Inroads difficult in some disciplines My increased interest in methodology

9. Designing a Qualitative Study

10. Does the problem merit a qualitative study? Exploratory Skills Resources Audience

11. Key initial decision What is the central phenomenon? Key idea Key concept Idea you would like to explore

12. What is your title? Central phenomenon Participants Site

13. What are your qualitative research questions? Central question The overall question Relates to the purpose statement Sub questions Additional questions Relates to the Central Question

14. Use qualitative wording Begin with words such as “how,” “what,” and “why” Tell the reader what you are attempting to “discover,” “explain, “explore,” “identify,” or “describe” Avoid: words such as “relate,” “influence,” ”impact,” “affect,” “cause”

15. 15 Postpositivism  Determination  Reductionism  Empirical observation and measurement  Theory verification Constructivism  Understanding  Multiple participant meanings  Social and historical construction  Theory generation Advocacy/Participatory  Political  Empowerment issue- oriented  Collaborative  Change-oriented Pragmatism  Consequences of actions  Problem-centered  Pluralistic  Real-world practice oriented What is your paradigm perspective?

16. Will you use an interpretive lens? Advocacy lens (feminist, racial, ethnic, disability, sexual orientation) (Mertens, 2003) Social science lens (social science theory) Components: What is it Who has used it in your field How it will shape your study (rephrase your guiding research question, if a lens applies)

17. DimensionNarrativePhenomenology Grounded TheoryEthnographyCase Study Focus Exploring the life of an individual Understanding the essence of experiences about a phenomenon Developing a theory grounded from data in the field Describing and interpreting a cultural or social group Developing an in-depth analysis of a single case or multiple cases Data Collection Primary interviews and documents Long interviews with up to 10 people Interviews with 20-30 individuals to “saturate” categories and detail a theory Primarily observations and interviews with additional artifacts during extended time in the field Multiple sources including documents, archival records, interviews, observations, Physical artifacts Data Analysis Stories Epiphanies Historical content Statements Meanings Meaning themes General description of the experience Open coding Axial Coding Selective Coding Conditional Matrix Description Analysis Interpretation Description Themes Assertions Product of the study Detailed picture of an individual’s life Description of the “essence” of the experience Theory or theoretical model Description of the cultural behavior of a group or an individual In-depth study of a “case” or “cases” What qualitative design will you use?

18. Methods checklist… Qualitative Data Collection Methods: ____ Discuss the site(s) that will be studied ____ Identify permissions that have been granted (include something about IRB permissions) ____ Indicate how participants will be recruited to the study ____ N of participants ____ Discuss the type of purposeful sampling to be used (inclusion criteria) ____ Demographics of participants ____ Indicate how the participants will benefit from the study (reciprocity) ____ Indicate the types of data to be collected (perhaps a table of data collection?) ____ Indicate the extent of data collection ____ Mention the use of protocols (interview, observations, records) used to record the data) ____ State the research questions that will be asked (if interviews)

19. Methods checklist Data Analysis Methods: ____ Discuss preparing the data (transcriptions) ____ Indicate the general procedure of data analysis (reading through the data and memoing, coding the data, description, developing themes, interrelating the themes) ____ Indicate any specific procedures related to your approach to qualitative research (e.g., in grounded theory, you may use open coding, axial coding, selective coding). ____ Discuss the use of qualitative data analysis software to help analyze the data (e.g., Atlas.ti, MAXqda, etc.) ____ Discuss use of multiple coders (i.e., intercoder agreement) if used in the study and how this process was accomplished with % agreement ____ Discuss validity strategies (e.g., member checking, triangulation, negative case analysis, peer audit, external audit, immersion in the field) ____ Discuss reflexivity – how the researchers’ experiences and role will influence the interpretation of the findings.

20. How will you describe the intent of your study? The purpose of this _________(biographical, phenomenological, grounded theory, ethnographic, case) study ____(is, was, will be) to ______(understand, describe, develop, discover) the ________(central focus) for _______(unit of analysis: person, process, groups, site). At this stage in the research, the _______(central focus) will generally be defined as ________(provide a general definition of the central focus).

21. What ethical issues do you anticipate? Permissions Field issues Power issues Ownership issues

22. Let’s look at the parts of a qualitative proposal Introduction Statement of the problem (including literature) Purpose of the study Research questions Delimitations and limitations

23. A qualitative proposal (cont’d) Procedures Characteristics of qualitative research (optional) Qualitative research design Role of the researcher Data collection procedures Data analysis procedures Strategies for validating findings Narrative structure

24. A qualitative proposal (cont’d) Anticipated ethical issues Significance of the study Preliminary pilot findings Expected outcomes Appendices interview questions observational forms timeline proposed budget

25. Mixed Methods Research

26. How would I position myself as a mixed methods researcher? 1 st generation Proponent of rigorous methods Friend of qualitative research Interested in advocating for mixed methods (e.g., education, health sciences)

27. A “methods” oriented definition I use… 27 Mixed methods research is a research methodology with philosophical assumptions as well as methods of inquiry. As a methodology, it involves philosophical assumptions that guide the direction of the collection and analysis of data and the mixture of qualitative and quantitative approaches in many phases in the research process. As a method, it focuses on collecting, analyzing, and mixing both quantitative and qualitative data in a single study or series of studies. Its central premise is that the use of quantitative and qualitative approaches, in combination, provides a better understanding of research problems than either approach alone. (adapted from Creswell & Plano Clark, 2007 )

28. 28 Quantitative DataQualitative Data Methodology Method Paradigm Perspective Incorporation into other designs Mixed Methods Do we need a common understanding of what it is?

29. This leads to the essential characteristics of a mixed methods study: It is a methodology (with steps spanning the entire process of research). It is premised on the idea that quantitative and qualitative research, in combination, provide a better understanding of research problems than either forms of research by itself (more evidence, more convincing arguments from different perspectives). It involves collecting two types of data – quantitative and qualitative. It involves the integration of these two types of data or research approaches. The procedures can be described as mixed methods designs in which data are collected concurrently, sequentially, or both; and in which emphasis given to quantitative and qualitative data will vary.

30. But is mixed methods really new? Prior to 1988 Quantitative and qualitative research used in studies Researchers often collected both strands Approach used in disciplines/fields such as evaluation, marketing Since 1988 More systematic use of both strands (calling it a distinct methodology) Specific language for discussing it has emerged (name, terms, diagrams of designs) Specific procedures for “mixing” have developed (designs, mixed methods questions, challenges to designs) Formation of international community using this approach (journals, conferences, books)

31. Designing a mixed methods study

32. Create a working title Writing the title Short Topic Participants Include the words “Mixed methods” Neutral –neither quan or qual 32

33. Pose the general question to be answered Write it as a question Look to see how it is phrased Make sure that it is specific enough and focused (an answerable question) Ask yourself, “when I end the study, what question would like to have answered?” 33

34. List your types of data collection (a review) Quantitative data (closed-ended) Instruments Behavioral checklists Records Qualitative data (open- ended) Interviews Observations Documents Audio-visual materials 34

35. List your approach to data analysis (a review) Qualitative analysis Use text and images, For coding For theme development For relating themes Design-type Quantitative analysis Use statistical analysis, For description For comparing groups For relating variables Design-type 35

36. 36 Postpositivism  Determination  Reductionism  Empirical observation and measurement  Theory verification Constructivism  Understanding  Multiple participant meanings  Social and historical construction  Theory generation Advocacy/Participatory  Political  Empowerment issue- oriented  Collaborative  Change-oriented Pragmatism  Consequences of actions  Problem-centered  Pluralistic  Real-world practice oriented Four worldviews

37. Make explicit your interpretive lens (theory) Advocacy lens (feminist, racial, ethnic, disability, sexual orientation) (Mertens, 2003) Social science lens (social science theory) Components: What is it Who has used it in your field How it will shape your study (rephrase your guiding research question, if a lens applies) 37

38. State your procedures (methods) Procedures for handling your qualitative and quantitative data Sequence – concurrent or sequential or both Emphasis – emphasis on qualitative or quantitative Sometimes both concurrent and sequential phases are used Designs may include more than two phases Think about using a simple, elegant design 38

39. 39 QUAN Data & Results Interpretation QUAL Data & Results QUAN Pre-test Data & Results QUAN Post-test Data & Results Intervention qual Process Interpretation Triangulation Design Embedded Design Concurrent Mixed Methods Designs Parsimonious designs (Creswell & Plano Clark, 2007)

40. 40 Sequential designs QUAN Data & Results Interpretation qual Data & Results Following up QUAL Data & Results quan Data & Results Interpretation Building to Before- intervention qual QUAN Intervention Trial QUAN Intervention Trial After- intervention qual Interpretation Exploratory Design Explanatory Design Sequential Embedded Design

41. 41 Design Name Equal priorityQUAN emphasisQUAL emphasis Concurrent, triangulationQUAL+QUANQUAN+qualQUAL+quan Concurrent, embeddedn/aQUAN(qual)QUAL(quan) Explanatory, sequential, quan first QUAN  QUALQUAN  qualquan  QUAL Exploratory sequential, qual first QUAL  QUANqual  QUANQUAL  quan Sequential, embedded n/a(qual) QUAN QUAN (qual) (quan) QUAL QUAL (quan) Design options

43. Explanatory sequential design Quantitative Data* Number of cigarettes CES - D6 Qualitative Data* Semi - structured interviews, audio recorded and transcribed * Data collected 10 times over the course of a calendar year for 40 participants Quantitative Analysis Graphic plot of CES - D6 scores over time for each participant Graphic plot of cigarettes/day values over time for each participant Case Selection Qualitative Analysis Description of each case Identification of life events occurring during critical months where smoking increased or decreased Thematic analysis of life events for each case Cross - case thematic analysis Interpretation Why did changes in smoking occur? Quantitative Data Collection (quan) Qualitative Data Collection Interpretation – based on quan ad QUAL results Qualitative Data Analysis (QUAL) Case Selection Quantitative Data Analysis Selected 5 cases maximally varying Identified critical months in which smoking varied Source: Creswell, Plano Clark, Shope, McVea. (in progress) +

44. Phase I Qualitative Research - Year 1 Qualitative Data Collection Unstructured Interviews - 50 participants 8 observations at the site 16 documents Qualitative Data Analysis Text Analysis: Using QSR N6 Qualitative Findings Development of codes and themes for each site Phase II Quantitative Research - Year 2 Quantitative Instrument Development Quantitative Test of the Instrument Create approximately a 80-item instrument plus demographics Administer survey to 500 individuals Determine factor structure of items and conduct reliability analysis for scales Quantitative Results Determine how groups differ using ANOVA test Exploratory sequential design

45. 45 Process – collection and analysis of qualitative data (before, during, after trial ) Embedded research design Experiment Intervention QUAN Data collection Pre-test QUAN Data collection Post-test

46. Methodological issues Concurrent Designs Use strategies to explore contradictory findings Use parallel questions Select sub-sample of quantitative for qualitative Be sensitive to bias from one data collection to the other Sequential designs In Explanatory Design, select qual sub-sample from quan sample In Explanatory Design, consider alternatives for followup qual sampling In Exploratory Design, samples can differ In Exploratory Instrument Design, consider qual data analysis approaches for developing instrument 46

47. 47 This mixed methods study will address _________________ (overall content-aim of the study). A triangulation mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results). Write a purpose statement for a triangulation design

48. Write a purpose statement for a embedded design This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose). 48

49. 49 Write a purpose statement for an explanatory design This study will address _______________ (content-aim of the study). An explanatory mixed methods design will be used, and it will involve collecting qualitative data after a quantitative phase in order to explain or follow up on the quantitative data in more depth. In the first quantitative phase of the study, ______________ instrument data to be collected from _______________ (participants) at ___________ (research site) to test _______ (the theory) that explains why ______________ independent variables) relate to the ____________ (dependent variables). The second qualitative phase will be conducted because ______________ (intent of the qualitative phase). In this exploratory follow-up, the ______________ (central phenomenon) will be tentatively explored with ___________ (participants) at _____________ (the research site). The reason for the exploratory follow-up is to _______________ (e.g., to help explain or build upon initial quantitative results).

50. 50 This study addresses _________________ (content-area of the study). The purpose of this exploratory sequential design will be to __________________ (e.g., develop an test an instrument, generate a taxonomy). The first phase of the study will be a qualitative exploration of ________________ (the central phenomenon) by collecting ___________________ (types of data) from _________________ (participants) at _____________ (the research site). The second quantitative phase will follow up on the qualitative phase for the purpose of __________(intent of this followup). In the quantitative phase, ___________ (instrument data) will be collected from ____________ (participants) at ______________ (research site). Quantitative research questions/hypotheses will be formulated after the completion of the initial qualitative phase. The reason for collecting qualitative data initially is that _____________________ (e.g., instruments are not available, variables are not known, there is little guiding theory). Write a purpose statement for an exploratory design

51. Qualitative research questions Qualitative central question Begin with “what” or “how” Focus on single phenomenon Use exploratory verbs (discover, understand, explore) Non-directional language A general question (allowing participants’ perspectives to emerge) 51

52. Quantitative research questions Can be hypotheses or questions State variables – independent, dependent, mediating, covariates Develop from theory Use distinct measures for independent and dependent variables Order variables from independent to dependent 52

53. Writing research questions/hypotheses in mixed methods research Write qualitative research questions and write quantitative research questions/hypotheses Also write a mixed methods research question Write these questions separately 53

54. Developing a new type of research question: A mixed methods question Three ways to write this question: Methodologically-focused: To what extent do the qualitative results confirm the quantitative results? Content-focused: How do the interviews with adolescent boys support the quantitative results that their self-esteem changes during the middle school years? Hybrid of quantitative and qualitative elements: What results emerge from comparing the exploratory qualitative data about boy’s self-esteem with outcome quantitative instrument data measured on a self-esteem instrument?

55. Order the topics for your plan Title Introduction (Problem) Worldview/theoretical lens Audience Purpose Statement Research Questions (Literature Review) Methods Type of Mixed Methods Design (also add definition of mixed methods) Types of Data Collection Types of Data Analysis Sequence/Emphasis/ Mixing Procedures Figure of procedures Anticipated methodological issues Ethical issues anticipated Validity issues Researcher resources and skills References, Appendices 55

56. Additional Readings Articles and Chapters: Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed- method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2), 195-207. Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage. Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in mixed methods research. London: Sage. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3), 255-274. Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1, 48-76. Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120-123. 56

57. Additional Readings Books: Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage. Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods approaches (3 rd ed.). Thousand Oaks, CA: Sage Publications. Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass. Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2 nd ed.). Thousand Oaks, CA: Sage Publications. Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage. Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications. Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications. 57

58. John W. Creswell, Ph.D. Professor of Educational Psychology, University of Nebraska-Lincoln, USA Founding Co-Editor, Journal of Mixed Methods Research © Please do not duplicate or use these slides without the express permission of the author. Qualitative and Mixed Methods Research Presentation at UKZN, 15 October 2008