1. Qualitative Research Methods and Data Collection
Research and Development in Medical Education:
Educational Skills Workshop
Arianne Teherani, PhD
Bridget O’Brien, PhD

2. Creative Commons License
Attribution-NonCommercial-Share Alike 3.0 Unported
You are free:
to Share — to copy, distribute and transmit the work
to Remix — to adapt the work
Under the following conditions:
Attribution. You must give the original authors credit (but not in any way that suggests that they endorse you or your use of the work).
Noncommercial. You may not use this work for commercial purposes.
Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one.
See for full license. 2

3. Overview of Workshop Evaluation of qualitative research
Definition of qualitative methods
Types of qualitative data
Design of qualitative data collection methods
Overview of qualitative research will address underlying assumptions of qual research (slides 3, 4, 5, 6). Slide 5 is a brief comparison between quan and qual research. Slide 6 includes sample qualitative questions

4. Qualitative Research
Research about lives, behavior, organizational functioning, interactional relationships
Exploratory
Open-ended
Data = Words
Analytic procedures are interpretivist
Analysis is concurrent with and post data collection
-Research about perceptions, beliefs, behavior, social interactions and relationships, organizational processes
-often exploratory or descriptive
-Data=descriptions, narrative accounts, observed behaviors
-Analytic procedures are interpretive and interact with data collection (moving back and forth between development of concepts or frameworks and inquiry through data collection)
-Data collection is complete when saturation is achieved – no new themes, no new areas of inquiry

5. Multiple Purposes of Qualitative Methods
Research
Evaluation
Instrument Development
Curriculum Development

6. Qualitative Research Questions
What are the qualities of an ideal mentor?
How are attending physicians prepared and trained to perform the tasks and duties after transitioning to their new positions?
How do clerkship students characterize the struggles they face when transitioning to the clerkships? How do clinical teachers characterize students’ struggles?
How do residents describe the contributions of their fourth year experiences to their professional development.

7. Qualitative Paradigms
Sets of beliefs that guide action
Qualitative research takes place within a vast spectrum of paradigms:
Positivism
Post-positivism
Interpretivism / Constructivism
Critical Theory
*Ginsburg article mentions a constructivist grounded theory approach.
*NOTE: Often authors are not explicit about this and may not even be mindful of the paradigm. But it is important to be mindful of ontology (what is reality), epistemology (what is the nature of knowledge), and methodology (research approach). For our purposes, the distinction btwn post-positivist and interpretive / constructive are most relevant. PP is hypothesis testing – assumes there is one truth / reality, but we can’t know it completely so we make hypotheses and measure as precisely as possible to approximate truth. Strive for objectivity. Constructivist assumes multiple realities and truths, knowledge is subjective, tries to capture multiple different perspectives and interpretations; inductive approach, purpose is understanding.
See: Bunniss & Kelly. Research paradigms in medical education. Medical Education, 2010, 44:

8. Qualitative Approaches
Explore social processes through interpretation of data
Four Most Common in Medical Education:
General Inductive Approach
Ethnography
Grounded theory
Case study
Phenomenology / Phenomenography
Hermeneutics
Narrative research
Action research
NOTES on general inductive approach: Often used to analyze evaluation data i.e. responses to open ended questions on evaluation forms; notes or transcripts from focus groups
No pre-existing framework or hypothesis, though may be organized around a set of questions
Purpose is typically to condense extensive and varied raw data (comments, transcripts, notes) into a brief summary or key points
Involves detailed review of raw data for frequent, dominant, or significant themes
(REF: Thomas, 2006)
Ethnography – purpose is to understand the nature of the culture (or certain cultural elements such as notions, beliefs, representations) of a group based on the point of view of different members, on observations of their actions and how they function, and/or on analysis of relevant documents.
Phenomenology – purpose is to understand the meaning or significance of a phenomenon based on the experience of those who go through it (e.g. continuity with a patient, reflective writing, witnessing or contributing to a medical error)
Grounded Theory – purpose is to understand a process or interaction (e.g. clinical teaching, formation of professional identity, development of a new skill or competency) – no theory or hypothesis guides the work.

9. Sampling
“Participants are not recruited on a representative basis, but rather because of their expert knowledge of the phenomenon under inquiry” (Green & Thorogood, 2005)
Purposeful
Seeking the best sources of information about the phenomenon of interest
Theoretical
Seeking the best sources of information to confirm / disconfirm your developing explanatory model
Ginsburg article mentions: Sample size was estimated to be sufficient based on the principle of theoretical saturation and our previous experience with this methodology (i.e. relatively homogenous population). ALSO note: invited all students in the class and took first 15 volunteers (not sure this is really theoretical sampling? I guess because there is no theory to expect differences?)
Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2005.

10. Qualitative Data Collection
Observation
Interview
Focus groups
Open-ended prompts
Artifacts (documents, photos)

11. Observation
Spend time with a group of people as they carry out their daily activities, to understand their way of life and how they make sense of the world in which they live.
To understand experiences of group members, their activities, interactions, discussions
To understand how contextual factors influence the activities and decisions of group members
Example: How do students participate in ambulatory clinic? What influences students’ level of participation? -

12. Observation Types of observation Field Notes Sketches
Relation diagrams
Role of Researcher
Continuum of roles that range from complete participation to complete observation
Type of role adopted depends on research question
To understand process of change
To generate hypotheses
Roles can change over time in research setting

13. Interviews Types of Interviews
Structured: verbal questionnaire, scripted questions
Semi-structured: outline of topics guides questions
Informal: conversational, questions arise in context
Retrospective: focuses on past events
Participants express their understandings and experiences in their own terms
Ginsburg – pretty scripted, response to video prompts.
Extent to which wording and sequencing ought to be predetermined depends on purpose of interview
To understand daily experiences and the meanings of those experiences

14. Focus Groups Small groups with a moderator
Concentrated data on topic of interest in short time span
Group interview with insight into interactions on topic (compared to interviews)
Important that topic of interest would be easy for participants to discuss in a group

15. Exercise 1: Designing Instruments
Each group will be charged with developing a qualitative instrument to answer the following research question:
How do students’ roles in patient care change over the course of the clerkship year?
Group 1: Observation tool
Group 2: Interview protocol
Group 3: Focus group protocol

16. Exercise 2: Interviewing Skills
We will role play 2 interviews between a qualitative researcher and participant.
After each interview, discuss:
How you feel the interview went
Thoughts about the questions asked and the responses elicited
What you think was done particularly well and/or what could be improved

17. Data Collection Decisions
Methods
Types of Data
How Interactive? Researcher’s Role?
Focus groups
Interviews
Open ended prompts
Observations
Collect Artifacts
Audio Recording
Notes
Transcripts
Written response
Typed response
Notes
Video
Documents
Photos / Visual
Video

18. Example: How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?
Observation of small groups pre & post
Interviews with small group leaders (retrospective comparison)
In-depth interviews with students
Focus group with students
Written evaluations of group process by students and faculty

19. Other topics / questions
Recording (audio, video)
Transcription
Software for qualitative analysis

20. Convincing a Colleague
“You are about to present the results of a survey evaluating your program to colleagues in your department. The results include both quantitative results (descriptive statistics for responses to structured questions) and an analysis of responses to open-ended questions. One of your colleagues says: “Let’s just look at the numbers, the hard data, not the anecdotes.” You would say (or do) . . .

21. Teaching Material References
Lofland & Lofland. Analyzing social settings.
Rubin, H.J. & Rubin, I.S. (1995). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications
Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage Publications
Cote, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education. Medical Teacher, 27(1),
Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2005.
Thomas D. A general inductive approach for analyzing qualitative evaluation data. Am J of Eval. 2006; 27:
Expanded bibliography distributed via handout

22. Exercise 1: Manuscript Discussion
Why was this study done? What questions or problems does the study examine?
Do you have a clear sense of the context of the study and the role of the researcher(s)?
How well do you feel the methodology used in this study captured the research question? Explain some of the strengths and/or limitations.
What other methods could have been used to study this topic?