Mixed Methods Research in Practice: Decision Making about Life Support in Intensive Care Units Douglas B. White, MD, MAS Assistant Professor of Medicine Division of Pulmonary and Critical Care Medicine Investigator, UCSF Program in Medical Ethics

Overview n What is qualitative research? n What is mixed methods research? n Criteria to evaluate the quality of mixed methods research n Examples of models of mixed methods research

n Previously healthy 71-year man admitted to the ICU with a large stroke. He develops severe pneumonia w/ resp failure, sepsis and renal failure. u Aphasic, R hemiparesis u APACHE II: 35; In-hospital mortality 70% u Significant functional impairment u Patient decisionally incapacitated

Should life support be continued? u Surrogate decision-making u No clear “right” medical answer u Preference-sensitive decision

Why study communication of prognosis? 1. Patients/Families have: u A right to know autonomy & informed DM u A need to know Prognostic info affects treatment choices 2. Prognostic misunderstandings are common

Lloyd CB. Crit Care Med 2004; 32: Prognostic Information Changes Patients Decisions about Life Support

I Shouldn't Have Had To Beg for a Prognosis With all the conflicting reports on his health, I didn't know if he was holding steady or dying. Aug. 22, 2005 issue - I was once a stalker. My victims—yes, there were several— were high on the social scale, but they were not celebrities. They were doctors.…

What causes physician-family discordance about prognosis? n No discussion about prognosis? n Poor MD communication skills? n Low health literacy/numeracy? n Undo optimism by families? n Lack of trust in physicians? n Different attitudes about whether clinicians can predict the future? n Different attitudes about what determines a patient’s prognosis Little empirical research about mechanisms

The Structure-Process-Outcome Paradigm: Prognosis Communication in the ICU Process of care: - # prognosis discussions - Process/Content of discussion Outcome MD-family agreement re: prognosis Family characteristics: - literacy/numeracy - optimism - depression -trust in physician - Explanatory models of future telling Physician characteristics: -Demographics -Skills - Attitudes

Goals of Doing Qualitative Research and the Alternatives Goals of Research n Develop hypotheses n Develop framework for understanding phenomenon n Understand multiple perspectives n Understand why patients or clinicians do what they do Alternative to Research n Make up hypothesis n Use pre-existing framework or conceptual model n Use expert or clinical perspective n Guess why they do what they do

Qualitative Methodologies n Grounded theory n Content analysis n Ethnography n Naturalistic inquiry n Discourse analysis n Phenomenology Unifying trait:Inductive reasoning n “Moving from detailed facts to general principles” Deductive reasoning: n “Reasoning from general principles to the particular” Ex.: Beta-blocker use in AMI

Questions begging for a qualitative approach n Why do patients not take their medications? n How do people make end-of-life decisions? n What are the barriers to implementing semi-recumbency to prevent pneumonia in ICUs?

A Quantitative or Qualitative Design? Quantitative n Establish incidence and prevalence n Measure risks and frequency of events n Determine treatment effectiveness Qualitative n Describe phenomenon n Understand thinking, behavior n Describe “why” interventions do or don’t work

What is Mixed Methods Research? “Research that includes both QUAL and QUANT data collection and analysis in parallel form (in which two types of data are collected and analyzed) or in sequential form (in which one type of data provides a basis for collection of another type of data). Tashakkori and Teddlie; 2003, Handbook of mixed methods in social and behavioral research

Why Use a Mixed Methods Approach? n THE reason: the research question demands it. n When neither method is sufficient to capture the trends and details of a subject. u Guide quantitative instrument development. u Triangulate findings on 1 topic from 2 methodologic approaches. u Comprehensive evaluation of an intervention.

NIH Guidelines for Rigor in Mixed Methods Research 1) What is the rationale for mixing methods? u Why valuable for the study aims? To identify questions relevant for instrument design To gain a comprehensive understanding of the phenomenon To triangulate findings (to enhance claims of validity) NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

NIH Guidelines for Rigor in Mixed Methods Research 2) Which method takes priority? u Depends on goal Develop instrument: QUAL informs QUANT goal of reliable and valid instrument. Develop theory: QUAL predominates; QUANT can provide useful ancillary data. Explain phenomenon: Equal weight to QUAL and QUANT inquiries. NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

NIH Guidelines for Rigor in Mixed Methods Research 3) How are the methods implemented? u Concurrent vs Sequential Concurrent: simultaneous QUAL and QUANT data collection Sequential: One method serves as foundation for subsequent method. –Qualitative data from semistructured interviews identify relevant domains for quantitative instrument. NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

NIH Guidelines for Rigor in Mixed Methods Research 4) When are the data integrated? During data collection: –Ex. 1: QUAL data examined, then transformed into QUANT instrument. –Ex. 2: QUAL coding of audiotapes transformed into quantitative group of predictors to determine association with outcome. During synthesis of results: –Ex. 1: Investigators use their expertise to synthesize data and draw inferences/conclusionsa. NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

Mixed Methods Designs: 1. Instrument Development n Create quantitative tools informed by qualitative methods n Pursue QUAL research to ensure comprehensiveness of instrument. QUALQUANT RESULTS

Assessing the Quality of Mixed Methods Research 1) Rationale for mixed methods: n To inductively develop an instrument 2) Priority: n Qualitative 3) Implementation: n Sequential 4) Integration: n During data collection NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

Mixed Methods Designs: 2. Explanatory Model n Qualitative data and analysis are used to explain quantitative data (e.g., patterns in the data, or the meaning of outliers) QUAL QUANT RESULTS

Example: Physician-family discordance about prognosis: Misunderstanding vs Different Belief? Background: Small qualitative studies in other domains of healthcare suggest that discordance might arise from different explanatory models of future telling. Aim: To determine whether different belief systems contribute to MD-family discordance about prognosis. Design: Prospective cohort study Subjects: 175 surrogate decision makers for ICU patients with >50% chance mortality.

Example: Physician-family discordance about prognosis: Misunderstanding vs Different Belief? MEASUREMENTS: Quantitative data: u Surrogate’s estimate of patient’s prognosis for hospital survival u Surrogate’s estimate of what the physician thinks is the prognosis. Qualitative data: u Semi-structured interview: u “I notice that your estimate is different from/the same as what you think the doctor thinks. Can you tell me a little about why?”

0% chance of survival 100% chance of survival 1.What do you think are the chances that the patient will survive this hospitalization if the current treatment plan is continued? Place a mark on the line… 0% chance of survival 100% chance of survival Outcome Measure- Prognostic Discordance What do you think the doctor thinks are the chances that the patient will survive this hospitalization if the current treatment plan is continued? Place a mark on the line…

Example: Physician-family discordance about prognosis: Misunderstanding vs Different Belief? QUALITATIVE ANALYSIS: n Interviews were audiotaped and transcribed n Coding by multidisciplinary team n Grounded theory methods u Performed open coding- line-by-line (5 transcripts) u Multiple investigator meetings  axial coding (grouping individual codes into organizing framework) u Coded 5 more transcripts  Revised codebook u Reliability: kappa > 0.60 for all codes

Example: Physician-family discordance about prognosis: Misunderstanding vs Different Belief? Quantitative Results: n 35% (50/143) held a different belief about the prognosis than their guess of the MD’s estimate. n Mean difference (absolute)=15%; (range %)

Example: Physician-family discordance about prognosis: Misunderstanding vs Different Belief? Qualitative Results: Disagreement PATIENT Physical strength Emotional strength “He’s a fighter” Past history of “beating odds” FAMILY Miracles Family presence can alter outcome Conscious optimism Need to disbelieve PHYSICIAN Overly pessimistic Lack faith Don’t know patient Can’t predict future

Assessing the Quality of Mixed Methods Research 1) Rationale for mixed methods: n To explain an observed quantitative relationship 2) Priority: n Qualitative 3) Implementation: n Concurrent 4) Integration: n During data collection and analysis NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

Mixed Methods Designs: 3. Triangulation Model n Pursue QUANT and QUAL methods in parallel n Research: Study subject using more than one way of knowing; more than one viewpoint n Enhance comprehensiveness of findings. QUALQUANT RESULTS CONCLUSIONS RESULTS SYNTHESIS

Assessing the Quality of Mixed Methods Research 1) Rationale for mixed methods: To gain a comprehensive understanding 2) Priority: Equal 3) Implementation: Concurrent 4) Integration: Analysis phase NIH. Qualitative Methods in Health Creswell JW. Ann Fam Med. 2004; 2:7-12.

Mixed Methods Designs: Data Transformation Model n Initial qualitative analysis of data to develop codes, then using descriptive statistics to describe how often they are observed n Transform QUAL findings into quantative and use conventional statistical methods. QUAL QUANT (outcome) RESULTS QUANT (predictor) Transform data

Example of Data Transformation Model: Emotional Support during ICU Family Conferences QUAL: To understand how physicians emotionally support family members making EOL decisions for ICU patients. QUANT: To determine whether specific physician behaviors are associated with higher family satisfaction.

Example: Emotional Support and Family Satisfaction during EOL Decision Making Design: Prospective multi-center, study of 51 audiotaped physician-family discussions about EOL decision-making Setting: Adult ICUs of 4 Seattle-area hospitals Measurements: QUAL: Transcripts inductively coded using grounded theory methods QUAL: Transcripts inductively coded using grounded theory methods QUANT: validated instrument assessing family satisfaction with communication QUANT: validated instrument assessing family satisfaction with communication Curtis JR. J Crit Care; 2000

Example: Emotional Support and Family Satisfaction during EOL Decision Making Qualitative Results: Support for end-of-life decisions Support for end-of-life decisions Addresses family issues Addresses patient as a person Nonabandonment All is being done Reassures about comfort Support for hopes Acknowledges emotions expressed Acknowledges the complexity Offers assistance Offers assistance Being direct about dying Being direct about dying Elicits questions Elicits questions Listens Listens Curtis JR. J Crit Care; 2000

Emotional Support Associated with Higher Family Satisfaction Higher family satisfaction when MDs: Expressed empathy (p <0.05) –“I imagine it must be hard for you to see your father this way” Assured nonabandonment (p<0.05): –“Even if we stop life support, we won’t stop taking care of her.” Assured comfort (p<0.05): –“We’ll make sure she’s not SOB and not in pain.” Expressed support for family’s decisions (p<0.05) –“I fully support this plan and this is what I would do if it were my son in the bed.” Stapleton, Crit Care Med, 2006 White DB. Under review 2007

The Effect of a Proactive Communication Strategy on Bereavement Outcomes Design: Multi-center, randomized controlled trial in 22 Adult ICUs Subjects: 126 family members of patients whom MD thought would not survive Intervention: MDs trained to do the following: Value the attitudes/decisions of the family Acknowledge family members’ emotions Listen Understand the patient as a person Elicit questions from the family Measurements: PTSD symptoms at 90 days assessed by IES (range 0-75) Results: Intervention group had significantly lower PTSD symptom scores compared to control group (27 vs 39; p=0.02) Lautrette A. NEJM 2007 ;356:

Conclusions Mixed methods studies are particularly advantageous for some, but certainly not all, topic areas “Grounds” our work, so that we ask the important questions, avoid omitting insights, and have realistic hypotheses. Increases the potential that research will be more easily translated into practice