1. Observational Study Designs and Confounding Michael A. Kohn, MD, MPP 26 August 2008

2. Outline Single Sentence Study Description REVIEW Observational study designs Definition of a confounder Coping with Confounding Examples of observational designs (“Name that Design”) and approaches to the problem of confounding

3. Single-Sentence Study Description “The [cute acronym] study is a [DESIGN] study of the association between [predictor] and [outcome] in [study population].” Exercise for section today: Present your study with a sentence like this.

4. Study Design Not just a matter of semantics Weaknesses and strengths associated with each study design Different measures of disease association Worth getting right or at least thinking about

5. Study Designs Experimental -- Randomized controlled trial (next week) Observational (today’s topic) -- Cohort -- Double Cohort (risk-no risk; exposure-no exposure) -- Cross-sectional -- Case-control

6. Experimental vs. Observational Predictor = intervention (treatment or therapy): -- experiment (randomized controlled trial)* -- observational study Predictor = exposure or risk factor: -- observational study *Not all treatments require RCTs to prove effectiveness.

7. OBSERVATIONAL STUDIES Usually the only option if the predictor is a potentially harmful exposure or risk factor Even if the predictor is an intervention, an RCT may not be feasible Confounding always an issue Many observational studies also qualify as secondary data analysis More intellectually interesting than RCTs?

8. Note on Figures The schematics of observational study designs that follow assume that you are studying the association between a risk factor and development of a disease. Predictor = Risk Factor Outcome = Disease

9. Cohort Study

10. Cohort Studies Determine predictor status on all patients in a single population (defined by something other than the predictor). Exclude any patients who already have the outcome. Follow patients over time and attempt to determine outcome in all subjects Can identify individuals lost to follow up Can measure overall incidence of outcome in the population (e.g., cases/person-year) Measure of disease association is the relative risk (RR) or relative hazard (RH)

11. Double Cohort Study

12. Double Cohort (Exposure-No Exposure) Studies Sample study subjects based on predictor status. Attempt to determine outcome in all subjects over time Can identify individuals lost to follow up Cannot measure overall incidence of outcome in the population (e.g., cases/person-year) Measure of disease association is the relative risk (RR) or relative hazard (RH)

13. Cohort Studies: Sampling Frame vs. Time Frame Time Frame: All cohort studies are longitudinal (follow patients over time). Sampling Frame: Double cohort study -- samples on predictor status, i.e. there are separate predictor- positive and predictor-negative samples. Cohort study -- starts with a cross-sectional sample (a population defined by something other than the predictor variable).

14. Cohort Studies: Prospective vs. Retrospective Prospective – Predictor status collected as part of this study Retrospective – Predictor status collected by someone else in the past (another study, medical records, etc.) It’s a cohort study if you follow predictor-positive and predictor-negative subjects over time for development of the outcome and if you can identify the subjects lost to follow up. Don’t worry too much about retrospective vs. prospective!

15. Case-Control Study

16. Separately sample subjects with the outcome (cases) and without the outcome (controls) Attempt to determine predictor status on all subjects in both outcome groups Cannot identify individuals lost to follow up (no such thing as “lost to follow up”, since by definition outcome status is known) Cannot calculate prevalence (or incidence) of outcome Measure of disease association is the Odds Ratio (OR) Trying to replicate a nested case control study in which the cases and controls come from the same cohort.

17. Nested Case-Control Study

18. Cross-Sectional Study

19. Attempt to determine predictor and outcome status on all patients in a single population (defined by something other than predictor and outcome). Cannot identify individuals lost to follow up (no such thing as “lost to follow up”) Can calculate prevalence but not incidence Measure of disease association is the Relative Prevalence (RP). Time frame is the same as for a case-control study; both discussed in Hulley, Chapter 8

20. Cohort Studies Start with a Cross- Sectional Study Eliminate subjects who already have disease

21. Causal Association Between Predictor and Outcome Does predictor cause outcome? For now, not interested in studies of diagnostic/prognostic tests. (Test result does not cause outcome. See Ch. 12.)* Not interested in descriptive studies. *How many of you are doing studies of diagnostic/prognostic tests?

22. Errors in Causal Inference Conclude that predictor causes outcome when, in truth, it doesn’t (false positive study) Fail to conclude that predictor causes outcome when, in truth, it does (false negative study)

23. Sources of Error Chance Bias Confounding

24. Sources of Error: Confounding Association between predictor and outcome is “real” – but not causal.

25. Confounding In Observational Studies of a Treatment Predictor = Treatment

26. Sources of Error: Confounding Treatment Outcome Confounder Reduces Risk Association or Looks Good Cause Falsely conclude that treatment improves outcomes

27. Confounding: Vitamin E Vitamin E Fewer CAD events Diet, Exercise, Health Awareness? True Cause Association False Cause Cause

28. Confounding: Suppression or Confounding by Indication Treatment Outcome Suppressor Worsens Association Improves Cause Falsely conclude that treatment does not improve or even worsens outcome Appears to Worsen

29. Confounding by Indication : Diuretic Therapy Diuretics CVD Mortality Severity of Disease Increases Association Decreases Cause Falsely concluded that diuretics increased CVD mortality Warram JH, et al. Excess mortality associated with diuretic therapy in diabetes mellitus. Arch Intern Med 1991;151(7):1350-6. Turnbull, F., et al. (2005). "Effects of different blood pressure-lowering regimens on major cardiovascular events in individuals with and without diabetes mellitus.” Arch Intern Med 165(12): 1410-9. Appears to Worsen

30. Intermediates An intermediate on the causal pathway between predictor and outcome will look like a confounder. A cohort study of HIV-infected pregnant women shows an association between protease inhibitor-based HAART and low birth weight. Should we control for maternal weight gain during pregnancy?

31. IS IT THIS? PI-Based HAART Low Birth Weight Low maternal weight gain True Cause Association False Cause Cause

32. OR THIS? PI-Based HAART Low Birth Weight Low maternal weight gain Cause

33. Coping with Confounding Design Phase Specification Matching Instrumental Variable –Associated with the predictor of interest but not independently associated with the outcome –Look for association between instrumental variable and outcome

34. Assessing the Importance of Confounding Alternative Outcome –Caused by confounders but not by the primary predictor –If still associated with the primary predictor, confounding is a problem Alternative Predictor –Associated with same confounders as main predictor but not thought to cause the outcome –If still associated with the outcome, confounding is a problem

35. Coping with Confounding Analysis Phase* Stratification Statistical Adjustment Propensity Scores *Have to measure potential confounders

36. Examples Name that observational study design Address issue of confounding

37. AZT Prophylaxis after Needlestick in Health Care Workers The proportion receiving AZT prophylaxis was compared between health care workers who sero-converted after a needlestick and health care workers who did not seroconvert after a needlestick. Cardo et al. N Engl J Med 1997;337(21):1485-90. DESIGN?

38. AZT Prophylaxis after Needlestick in Health Care Workers Cardo et al. N Engl J Med 1997;337(21):1485-90. Seroconversion YesNo AZT Yes9 (27%)247 (36%) No24432 33679 OR =0.66 (95% CI 0.26 - 1.5) Confounding?

39. Suppression: AZT after Needlestick AZT HIV Sero- conversion Deep Stick, Visible Blood on Needle Increases Association Decreases Cause Unadjusted analysis showed statistically insignificant effect of AZT in reducing seroconversion Appears to Increase

40. AZT Prophylaxis after Needlestick in Health Care Workers Logistic Regression Model Adjusting for Deep Stick, Visible Blood on Device Cardo et al. N Engl J Med 1997;337(21):1485-90. MVOR = 0.14 (95% CI 0.03-0.47)

41. Enhancing Causal Inference Statistical Adjustment (Multivariable Modeling) Katz, M. H. (2006). Multivariable analysis : a practical guide for clinicians. Cambridge ; New York, Cambridge University Press.

42. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season The study population included almost 300,000 subjects at least 65 years old, of whom about 58% were vaccinated. Among vaccinated and unvaccinated subjects, 1.2% and 2.0% respectively died during the flu season. Nichol et al. N Engl J Med. Apr 3 2003;348(14):1322-1332. Pooled computerized data from 3 large managed care organizations. Study Design?

43. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season Could confounding explain the association between flu vaccination and decreased risk of death during the flu season?

44. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season “Healthy Vaccinee Bias” Flu Vaccination Death During Flu Season Better health, better health care, other confounders Decreases Association False Effect Cause

45. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season How might you assess the importance of confounding?

46. If worried about confounding, measure another outcome that could be caused by the confounder but NOT by the predictor of interest. If there is NO association between the predictor of interest and the alternative, causally unrelated outcome, then the relationship between the primary predictor and the outcome is more likely to be causal. Enhancing Causal Inference Measure Alternative Outcome

47. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season Flu Vaccination Death During Summer Months Better health care, other confounders Decreases Association Should Not Affect Cause If the association of flu vaccination and decreased mortality during the flu season were due to confounding, then vaccination should also appear to reduce death during the summer months. Appears to Decrease

48. “Our analysis for evidence of a healthy-vaccinee bias was negative; during the two noninfluenza periods for which we had information, vaccinated and unvaccinated persons had similar risks of hospitalization” Effect of Flu Vaccination on All-Cause Mortality During the Flu Season Nichol, K. L., J. D. Nordin, et al. (2007). "Effectiveness of influenza vaccine in the community-dwelling elderly." N Engl J Med 357(14): 1373-81. Time Period Adjusted Odds Ratio 95% CI 6/ – 9/19991.00.78 - 1.28 6/ – 9/20000.940.74 - 1.19

49. Health Professionals Follow-up Study In 1986, 39,910 male dentists, veterinarians, pharmacists, optometrists, and podiatrists filled out a questionnaire including information about Vitamin E intake (the predictor) In 1988 and 1990, vital status was determined and those who were still alive were re-contacted to see if they had developed coronary artery disease (the outcome). Death/disease rates were lower in those who took Vitamin E. Rimm, E. B., M. J. Stampfer, et al. (1993). "Vitamin E consumption and the risk of coronary heart disease in men." N Engl J Med 328(20): 1450-6. Design?

50. Health Professionals Follow-up Study Vit E – med. intake (IU/day) 6.48.511.225.2419 Multivariate Odds Ratio 1.0 (Ref) 0.900.820.770.64

51. Confounding: Vitamin E Vitamin E Fewer CAD events Diet, Exercise, Health Awareness? True Cause Association False Cause Cause

52. Health Professionals Follow-up Study Vit E – med. intake (IU/day) 6.48.511.225.2419 Multivariate Odds Ratio 1.0 (Ref) 0.900.820.770.64 Vit C – med. intake (mg/day) 921492183921,162 Multivariate Odds Ratio 1.0 (Ref) 1.021.151.020.89

53. If worried about confounding, measure another predictor that is associated with the same confounders. If this second predictor is NOT associated with the primary outcome, then the relationship between the primary predictor and the outcome is more likely to be causal. Enhancing Causal Inference Measure Alternative Predictor

54. Confounding: Alternative Predictor Vitamin C Fewer CAD events Diet, Exercise, Health Awareness? True Cause Association False Cause Cause If the confounding accounted for the benefits of Vitamin E, then Vitamin C should also appear beneficial.

55. Randomized Trials “Trump” Observational Results Combined results of 7 large-scale randomized trials of the effectiveness vitamin E in the treatment and prevention of cardiovascular disease “provide strong support for a lack of statistically significant or clinically important effects of vitamin E on cardiovascular disease.” Eidelman, R. S., D. Hollar, et al. (2004). "Randomized trials of vitamin E in the treatment and prevention of cardiovascular disease." Arch Intern Med 164(14): 1552-6.

56. Year of Visit and Provision of Analgesia in ED Patients with RLQ Pain Subjects: Patients presenting to the SFGH ED with RLQ pain in 1998 and 2003 Predictor: Year of presentation Outcome: Receipt of parenteral analgesia Neighbor ML, Baird C*, Kohn MA. Changing Opioid Use for Right Lower Quadrant Abdominal Pain in the ED. Acad Emerg Med 2005; 12(12): 1216-20.. *UCSF MSIII

57. Year of Visit and Provision of Analgesia in ED Patients with RLQ Pain Analgesia YearYesNoTotalPrevalence 2003726513753% 19984314418723% Relative Prevalence = 2.3 Design?

58. Year of Visit and Provision of Analgesia in ED Patients with RLQ Pain 1. Patients who get abdominal CTs are much more likely to get analgesia (59% vs. 22%) 2. CTs were much more common in 2003 than in 1998 (56% vs 20%) Is the increase in analgesia rates between 1998 and 2003 wholly explained by increased CT scanning?

59. Year of Visit and Provision of Analgesia in ED Patients with RLQ Pain CT YESCT NO YearAnalgNo AnalgTotalPrevYearAnalgNo AnalgTotalPrev. 2003 47307761% 2003 25356042% 1998 21173855% 1998 2212714915% Prev. Ratio1.1 Prev. Ratio2.8

60. Year of Visit and Provision of Analgesia in ED Patients with RLQ Pain Is the increase in analgesia rates between 1998 and 2003 wholly explained by increased CT scanning? NO. In the group that did not receive CTs, analgesia rate was almost 3x higher in 2003 than in 1998.

61. Enhancing Causal Inference Stratification DCR 3 rd Ed. Chapter 9 pp. 137. Also, Appendix 9A Smoking as a confounder of the relationship between coffee drinking and MI.

62. JIFee Babies born at Kaiser with neonatal hyperbilirubinemia (Bili > 25) are compared with randomly selected “controls” from the same birth cohort. Outcome measure is IQ and neurologic status at age 5 years. No difference in IQ or fraction with neurologic disability between the “case” and “control” groups. Newman, T. B., P. Liljestrand, et al. (2006). "Outcomes among newborns with total serum bilirubin levels of 25 mg per deciliter or more." N Engl J Med 354(18): 1889-900. Design?

63. RRISK (Reproductive Risk Factors for Incontinence at Kaiser) Random sample of 2100 women aged 40- 69 yo Interview, self report, diaries to determine whether they have the outcome, urinary incontinence. Chart abstraction of L&D/surgical records to establish predictor status Design?

64. RRISK *There is a longitudinal element in this study, and it was funded with an R01 by the NIDDK as a “retrospective cohort study.” Can you identify losses to follow up? (e.g., patients who had surgical procedures in the past but for whom outcome status cannot be determined)

65. HIV Tropism and Rapid Progression* Is HIV CXCR4 (as opposed to CCR5) tropism a predictor of rapid progression in acutely infected HIV patients? Molecular tropism assay is “high end” and labor- intensive. Have funding to perform a total of 80 assays. UCSF OPTIONS cohort follows patients acutely infected with HIV. Has banked serum from near time of acute infection. * Vivek Jain’s Project

66. Identify the 40 patients with the most rapid progression (Group 1) and randomly select 40 others from the UCSF Options cohort (Group 2). Run the tropism assay on banked serum for these 80 patients and compare results between Group 1 and Group 2. HIV Tropism and Rapid Progression (continued) Design?

67. Screening Sigmoidoscopy and Risk of Death from Colon Cancer Compared rates of previous sigmoidoscopy in patients who had died of colon cancer (of the rectum and distal colon) with sigmoidoscopy rate in patients who had not died of colon cancer. Sigmoidoscopy rates were lower in the group of patients who died of cancer. Selby JV, et al. N Engl J Med 1992;326(10):653-7. Design?

68. Screening Sigmoidoscopy and Risk of Death from Colon Cancer Selby JV, et al. N Engl J Med 1992;326(10):653-7. The authors did a second comparison between patients who died of proximal colon cancers and patients who did not die of proximal colon cancers. Sigmoidoscopy rates were the same in both. (See Hulley, Ch 12, page 195.) Confounding?

69. Enhancing Causal Inference Measure Alternative Outcome If worried about confounding, measure another outcome that could be caused by the confounder but NOT by the predictor of interest. If there is NO association between the predictor of interest and the alternative, causally unrelated outcome, then the relationship between the predictor and the primary outcome is more likely to be causal.

70. Sigmoidoscopy and Death from Colon CA Screening sigmoidoscopy Death from Proximal Colon CA Better health care, other confounders Decreases Association Should Not Affect Cause If the association of sigmoidoscopy and decreased colon cancer death were due to confounding, then sigmoidoscopy should also appear to reduce cancers beyond the reach of the sigmoidoscope Appears to Decrease