1. 1 EPI235: Epi Methods in HSR April 12, 2007 L4 Evaluating Health Services using administrative data 3: Advanced Topics on Risk Adjustment and Sensitivity Analysis (Dr. Schneeweiss) Risk adjustment in studies using administrative databases is limited to observed confounders. Dr. Schneeweiss will illustrate theory and practice of assessing the sensitivity of epidemiologic risk estimates towards unobserved confounding. An interactive Excel program will be used for illustration. Background reading: Walker AM: Observation and inference, Chapter 9. Epidemiology Resources, Newton Lower Falls, 1991. Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitiors and myocardial infarction. Epidemiology 2005;16:17-24.

2. 2 Unmeasured (residual) Confounding  Confounding factors that are not measured are hard to adjust for in observational analyses  If unadjusted they lead to residual confounding

3. 3 Unmeasured (residual) Confounding: [smoking, healthy lifestyle, etc.] Drug exposure Outcome RR EO OR EC RR CO CUCU CMCM

4. 4 Unmeasured Confounding in Claims Data  Database studies are criticized for their inability to measure clinical and life-style parameters that are potential confounders in many pharmacoepi studies  OTC drug use  BMI  Clinical parameters: Lab values, blood pressure, X-ray  Physical functioning, ADL (activities of daily living)  Cognitive status

5. 5 Strategies to Discuss Residual Confounding  Qualitative discussions of potential biases  Sensitivity analysis  SA is often seen as the ‘last line of defense’  A) SA to explore the strength of an association as a function of the strength of the unmeasured confounder  B) Answers the question “How strong must a confounder be to fully explain the observed association”  Several examples in Occupational Epi but also for claims data Greenland S et al: Int Arch Occup Env Health 1994 Wang PS et al: J Am Geriatr Soc 2001

6. 6 Foot-in-Mouth Award (Economist ‘04): “… there are known knowns; there are things we know we know. We also know that there are known unknowns; that is to say we know that there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know. …, it is the latter category that tend to be the difficult ones.” (Wisely unknowing) Donald Rumsfeld

7. 7 Notation

8. 8 A simple sensitivity analysis  The apparent RR is a function of the adjusted RR times ‘the imbalance of the unobserved confounder’  After solving for RR we can plug in values ofr the prevalence and strength of the confounder:

9. 9 A made-up example  Association between TNF-a blocking agents and NH lymphoma in RA patients  Let’s assume and observed RR of 2.0  Let’s assume 50% of RA patients have a more progressive immunologic disease  … and that more progressive disease is more likely to lead to NH lymphoma  Let’s now vary the imbalance of the hypothetical unobserved confounder

10. 10 Bias by residual confounding

11. 11

12. 12 Pros and cons of “Array approach”  Very easy to perform using Excel  Very informative to explore confounding with little prior knowledge Problems:  It usually does not really provide an answer to a specific research question  4 parameters can vary -> in a 3-D plot 2 parameter have to be kept constant  The optical impression can be manipulated by choosing different ranges for the axes

13. 13 Same example, different parameter ranges

14. 14 Conclusion of “Array Approach”  Great tool but you need to be honest to yourself  For all but one tool that I present today:  Assuming conditional independence of C U and C M given the exposure status  If violated than residual bias may be overestimated Drug exposure Outcome RR EO OR EC RR CO CUCU CMCM Hernan, Robins: Biometrics 1999 ?

15. 15 More advanced techniques  Wouldn’t it be more interesting to know  How strong and imbalanced does a confounder have to be in order to fully explain the observed findings? RR CO OR EC

16. 16 Example: Wang et al: JAGS 2001;49:1685 Zolpidem use and hip fractures in older people. The issue: Are there any unmeasured factors that may lead to a preferred prescribing of zolpidem to people at higher risk for falling and fracturing? > Frailty is a hard to measure concept in claims data RR CO OR EC ARR PCPC

17. 17 How do we do that?  We want to express as a function of, ARR, P C, P E OR EC RR CO Walker AM: Observation and Inference. Epidemiology Resources Inc., Newton, 1991

18. 18

19. 19

20. 20

21. 21 Example: Psaty et al: JAGS 1999;47:749 CCB use and acute MI. The issue: Are there any unmeasured factors that may lead to a preferred prescribing of CCB to people at higher risk for AMI? OR EC RR CO ARR = 1.57 ARR = 1.30

22. 22

23. 23 Caution!  Psaty et al. concluded that it is unlikely that an unmeasured confounder of that magnitude exists  However, the randomized trial ALLHAT showed no association between CCB use and AMI  Alternative explanations:  Joint residual confounding may be larger than anticipated from individual unmeasured confounders  Not an issue of residual confounding but other biases, e.g. control selection?

24. 24 Pros and cons of “Rule Out Approach”  Very easy to perform using Excel  Meaningful and easy to communicate interpretation  Study-specific interpretation Problems:  Still assuming conditional independence of C U and C M  “Rule Out” lacks any quantitative assessment of potential confounders that are unmeasured

25. 25 External Adjustment  One step beyond sensitivity analyses  Using additional information not available in the main study  Often survey information

26. 26 Strategies to Adjust residual con- founding using external information  Survey information in a representative sample can be used to quantify the imbalance of risk factors that are not measured in claims among exposure groups  The association of such risk factors with the outcome can be assess from the medical literature (RCTs, observational studies) Velentgas et al: PDS, under review Schneeweiss et al: Epidemiology, in press 2004

27. 27 How do we do that?  We want to express ARR as a function of,, ARR, P C, P E OR EC RR CO Walker AM: Observation an Inference. Epidemiology Resources Inc., Newton, 1991

28. 28

29. 29 Example: COX-2 inhibitors use and MI  Ray et al., Lancet 2002:  >25mg roficoxib vs. non NSAID users, RR=1.9 (1.1-3.4)  Medicaid patients, new users  Solomon et al., Circulation in press:  >25mg roficoxib vs. non NSAID users, RR=1.6 (1.04-2.4)  Medicare patients with drug coverage through PACE  Can these associations be due to confounding by factors not measured in claims data?  e.g. BMI, OTC aspirin use, smoking, education etc.

30. 30 In our example: RofecoxibAcute MI RR EO From Survey data in a subsample From medical literature OR EC RR CO [smoking, aspirin, BMI, etc.] CUCU CMCM

31. 31 Where can we get detailed information on unmeasured confounders?  MCBS: Medicare Current Beneficiary Survey  Representative Sample  12,000 Medicare beneficiaries each year (majority > 65y)  Face-to-face interview in beneficiary’s home  ‘Cost and Use’ file include drug utilization  98% response rate  >95% data completeness  Low cost ($900 / year)  Readily available, but 2-year lag time)

32. 32 Unobserved confounders in our example  Independent predictors of MI:  Aspirin use  Smoking  BMI  Educational attainment  Income status  Expl. 2: Independent predictors of hip fracturs:  Cognitive impairment  Physical impairment  Restrictions in ADL (Rubinstein L)

33. 33 Our survey population  1999 MCBS  Restricted to >64 years  Restricted to community sample (no proxi interviews)  N = 8,785

34. 34 Distribution of unmeasured confounders among drug users

35. 35 Celecoxib vs. Rofecoxib users …

36. 36 Literature estimates of RR CO

37. 37 Calculating bias

38. 38 More contrasts

39. 39 What does it mean?  Ray et al.: RR of 1.9 is an underestimation of the unconfounded RR by 5% (max)  So the effect estimate corrected for 5 unobserved confounders would be about 2.0  Solomon et al.: RR of 1.6 would move to 1.7

40. 40 Sensitivity of Bias as a Function of a Misspecified RR CD : Obesity (BMI >=30 vs. BMI<30)

41. 41 Sensitivity towards a misspecified RR CO from the literature: OTC aspirin use (y/n)

42. 42

43. 43 Variance estimate of externally adjusted parameters

44. 44 Summary External Adjustment  This method provides a quantitative assessment of the effect of selected unobserved confounders  Easy to use (Excel program available from author)  MCBS is available from CMS for $900 per annual survey  Should be more frequently used in Pharmacoepi studies using claims data

45. 45 Limitations (1)  Example is limited to 5 potential confounders  No lab values, physical activity, blood pressure etc.  What about the ‘unknow unknowns’?  We currently explore NHANES ’99/’00  Lab values, dietary suppl. (Ca 2+ ),  Drug data quality?  To assess the bias we assume an exposure–disease association of 1 (null hypothesis)  The more the truth is away from the null the more bias in our bias estimate  However the less relevant unmeasured confounders become

46. 46 Limitations (2)  Validity depends on representativenes of sampling with regard to the unmeasured confounders  We could not consider the joint distribution of confounders  Limited to a binary world

47. 47 Solving the Main Limitations  Need a method  That addresses the joint distribution of several unmeasured confounders  That can handle binary, ordinal or normally distributed unmeasured confounders  Lin et al. (Biometrics 1998) :  Can handle a single unmeasured covariate of any distribution  But can handle only 1 covariate  Sturmer, Schneeweiss et al. (AJE 2005 in press) :  Propensity Score Calibration can handle multiple unmeasured covariates of any distribution