1. Relative Efficacy and Safety of Aprotinin and Tranexamic Acid in Cardiac Surgery Keyvan Karkouti, MD, FRCPC, MSc Toronto General Hospital University Health Network University of Toronto

2. Disclosure n I have received $3,000 from Bayer Inc. for speaking engagements

3. Overview n Propensity analysis primer n Aprotinin versus Tranexamic Acid - Karkouti et al. u Methodology and Results  Aprotinin not superior to tranexamic acid u Major criticisms and putting it all in context  Not randomized, therefore “results are irredeemably biased”  Results not consistent with findings of placebo-controlled RCTs, therefore must be wrong n My Conclusions  No convincing evidence that aprotinin is better, or worse, than alternatives

4. Propensity Analysis Primer n Multivariable analyses in observational studies n Multiple logistic regression versus propensity analysis  Newgard et al. Acad Emerg Med 2004;11:953-961  Shah et al. J Clin Epid 2005;58:550-559

5. Logistic Regression vs. Propensity Analysis Logistic Regression n Objective: to reduce bias by adjusting for measured confounders n Models for outcome, covariates include exposure and confounders n Get adjusted odds ratio of outcome for exposure Propensity Analysis n Objective: to reduce bias by adjusting for measured confounders n Models for exposure, covariates include confounders n Get a propensity score (PS) for exposure for each patient n Difference between exposed and unexposed with equal PS is an unbiased est. of treatment effect

6. Logistic Regression vs. Propensity Analysis Logistic Regression n Cannot adjust for confounders with large differences in distribution between treatment groups n Assumes linear relationship between confounder and outcome n # of covariates that can be included is limited by # of outcomes n Rare covariates cannot be included Propensity Analysis n Can adjust for confounders with large differences in distribution between treatment groups n No underlying assumptions in modeling n Unlimited # of covariates can be included because it is used to balance treatment groups, not to make inferential statements about treatment groups

7. What Makes for a ‘Good’ Propensity Score Analysis? n Exploit its advantages over logistic regression n Use it appropriately u Use of PS to match exposed/unexposed patients vs. adjusting for PS as part of logistic regression model for outcome n Confirm its validity u Must demonstrate if adequately balanced confounders n Realize its limitations u Does not match for unmeasured confounders

8. Use of PS as part of logistic regression Newgard et al. Acad Emerg Med 2004;11:953-961

9. Aprotinin vs. Tranexamic Acid - Karkouti et al. n Hypothesis: Aprotinin is superior to tranexamic acid; therefore, it is associated with reduced transfusions and improved outcomes n Site: Tertiary care, aprotinin used in high risk cases only n Sample: Consecutive CPB patients from 1999-2004 n Data Source: Prospectively collected clinical database n Analysis: Propensity score matching of aprotinin with tranexamic acid u Propensity score derivation model developed for likelihood of aprotinin use using perioperative clinical variables

17. Study Conclusion n Study Hypothesis: Aprotinin is superior to tranexamic acid; therefore, it is associated with reduced transfusions and improved outcomes n Study Result: Aprotinin was not associated with reduced transfusions or improved outcomes n Study Conclusion: Reject hypothesis

18. Major criticisms and putting it in context n Not randomized, therefore “results are irredeemably biased” because of confounding by indication u Aprotinin group was higher risk, and no statistical analysis can fully adjust for this.  Thus, aprotinin was more effective because transfusion rates were similar to a lower risk group  Thus, aprotinin was not more harmful because a higher renal dysfunction rate is expected in the higher risk group

20. Are observational studies irredeemably biased? n Best evidence on EFFICACY of therapy comes from randomized trials u Caveat: Low quality RCTs may overestimate benefits of therapy n Best evidence on HARM of therapy will often come from large, properly analyzed nonrandomized trials u RCTs are not ideal for identifying adverse events if:  Low frequency owing to restrictive inclusion/exclusion criteria  Short follow-up periods  Small sample sizes Vandenbroucke (Editorial) CMAJ 2006 174(5)

21. Randomized versus Nonrandomized Studies n Data from observational studies on adverse effects may be as valid as those from RCTs u Adverse effects often not linked to treatment indications  confounding by indication not an issue  little need for randomization to quantify them u The same is not true for beneficial effects  Beware of observational studies that claim beneficial effects, even if unanticipated  Patients taking the drug were sufficiently well to continue taking them over a longer period Vandenbroucke (Editorial) CMAJ 2006 174(5)

22. Panagiotis CMAJ 2006;175:635-41 n Comparison of evidence on HARMS of medical interventions in randomized and nonrandomized studies u Targeted medical interventions that had randomized and nonrandomized studies with > 4000 subjects u Compared relative risks u Found 15 harms that could be assessed

23. Papanikolaou, P. N. et al. CMAJ 2006;174:635-641 Comparison of relative risk estimates for specific harms of medical interventions from randomized and nonrandomized studies

24. Randomized versus Nonrandomized Studies n Thus: “It may be unfair to invoke bias and confounding to discredit observational studies as a source of evidence on harms.” n Caveat: Observational study must be of high quality u Large sample size u Proper adjustment for baseline differences to reduce confounding by indication u Transparent n Example: Risk of platelet transfusion in cardiac surgery

25. Spiess et al. Transfusion 2004 n Used data from phase III RCT aprotinin studies n N = 284 platelets; 1436 no platelets n Logistic regression and propensity analysis Adjusted for: Adjusted for: age, sex, race, weight, history of diabetes, history of unstable angina, previous history of coronary artery disease (CAD), history of hypertension, history of chronic obstructive pulmonary disease (COPD), history of congestive heart failure (CHF), New York Heart Association classification (identified as ≥II [NYHA II], ≥III [NYHA III], ≥IV [NYHA IV]), left ventricular ejection fraction less than 30 percent, left ventricular ejection fraction less than 50 percent, return to surgery for reexploration for surgical bleeding, return to surgery for reexploration for diffuse bleeding, return to surgery for any reason, left ventricular assist device, volume of reinfused shed mediastinal blood, RBC transfusion (yes/no), duration of surgery, total heparin dose, total protamine dose, minimum intraoperative Hct, minimum intraoperative Hb.

26. Platelets Associated with Adverse Outcomes

27. Karkouti et al. CJA 2006 n Used data from our large database n N = 2174 platelets, 9285 no platelets n Logistic regression and propensity analysis n When adjusted for same variables as previous study, platelets were associated with LOS, renal failure, and death n Then we adjusted for three additional variables: baseline platelet count, difficult wean from CPB, and massive blood loss

28. Platelets NOT Associated with Adverse Outcomes

29. Major criticisms and putting it in context n Not randomized, therefore “results are irredeemably biased”* because of confounding by indication u Thus, this assertion is irredeemably invalid u However, we agree that neither our study, nor the study by Mangano et al., is conclusive  Not yet reproduced  Cannot rule out the effect of unmeasured confounders The results of our study on 3500 patients who had surgery at seven Canadian hospitals in 2004 highlight the importance of the site-effect in multi-center observational studies The results of our study on 3500 patients who had surgery at seven Canadian hospitals in 2004 highlight the importance of the site-effect in multi-center observational studies *Weiskopf RB. J Thromb Haemost 2066;4:2074-8

30. Major criticisms and putting it in context n Results not consistent with findings of placebo-controlled RCTs, therefore must be wrong

31. Placebo-controlled RCTs Levi et al. Lancet 1999;354:1940-47

32. Placebo-controlled RCTs Brown NEJM 2006

33. What about head-to-head studies? n Meta-analysis: Carless et al. BMC Cardiovascular Disorders 2005,5:19 n 20 head-to-head RCTs identified n Total N = 2430 n Median trial arms N = 25; Range = 14-522 n Methodological quality = Poor n Very little high-risk data (e.g., primary CABG = 12 studies)

34. 24-hour blood loss

35. RBC Transfusion Rate 37.2%36.5%

36. Other Outcomes n Uninformative because the data are sparse

37. Major criticisms and putting it in context n Results not consistent with findings of placebo-controlled RCTs, therefore must be wrong u Our results are consistent with totality of existing evidence

38. Conclusions n No conclusive evidence exists that aprotinin is better, or worse, than tranexamic acid n There is a glaring lack of high-quality data in high-risk patients that the BART trial will only partly resolve