v.: 9/7/2007 AC Submit1 Statistical Review of the Observational Studies of Aprotinin Safety Part I: Methods, Mangano and Karkouti Studies CRDAC and DSaRM Meeting September 12, 2007 Mark Levenson, Ph.D. Quantitative Safety & Pharmacoepidemiology Group Office of Biostatistics

2 Acknowledgements to Dr. Mangano and Dr. Karkouti

3 Outline 1.Review Objectives 2.Statistical Methods 3.Mangano Review 4.Karkouti Review 5.Summary

4 1. Review Objectives 1.To confirm the reported findings based on investigators’ methods 2.To evaluate the statistical robustness of the findings –FDA analyzed the 3 studies Same methods applied to all three studies Robust Diagnostics

5 Outline 1.Review Objectives 2.Statistical Methods 3.Mangano Review 4.Karkouti Review 5.Summary

6 Propensity Scores (PS) Adjust for differences in baseline risk factors between two treatment groups (Treatment A and Treatment B) Definition: Probability of assignment for a patient to Treatment A versus Treatment B based on measured risk factors Intuition: –Suppose Treatment A patient and Treatment B patient have the same PS. –Comparisons between these two patients are fair

7 Propensity Scores Practice Balance: similarity of distributions of a risk factor between treatment groups PS methods cannot account for unmeasured confounders The PS are estimated based on statistical modeling –Diagnostics important

8 Propensity Scores Practice (Cont.) Estimating treatment effects using PS –Matching (Karkouti) –Stratification (FDA) –Multivariate regression (Mangano, i3)

9 FDA Analysis Methods Pre-specified Propensity scores with stratification was used to adjust for baseline risk factors Medical, epidemiological, and statistical expertise was used to choose risk factors Diagnostics (analytical and graphical) were used to evaluate balance and explore findings

10 Outline 1.Review Objectives 2.Statistical Methods 3.Mangano Review 4.Karkouti Review 5.Summary

11 Mangano Study: Key Points Prospectively specified –Inclusion criteria –Outcome definitions –Subgroups –Analysis methods In-hospital outcomes (NEJM) Long-term mortality follow-up outcomes (JAMA) 7 of 69 centers did not participate in the long- term follow-up

12 Mangano Study: Key Points Analysis Methods Multivariate regression with and without propensity score as a covariate –Logistic for in-hospital outcomes –Cox PH for long-term mortality Propensity score for any active agent versus no agent for full analysis group No adjustment for geographical differences

13 Mangano Study: Patients by Geographical Region and Treatment Group Region No Agent N=1374 % Aprotinin N=1295 % Amino- caproic N=883 % Tran- examic N=822 % Europe North America Other191422

14 Mangano Study: Long-Term Follow-up No Agent N=1374 % Aprotinin N=1295 % Completed 5-year follow-up or died 7383 No post-hospital follow-up 101 Lost to follow-up in the post- hospital period 1716

15 Mangano Study: Demographic Factors Characteristic No Agent N=1374 Aprotinin N=1295P-Value Age (mean ± sd)63 ± 1065 ± 9<.001 Male (%) African American or Hispanic (%)

16 Mangano Study: Selected Baseline Risk Factors Characteristic No Agent N=1374 (%) Aprotinin N=1295 (%)P-Value Surgical: CABG + Other1119<.001 Surgical: Non-Elective2115<.001 History of liver disease812<.001 History of renal disease1319<.001 Previous sternotomy313<.001 Preop: Creatinine >1.3 mg/dL Preop: Ejection fraction ≤ 44% Preop: MI

17 Mangano Study: Study Reported Findings and Methods Primary findings of NEJM and JAMA based on investigators’ methods reproduced Imbalances in baseline risk factors and geographical regions between aprotinin and no-agent groups after PS adjustment Lack of overlap in propensity score distributions between aprotinin and no- agent groups

18 FDA Analysis Results

19 Mangano Study: FDA Analysis Baseline Risk Factors Before and After PS Adjustment Before PS AdjustmentAfter PS Adjustment No Agent % Aprotinin %P-Value No Agent % Aprotinin %P-Value Surgical: CABG + Other1119< Surgical: Non-Elective2115< History of liver disease812< History of renal disease1319< Previous sternotomy313< Preop: Elev. Creatinine Preop: Eject. Fr. ≤ 44% Preop: MI

20 Mangano Study: FDA Analysis In-Hospital Outcome Adjusted Estimates Aprotinin vs. No Agent* Outcome No Agent (%) Aprotinin (%) Risk Ratio Aprotinin/No Agent (95% CI) Renal Composite (1.03, 2.60) Renal Failure (1.05, 3.99) Renal Dysfunction (0.76, 2.11) Cardiovascular Composite (0.94, 1.38) Myocardial Infarction (0.88, 1.39) Congestive Heart Failure (0.75, 1.47) Stroke (0.70, 2.64) Death (in-hospital) (0.54, 1.53) *Analysis based on 1307 no agent patients and 1222 aprotinin patients

21 Mangano Study: FDA Analysis Renal Composite

22 Mangano Study: FDA Analysis Long-Term Mortality Adjusted Estimates Aprotinin vs. No Agent* Outcome No Agent (%) Aprotinin (%) Risk Ratio Aprotinin/No Agent (95% CI) 6 Weeks (0.57, 1.51) 6 Months (0.73, 1.68) 1 Year (0.79, 1.71) 2 Years (0.90, 1.79) 3 Years (0.98, 1.83) 4 Years (1.05, 1.84) 5 Years (0.98, 1.62) *Analysis based on 1307 no agent patients and 1222 aprotinin patients

23 Mangano Study: FDA Analysis Long-Term Mortality Adjusted Estimates

24 North American Subgroup

25 Mangano Study: FDA Analysis by Region and Treatment Group

26 Mangano Study: FDA Analysis North America In-Hospital Outcome Adjusted Estimates Aprotinin vs. Aminocaproic* Outcome Amino. (%) Aprotinin (%) Risk Ratio Aprotinin/Amino. (95% CI) Renal Composite (1.98, 7.70) Renal Failure (3.10, 27.15) Renal Dysfunction (1.54, 7.44) Cardiovascular Composite (0.94, 1.74) Myocardial Infarction (0.83, 1.81) Congestive Heart Failure (0.84, 2.35) Stroke (0.76, 5.81) Death (in-hospital) (0.88, 5.52) *Analysis based on 789 aminocaproic patients and 342 aprotinin patients

27 Mangano Study: FDA Analysis North America: Long-Term Mortality Adjusted Estimates Aprotinin vs. Aminocaproic* Outcome Amino. (%) Aprotinin (%) Risk Ratio Aprotinin/Amino. (95% CI) 6 Weeks (0.82, 4.47) 6 Months (1.04, 4.28) 1 Year (1.10, 3.47) 2 Years (1.19, 3.00) 3 Years (1.23, 2.73) 4 Years (1.03, 2.04) 5 Years (0.92, 1.67) *Analysis based on 789 aminocaproic patients and 342 aprotinin patients

28 Mangano Study: Review Summary Renal outcomes (particularly renal failure) effect in a range of patients and in North American region subgroup Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated Long-term mortality effects in a range of patients and in North American region subgroup

29 Outline 1.Statistical Review Objectives 2.Statistical Methods 3.Mangano Review 4.Karkouti Review 5.Summary

30 Karkouti Study: Key Points Retrospective study of 5 years of patient data from a single center Patient population: Cardiac surgery (CABG and non-CABG) with cardio- pulmonary bypass Aprotinin used for high risk patients, tranexamic acid used for other patients Used propensity scores with 1-1 matching –449 of 586 aprotinin patients matched

31 Karkouti Study: Demographic Factors Characteristic Tranexamic Acid N=10251 Aprotinin N=586P-Value Age (mean ± sd)63 ± 1255 ± 17<.001 Male (%)7565<.001

32 Karkouti Study: Selected Baseline Risk Factors Characteristic Tranexamic Acid N=10251 (%) Aprotinin N=586 (%)P-Value Surgical: Not CABG only3389<.001 Surgical: Non-Elective819<.001 Previous sternotomy561<.001 Preop: Creatinine abnormal1926<.001 Preop: Ejection fraction <40% Preop: MI177<.001

33 Karkouti Study: Study Reported Findings and Methods Primary findings using investigators’ methods reproduced Observed risk factors balanced with propensity score matching approach

34 Karkouti Study: FDA Analysis A subgroup of patients with overlap in propensity scores was defined to enable treatment comparison Subgroup contained –553/586 (94%) of the aprotinin patients –3759/10251 (37%) of tranexamic patients Baseline risk factors more similar between treatment groups in subgroup than full group

35 Karkouti: FDA Analysis Analysis Subgroup, Baseline Risk Factors Before and After PS Adjustment Before PS AdjustmentAfter PS Adjustment Tran. % Aprotinin %P-Value Tran. % Aprotinin %P-Value Surgical: Not CABG only8392< Surgical: Non-Elective Previous sternotomy1464< Preop: Elev. Creatinine Preop: Eject. Fr. <40% Preop: MI

36 Karkouti Study: FDA Analysis Treatment Effect Estimates Aprotinin vs. Tranexamic Acid* Outcome FDA Analysis Risk Ratio Aprotinin/Tran. Matched-Pair Odds Ratio Aprotinin/Tran. Renal dysfunction1.53 (1.11, 2.12) Renal failure1.38 (0.86, 2.23)1.85 (0.94, 3.63) Myocardial Infarction1.42 (0.71, 2.83)1.22 (0.51, 2.95) Stroke1.72 (0.93, 3.19)1.15 (0.55, 2.43) Death (in-hospital)1.18 (0.79, 1.76)0.90 (0.54, 1.51) *Analysis based on 3759 tran. patients and 553 aprotinin patients

37 Karkouti Study: FDA Analysis Renal Failure

38 Karkouti Study: Review Summary Renal dysfunction effect statistically significant Some evidence for renal failure effect Effects for myocardial infarction, stroke, and in-hospital death outcomes not statistically demonstrated

39 Outline 1.Review Objectives 2.Statistical Methods 3.Mangano Review 4.Karkouti Review 5.Summary

40 Summary Evidence for renal effect, including renal failure consistent Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated Evidence for long-term mortality effect Potential for unadjusted confounders between the treatment groups which may bias the treatment effect estimates