Interpretation of Medical Literature Statistics Karen Pieper, MS Duke Clinical Research Institute

Website for Power and Sample Size program used in second class

To be an informed consumer of published statistical analyses, one should know: ■What needs to appear in a manuscript ■What to keep in mind when looking at data ■The dangers of subgroup analyses

0.89 (0.79, 0.99) 0.79 (0.69, 0.91) 1.10 (0.91, 1.34) 0.78 (0.66, 0.93) 0.98 (0.84, 1.13) 0.87 (0.76, 1.00) 0.96 (0.77, 1.19) 0.93 (0.80, 1.08) 0.85 (0.72, 1.00) 0.89 (0.79, 1.00) 0.90 (0.66, 1.24) Death or MI at 30 Days

PRISM PRISM+ PARAGON-A PURSUIT PARAGON-B GP IIb/IIIa Better Placebo/ Control Better GUSTO-IV All GP IIb/IIIa Better Placebo/ Control Better FemaleMale OR and 95% CI

Sex Issue OR (95% CI)P-value for MalesFemales Heterogeneity All patients Death0.83 ( )1.08 ( )0.030 Death or MI0.81 ( )1.15 ( )< Patients with missing data on baseline cardiac troponin Death0.81 ( )1.20 ( )0.011 Death or MI0.78 ( )1.18 ( )< Patients with data on baseline cardiac troponin Death0.85 ( )0.91 ( )0.83 Death or MI0.93 ( )1.07 ( )0.38 Patients with baseline cardiac troponin T or I < 0.1  g/L Death1.07 ( )1.20 ( )0.84 Death or MI1.10 ( )1.29 ( )0.65 Patients with baseline cardiac troponin T or I  0.1  g/L Death0.75 ( )0.80 ( )0.88 Death or MI0.82 ( )0.93 ( )0.48

N Engl J Med 2001;344: Do Tirofiban And ReoPro Give Similar Efficacy Outcomes Trial Primary Hypothesis Tirofiban will have comparable efficacy to abciximab in reducing the incidence of adverse cardiac ischemic events during the first 30 days after intracoronary stent placement.

Statistical Considerations Sample size provides 88% power to declare tirofiban noninferior to abciximab, based on the relative efficacy of abciximab to placebo in EPISTENT.* * The upper bound of the one-sided 95% C.I. for the odds ratio (tirofiban relative to abciximab) must be below N Engl J Med 2001;344:

Primary Endpoint 30-day composite of: ■Death ■Myocardial infarction ■CK-MB > 3x ULN in two samples ■New Q waves ■Urgent TVR ■PCI or CABG N Engl J Med 2001;344:

Primary Endpoint N Engl J Med 2001;344: day Death, MI, Urgent TVR Upper bound of 95% C.I. = 1.51 Noninferiority boundary R.R. = 1.26 AbciximabBetter TirofibanBetter p = % 6.0% 0% 2% 4% 6% 8% 10% R.R. = 1.26 TirofibanAbciximab 30-day Death, MI, Urgent TVR (%)

Primary Endpoint Analysis N Engl J Med 2001;344: p = 0.038p = 0.04 p = 0.66p = 0.49

Subgroup Analysis Diabetes No Diabetes Age < 65 Age  65 MaleFemale 1 Tirofiban Better TirofibanAbciximab %RRCI , , , , , , 2.89 N Engl J Med 2001;344: Abciximab Better

Subgroup Analysis Pre-procedure Clopidogrel Yes Yes No NoACSNon-ACSU.S.Ex-U.S. TirofibanAbciximab %RRCI , , , , , , 4.64 N Engl J Med 2001;344: Tirofiban Better Abciximab Better

30-Day Conclusions ■Abciximab was superior to tirofiban in reducing the incidence of the composite endpoint of death/MI/urgent target vessel revascularization at 30 days after intracoronary stent placement. ■There were no differences in rates of TIMI major bleeding, but significant differences in minor bleeding and thrombocytopenia were observed and favored tirofiban. N Engl J Med 2001;344:

95% CI: 0.90, % CI: 1.05,1.52 Composite Endpoint (Death/MI/TVR) p = 0.038p = 0.509

95% CI: 0.96, % CI: 1.01,1.58 Death/MI p = 0.04p = NS

95% CI: 0.76, % CI: 0.65,2.44 Target Vessel Revascularization p = NS Urgent TVR

95% CI: 0.67, % CI: 0.72,1.90 Diabetics: Composite Endpoint (Death/MI/TVR) p = NS

Subgroups: EPISTENT Diabetes Paper 6-month Event Rates for Study Group Stent/PlaceboStent/Abciximabp Death, MI, TVR Diabetics43 (25.2)21 (13.0)0.005 Nondiabetics104 (16.5)81 (13.0)0.062 Death or MI Diabetics22 (12.7)10 (6.2)0.041 Nondiabetics70 (11.0)34 (5.4)< MI Diabetics19 (11.0)10 (6.2)0.11 Nondiabetics64 (10.1)31 (4.9)< Death Diabetics3 (1.7)1 (0.6)0.35 Nondiabetics7 (1.1)3 (0.5)0.21 Diabetics (1 yr)8 (4.8)2 (0.9)0.08

Post-randomization Subgroups

Issues Specific to the “Post-randomization” Component Example: ■A clinical trial evaluated the treatment effect of a new drug (A) versus placebo (P) in ACS patients. The primary endpoint of the trial was 30-day death or MI. Of special interest was the effectiveness of the new drug in patients who had received a PCI versus those who had not.

Sample Patient 1 Randomization PCI Death or MI 30-day Assessment

Sample Patient 2 Randomization Death or MI 30-day Assessment

Incidence of 1  EndpointOdds Ratio EptifibatidePlaceboP(95% CI) Incidence of 1  EndpointOdds Ratio EptifibatidePlaceboP(95% CI) PCI < 72 hours(N = 606)(N = 622) 96 hours57 (9.4)95 (15.3) (0.406, 0.817) 7 days62 (10.2)100 (16.1) (0.424, 0.835) 30 days70 (11.6)104 (16.7) (0.469, 0.901) No PCI < 72 hrs(N = 4116)(N = 4117) 96 hours302 (7.3)334 (8.1) (0.763, 1.055) 7 days415 (10.1)452 (11.0) (0.790, 1.047) 30 days602 (14.6)641 (15.6) (0.823, 1.048) PCI = percutaneous coronary intervention; CI = confidence interval Improper Subgroups

Sample Patient 3 Randomization MI PCI 30-day Assessment

EptifibatidePlaceboAbsolute Time Interval(N = 606)(N = 622)ReductionP-value Before PTCA Death/MI1.7%5.5%3.8%< hours Death/MI*8.1%10.9%2.9% days Death/MI*8.9%11.7%2.8% days Death/MI*10.2%12.4%2.2%0.235 *Composite only includes myocardial infarctions (MI) occurring after the percutaneous

SUPPORT JAMA 1996; 276: Study of effectiveness of Right Heart Catheterization (RHC) in the initial care of critically ill patients. Issues: Timing of RHC relative to hospital arrival Timing of RHC relative to death Need to survive long enough to get a RHC – patients who die early will be in the no RHC group. Bias in who is chosen for a RHC Study of effectiveness of Right Heart Catheterization (RHC) in the initial care of critically ill patients. Issues: Timing of RHC relative to hospital arrival Timing of RHC relative to death Need to survive long enough to get a RHC – patients who die early will be in the no RHC group. Bias in who is chosen for a RHC

Unadjusted Survival Rates from SUPPORT study p < JAMA 1996; 276:

SUPPORT study – Adjustments JAMA 1996; 276: Only included RHC that occurred within the first 24 hrs. This cut down on the problem of survivor bias. Calculated the “propensity” to receive a RHC and adjusted for both baseline differences and this propensity score. Only included RHC that occurred within the first 24 hrs. This cut down on the problem of survivor bias. Calculated the “propensity” to receive a RHC and adjusted for both baseline differences and this propensity score.

SUPPORT study – Adjusted results JAMA 1996; 276: Hazard Ratio: 1.21 (1.09, 1.25), (p<0.001)

EMBARGOED FOR RELEASE Tuesday, November 9, 2004 CONTACT: NHLBI Communications Office (301) AHA Scientific Sessions News Media Center: (504) (8 a.m. to 8 p.m. EST, Sun., Nov 7, through Tues., Nov 9, and 8 a.m. to 6 p.m. EST, Weds. Nov 10.) No Increase in Deaths or Hospitalizations for Heart Failure Patients Who Have a Pulmonary Artery Catheter EMBARGOED FOR RELEASE Tuesday, November 9, 2004 CONTACT: NHLBI Communications Office (301) AHA Scientific Sessions News Media Center: (504) (8 a.m. to 8 p.m. EST, Sun., Nov 7, through Tues., Nov 9, and 8 a.m. to 6 p.m. EST, Weds. Nov 10.) No Increase in Deaths or Hospitalizations for Heart Failure Patients Who Have a Pulmonary Artery Catheter

Safeguards ■Remember that the primary hypothesis is the only one the study was designed to answer ■Nonsignificant results may indicate that there were too few patients studied to detect a small meaningful difference. ■Subgroup results should be confirmed in subsequent studies before acceptance.

Baseline Table ■The baseline table is needed to show where the differences in baseline characteristics exist, especially when the sample size is small or the groups being compared are not randomized. ■ If baseline characteristics are not equally distributed, be sure that there is at least one analysis of the endpoint(s) adjusted for the other factors associated with the endpoint(s).

Example—ESPRIT Diabetes Baseline Characteristics for Diabetics and Nondiabetics Diabetics (%)Nondiabetics (%) VariableN = 466N = 1595P-value Age, yrs62.0 (55.0, 70.0)62.0 (54.0, 71.0)0.668 Weight, kg89.0 (77.0, 102.0)83.5 (73.2, 94.5)< Women 175 (37.6)386 (24.2)0.001 Previous MI142 (30.5)509 (31.9)0.556 Previous PCI125 (26.8)357 (22.4)0.046 Previous CABG55 (11.8)156 (9.8)0.205 Hypertension334 (71.7)877 (55.0)0.001 Hypercholesterolemia294 (63.1)905 (56.8)0.015 PVD51 (10.9)86 (5.4)0.001 Previous stroke29 (6.2)60 (3.8)0.021 Current smoker89 (19.3)389 (24.6)0.012

Be Careful of Axes Sizes! Compare this…

To this… (It’s the same data!)

Other Precautions ■Check the Ns—do things add up? You’d be surprised how many times they don’t! ■Watch for missing data. Does it appear that the sample size has dropped for some variables? What has been done about missing data? How will that influence the results?

Other Precautions ■Can you account for every patient in how the sample for this study was drawn from the original study population? ■Are confidence intervals or error bars included for estimates?

References: ■Bailar JC III, Mosteller F. Guidelines for statistical reporting in articles for medical journals, Ann Intern Med, 108: , ■DerSimonian R, Charette LJ, McPeek B, Mosteller F. Reporting on methods in clinical trials. In Medical Uses of Statistics, 2nd ed, Bailar JC III et al (ed), Boston, NEJM, 1992, pp ■Gardner MJ, Machin D, Campbell MJ. Use of check lists in assessing the statistical content of medical studies. In Statistics with Confidence, Gardner MJ, et al (eds), London, British Medical Journal, 1989, pp