Presentation is loading. Please wait.

Presentation is loading. Please wait.

Statistical Issues in Interpreting Clinical Trials D. L. DeMets Journal of Internal Medicine 255: 529-537. 2004 “Lies, Damn Lies, and Clinical Statistics”

Similar presentations

Presentation on theme: "Statistical Issues in Interpreting Clinical Trials D. L. DeMets Journal of Internal Medicine 255: 529-537. 2004 “Lies, Damn Lies, and Clinical Statistics”"— Presentation transcript:

1 Statistical Issues in Interpreting Clinical Trials D. L. DeMets Journal of Internal Medicine 255: 529-537. 2004 “Lies, Damn Lies, and Clinical Statistics” Justin L. Grobe September 1, 2004

2 Drug Development Paradigm  Medicinal Chemistry Targeted development of new compounds  Animal Testing Test efficacy, potency, safety  Human Clinical Trials Multiple phases to test efficacy, potency, safety, and to compare new intervention to standard

3 Clinical Trials – Design Paradigm  Randomization Assignment to treatment group Order effects  Placebo “Control” (Ethical considerations)

4 Input = Return  “No clever analysis can rescue a flawed design or poorly conducted trial.”  Compliance issues

5 Five Major Statistical Issues 1. Intention-to-treat principle 2. Surrogate outcome measures 3. Subgroup analyses 4. Missing data 5. Noninferiority trials

6 Statistical Issue 1: Intention-to-treat principle  “… all patients are accounted for in the primary analysis, and primary events observed during the follow- up period are to be accounted for as well.”  Results can be biased if either of these aspects are not adhered to

7 Myths and examples  Myth: Large trials are free of these concerns Increased numbers of patients decreases variability of response variable, thereby making detection of differences easier; EXCEPT, this amplifies biases in the outcome measurement WHICH MAY cause detection of differences which do not actually exist

8 To include or not to include…  Two common reasons to drop patient data Post hoc ineligibility assessment Lack of patient compliance

9 TABLE 1: Post-hoc ineligibility assessment Anturane Reinfarction Trial  1629 patients who had survived a heart attack 813 patients received Anturane 816 patients received placebo 71 patients deemed “ineligible” for analysis by protocol Table 1 1980 Anturane mortality results Anturane (%) Placebo (%) P-value Randomized 74/813 (9.1) 89/816 (10.9) 0.20 ‘Eligible’ 64/775 (8.3) 85/783 (10.9) 0.07 ‘Ineligible’ 10/38 (26.3) 4/33 (12.1) 0.12 P-values for eligible 0.0001 0.92 versus ineligible Striking statistical comparisons are made by including/excluding patients in each group: thus the results are biased by post hoc exclusions

10 TABLE 2: Patient compliance Coronary Drug Project  3885 post-heart attack men were given clofibrate or placebo 708 clofibrate and 1813 placebo patients were at least 80% compliant Table 2 Coronary drug project 5-year mortality Clofibrate Placebo n % Deaths n % Deaths Total (as reported) 1103 20.0 2782 20.9 By compliance 1065 18.2 2695 19.4 <80% 357 24.6 882 28.2 >80% 708 15.0 1813 15.1 Compliance itself is considered an outcome: thus to base the interpretation of the ‘drug outcome’ on the ‘compliance outcome’ is confounding

11 Dealing with noncompliance  Larger sample sizes are required to compensate for the dilution effect of noncompliance 10% noncompliance requires 23% increase in sample size 20% noncompliance requires a 56% increase in sample size

12 Statistical Issue 2: Surrogate outcome measures  Outcome measures of primary question must be: Clinically relevant Sensitive to intervention Ascertainable in all patients Resistant to bias  Result: Large, time-consuming, costly studies  Alternative approach: surrogate outcome measures

13 Surrogate outcome measure: Assumption  If the intervention will modify surrogate outcome, it will modify the primary clinical outcome

14 Surrogate outcome measure: Requirements 1. Surrogate outcome must be predictive of clinical outcome 2. Surrogate outcome must fully capture the total effect of the intervention on the clinical outcome “Necessary and sufficient”

15 Surrogate outcome measures: Difficult to obtain and validate  Intervention may modify the surrogate and have no or only partial effect on the clinical outcome  Intervention may modify the clinical outcome without affecting the surrogate (Note: NOT surprisingly, track record for use of surrogate outcome measures is very bad)

16 Surrogate outcome measures: Example: Cardiac Arrhythmia Suppresion Trial (CAST)  Three drugs tested for suppression of cardiac arrhythmias All three drugs had been shown to suppress premature cardiac ventricular contractions (surrogate)  Two drugs terminated early (10-15% into study) because both drugs dramatically increased cause- specific sudden death and total mortality Table 3 Cardiac Arrhythmia Suppression Trial Early termination in two drug arms Drugs Placebo Sudden death339 Total mortality 56 22 Clearly the interventions (drugs) had differential effects on the surrogate measure (premature cardiac ventricular contractions) and the clinical outcome (mortality)

17 Statistical Issue 3: Subgroup analyses  Clinical trials usually try to include as many (diverse) patients as possible for multiple reasons: Large sample size Reasonable recruitment time Assess internal consistency of results  Seemingly logical use of the large data set is to do many post hoc analyses on subgroups

18 Subgroup analysis: Mathematical problems  Introduction of subgroups increases probability of false positives 5 subgroups yields greater than 20% chance of at least one (p=0.05) statistically significant difference BY CHANCE

19 Subgroup analysis: MERIT trial  Beta-blocker (metoprolol) treatment for patients with congestive heart failure Showed a 34% reduction in mortality overall

20 Subgroup analysis: MERIT trial Consistency of mortality results across lots of subgroups found with subgroup analysis:

21 Subgroup analysis: MERIT trial In the USA, total mortality is not reduced, yet total mortality plus any hospitalization is…?

22 Subgroup analysis: MERIT trial  Two other similar heart failure trials evaluating other beta-blockers showed no regional difference;  THUS, it is likely that the MERIT finding is due to chance alone.

23 Subgroup analysis: PRAISE-I and PRAISE-II trials  PRAISE-I performed to evaluate amlodipine for the treatment of congestive heart failure Subgroups:  Ischemia  Nonischemia  Analysis of subgroups separately showed a significant (p<0.001) effect of amlodipine on heart failure in nonischemic patients, but no effect on ischemic patients  Researchers decided to perform PRAISE- II trial on nonischemic patients only

24 Subgroup analysis: PRAISE-I and PRAISE-II trials  PRAISE-II showed remarkably similar mortality results in the drug and placebo groups  PRAISE-II directly opposed the exciting results of PRAISE-I’s subgroup analysis

25 Statistical Issue 4: Missing data  Missing data is often simply “dropped”  This violates two rules: 1. Intention-to-treat rule  all patients must be accounted for in primary outcome analysis 2. Common sense rule  if patient is too sick to complete trial, this may be informative!

26 Missing data  In “time to event” trials (like mortality), data can be missing because the study ends before the event happens Patients are then “censored” (dropped)  This can introduce serious mathematical bias  (Mortality studies in USA have no excuse  death indices allow follow-up without help from patient)

27 Statistical Issue 5: Noninferiority trials  “New intervention is not worse than the standard” New intervention may be:  Easier to administer  Better tolerated  Less toxic  Less expensive  Any given study may be a superiority and/or noninferiority trial, depending on results

28 Noninferiority trials

29  Three challenges must be met: 1. Noninferiority trial must be of highest quality to detect clinically meaningful differences 2. Noninferiority trial must have a strong, effective control intervention (state-of-the-art care) 3. Margin of indifference is arbitrary, depending on medical importance of treatment and risk-to-benefit tradeoffs

30 Noninferiority trials: OPTIMAAL Trial  Losartan (angiotensin II receptor blocker) vs captopril (ACE inhibitor) in heart failure patient population Losartan has fewer (and less severe) side effects than captopril  OPTIMAAL Designed to detect 20% reduction in relative risk, with 95% power Margin of indifference set at 1.1  Thus 95% confidence interval needed to exclude risk of 1.1 to declare losartan “noninferior” to captopril

31 Noninferiority trials: OPTIMAAL Trial  Mortality results for OPTIMAAL Relative risk of 1.126 with 95% confidence interval of 1.28  NEITHER superiority nor noninferiority were achieved Researchers computed that captopril had (historical data) a relative risk of 0.806 vs. placebo, and thus calculated that losartan must therefore have a relative risk of 0.906 vs. placebo…  The statistically appropriate conclusion at this point is: NO ACCEPTABLE CONCLUSIONS POSSIBLE FROM THIS DATA

32 CONCLUSIONS  Statistics can not make up for bad design  Statistics can not make up for poor execution of design  Statistics is very limited in being able to compensate for Ineligible patients being enrolled Noncompliance Unreliable outcome measures Missing data Underpowered trials

Download ppt "Statistical Issues in Interpreting Clinical Trials D. L. DeMets Journal of Internal Medicine 255: 529-537. 2004 “Lies, Damn Lies, and Clinical Statistics”"

Similar presentations

Ads by Google