1. ISSUES THAT PLAGUE NON- INFERIORITY TRIALS PAST AND FUTURE RALPH B. DAGOSTINO, SR. BOSTON UNIVERSITY HARVARD CLINICAL RESEARCH INSTITUTE

2. OBJECTIVES 1.REVIEW ISSUES: PAST, PRESENT AND FUTURE IN NON-INFERIORITY (NI) STUDIES 2.PRESENT/ DISCUSS EXAMPLES 3.MAKE SOME COMMENTS FOR IMPROVEMENTS 4.PRESENT A PERSONAL VIEW

3. OUTLINE 1.Early Objectives and Issues 2.Approaches to Non-inferiority Trials 3.Examples (Here are some Problems) 4.Non-Inferiority AND/OR Superiority 5.All is Non-Inferiority 6.Intent-to-Treat vs. Per Protocol 7.New Major Issues

4. EARLY OBJECTIVES AND ISSUES: EQUIVALENCY American Dental Association (ADA 1980s) CREST equivalent to COLGATE? Ho: A-B>= 10% or A-B<= 10% What does the 10% mean? –DFMS or DFMT for 2 years, 3 years? Study done on differences and ratio used as descriptive measure of effect –5.0 vs 5.4 becomes (5.4-5.0)/5.0 =.4/5.= 8%

5. EARLY OBJECTIVES M = 10% CAME FROM NOWHERE, BUT WE KNEW WHAT IT WAS, That is, 10% TREATMENT DIFFERENCES CONCERNED DIFFERENCES (RATIOS) BETWEEN ACTIVE TREATMENTS WE WERE LOST BUT WE BELIEVED WE HAD A SENSE ABOUT IT

6. APPROACHES TO NI TESTS MUST DO BETTER THAN PLACEBO But you cannot use a Placebo (P) 1.Putative Placebo Approach 2.Test Treatment (T) vs Positive Control (C) directly with given Margin M (Assay Sensitivity approach)

7. APPROACH 1 (Putative Placebo) Stellar Example from the Past CAPRIE Study. Hasselblad and Kong (2001) present this as their major example for using meta-analyses for dealing with estimating assay sensitivity (T vs. P) Want T vs. C, C vs. P, T vs. P

8. 21

9. CAPRIE STUDY (cont) Can we obtain effect of Clopidogrel vs. Aspirin Yes, if we can locate Asprin vs. Placebo Do we believe what we get?

10. For Aspirin vs. Placebo Antiplatelet Trialists Collaboration Meta-Analysis Meta-analysis of all published and unpublished unconfounded randomized trials available March 1990 Trials identified by literature search, trial registry and inquiry of investigators and pharmaceutical manufacturers Clear definitions of endpoints Well defined statistical methodology

11. APPROACH T vs. C(from Caprie trial) C vs. P(from Meta-analysis) Obtain T vs. P (from multiplication) (T/C) (C/P) = (T/P)

12. Clopidogrel Vs. Synthetic Placebo Control Odds Ratios and 95% Confidence Intervals Overall Patient Population Endpoint All Strokes, MIs, or Vascular Deathsp < 0.000001 All Strokes, MIs or Death from p < 0.000001 Any Cause Vascular p < 0.0016 Deaths All Cause p < 0.0045 Deaths 0.4 0.6 0.8 1.0 1.2 1.4 1.6 First Drug BetterSecond Drug Better CAPRIE: Clopidogrel Vs. Aspirin Meta-Analysis: Aspirin Vs. Placebo Estimated: Clopidogrel Vs. Placebo

13. Meta-analysis studies contain very old studies (only up to 1990), many prior to all of the elaborate medical interventions (procedures) now routinely provided Many of the studies did not have MI, IS or vascular death as their outcomes (the meta-analysis went back to original investigators who in turn, had to generate data). Ever try to get data on something you did not collect? None of the studies used for Clopidogrel with aspirin comparison had PAD as an entry criteria (PAD represented 1/3 of Clopidogrel Study)

14. EFFECT SIZE: Relative Risk Reduction by Qualifying Condition (ASA vs Clopidogrel) IS n = 6431 MI n = 6302 PAD n = 6452 Total n =19185 30 20 10 0 10 20 Clopidogrel BetterAspirin Better

15. Problems With Historical Controls Biases –Time Biases Change in recognition or diagnosis of disease Changing disease process Change in usual therapy (Myocardial Infarctions MI, Dx, Tx) –Selection Biases Patients/Health care systems Are we really seeing the same patients in historical studies as those in active control trial?

16. Problems With Meta-Analyses So What Is Sponsor to DO? If we plan to use placebo controlled trials, what should we require of the historical placebo trials? Same Disease/Conditions? Same Population Same Dose and Administration Levels of Active Control C? Same Outcomes? Combine All or Some (good) Placebo Controlled Studies

17. Still Other Problems With Meta-Analyses What if previous studies had multiple arms? How to put correctly into meta-analysis? What if none of the individual studies achieved significance? What are we to believe from meta-analyses? Do we believe the p-levels of the meta-analysis? (I do not think we should.)

18. APPROACH 2 NON-INFERIORITY STUDIES ACTIVE CONTROL STUDIES NON-INFERIORITY TEST H 0 : T-C >= M vs. H 1 : T-C < M (Say data are event rates) T is new treatment C is positive control M IS NON-INFERIORITY MARGIN

19. NON-INFERIORITY STUDIES APPROACH 2 SELECT A VALUE OF M THAT MAKES SENSE WANT ASSURANCE THAT ASSAY SENSITIVITY IS PRESENT (Placebo is working) WANT CONSISTENCY WITH PAST

20. NON-INFERIORITY STUDIES Statistical Approach 1.Need Active Control C vs. Placebo P data from Historical data (C vs. P) 2.Need to test effectiveness of T vs. C 3.Need estimate of fraction of C-P preserved by T (e.g., (T-P)/(C-P) = M) M=0.5 (C-P) METHODS EXIST THAT ALLOW TEST TO BUILD IN NEW AND HISTORICAL DATA (STATISTICS IN MEDICINE, 2002)

21. WHAT IS NEEDED FOR 2 CONFIDENCE INTERVAL IS OFTEN USED. WANT M=1.11 (SAY) OUTSIDE UPPER LIMIT OF CONFIDENCE INTERVAL (M is relative risk) FDA ODAC 8/04 ( non-small cell lung cancer) 1.01.11= M

22. SOME REALITIES Sounds nice What happens

23. Anti-infective Product No placebo data Historical data is not Placebo, but C VRE (vancomycin resistant enterococcal) High dose Low dose MITT 60.0 % (N=65) vs. 46.2 % (N=52) Bacteremic 55.6 (N=18) vs. 25.0 (N=16) What is M? One trial OK? Any superiority?

24. ANOTHER EXAMPLE Respiratory Distress Respiratory Distress Syndrome in Premature Infants –Treatments New Drug Comparator –Outcome Survival at 28 Day

25. Respiratory Distress (cont) Survanta versus Sham (two studies one positive, other negative) All Cause mortality Study 1: 8% vs. 23% Study2: 17% vs. 14% What is M?.23-.08?.180-.125?

26. CONSISTENCY Example Control rate different from historical Historical Data says C=0.5 and P=0.6 Want T<=0.55 P-C=0.10, M=0.5(0.10) = 0.05 (T-C)/C = 0.05/0.50 = 10% Data is C=0.30 and T=0.33, T-C=0.03 (T-C)/C = 0.03/0.30 = 10% IS STUDY A SUCCESS? USE RATIOS?

27. ANSWER TO CONSISTENCY There was consistency Differences related to birth weight

28. Non-Inferiority and Inferiority at the same time Sponsor falls apart 0 M

29. Non-Inferiority and Superiority Sponsor jumps for joy (Sequential test) 0M

30. Switching trial design (Cardiac Stent Trials) (1) New drug coated stents, we can do non- inferiority study with margin set (15%) (2) We can do superiority study with non- coated stent as control With first option we have to worry about evaluating Ms, Effect size and CREEP With superiority trial clean results

31. Respiratory Distress Compare new surfaxin to another not so great one, but still used in practice

32. Switching from Superiority to non- Inferiority HOW CAN WE SWITCH FROM A SUPERIORITY TEST TO NON- INFERIORITY ? This is a question thrown at me constantly

33. Assessing Efficacy Non- Inferiority and Safety Superiority Carotid artery Magnetic Resonance Imaging agent Imaging Agents –Agent N (New) Agent C (Comparator) Non-inferiority Outcome –Endpoint: agents ability to classify correctly patients with > 25% stenosis (sensitivity) –Sensitivity of Comparator is.80 or 80% –Non-inferiority margin M set to 0.10

34. Assessing Efficacy Non-Inferiority and Safety Superiority (Contd) There is a specific adverse event that is hypothesized to occur less often with New than with Comparator –Do we want to make the specific adverse event rate an additional primary endpoint? WHY NOT?

35. Non US STUDIES Forced off shore (ethical and other reasons)

36. The BLOB EFFECT Everything is suddenly Non-Inferiority

37. ALLHAT STUDY COMPARISON OF ANTI- HYPERTENSIVE MEDICATIONS (MULTIPLE ARMS) NOT A NON-INFERIORITY STUDY

38. Safety Studies Safety studies have become carefully designed and executed studies Should they be non-inferiority studies?

39. SAFETY STUDIES (PHASE 4) HISTORICAL APPROACH: NEW RATE > OLD H 01 : T-C 0 H 02 : RR=T/C 1 STUDY POWERED TO REJECT T/C >1.5 (SAY) SHIFT IS TO MAKING THESE NON- INFERIORITY STUDIES H 0 : T-C >= M vs. H 1 : T-C < M H 0 : RR=T/C >= M vs. H 1 : RR=T/C < M

40. Safety Studies OLD NEW 1 M

41. SAFETY STUDY TO NON- INFERIORITY STUDY (QT LONGATION) Safety issue: drug may cause QT problem Ho: A/B = 1.0 vs H1: R = A/B > 1.0 Study powered for R > 1.0 When interest in risk fades can we suddenly say this should be a non-inferiority study? Ho:R >= 1.5 vs. H1:R < 1.5 was not original objective If we do not reject Ho is that enough?

42. Form of Interest and Sample Size Ho: p1-p2 >= M Ho: p1-p2>=Rp2 Ho: p1/p2 >= R Best Choice does depend on p2 (control rates)

43. Intent-to-Treat vs. Per-Protocol In superiority trials, the primary analysis is often on intent-to-treat (ITT) population Per Protocol (PP) bigger differences of treatments In non-inferiority should we use PP?

44. Intent-to-Treat vs. Per-Protocol (Contd) PP as primary not always accepted –the ITT analysis is as important as the PP analysis –need to reconcile differences between ITT and PP analysis –Perform sensitivity analyses. Results should be similar in both populations (ROBUSTNESS). –The Committee on Proprietary Medicinal Products draft Points to Consider: …similar conclusions from both the ITT and PP are required in a noninferiority trial.

45. We ask sponsor to do both (ITT and PP) and expect to achiev the sam significant result on both. What is the true alpha associated with this?

46. NEW MAJOR ISSUES Missing Data Noncompliance Interim Analysis OUR USUAL LOGIC INCREASES CHANCE OF ACCEPTANCE OF non- inferiority

47. MORE NEW ISSUES Multiple endpoints Multiple groups Repeated Measures

48. WHERE ARE WE? NON-INFERIORITY TRIALS HAVE MADE A BIG IMPACT They have brought many new problems and challenges with them