1 Multiplicity Adjustment For Clinical Trials with Two Doses of an Active Treatment and Multiple Primary and Secondary Endpoints Hui Quan 1, Tom Capizzi 1, Ji Zhang 2 1 Merck Research Laboratories 2 Sanofi-Synthelabo Research FDA/Industry Statistics Workshop September 21-23, 2004

2 Outline zBackground and clinical trial setting zProcedures for a two-dimensional problem zProcedures for a three-dimensional problem zNumerical examples for comparing powers zReal data example zProcedures without overall strong control zDiscussion

3 Background zMultiple primary and secondary endpoints are often simultaneously considered in Phase III trials. zPositive findings on secondary endpoints could be for: 1: Supporting primary results or Hypotheses generating 2: Additional claims or description in drug label zFor category 2 recent regulatory guidances and publications emphasized to have Type I error rate controlled for secondary endpoints.

4 Background (2) Different approaches for interpreting trial results: zConventional standard for positive trial: positive effect on primary endpoints. zOthers: a trial as positive even when effect is positive only on a secondary endpoint secondary endpoints must be considered in multiplicity adjustment. For example: Moyé (2000) proposed a PAAS for controlling the FWE rate α F with α P for the primary endpoints and α F - α P = α S for the secondary endpoints.

5 Trial Setting Trial examples: zParathyroid hormone fracture trial (Neer et al. 2000): 20 and 40 μg vs. placebo on vertebral fracture and other fractures zRofecoxib RA trial (Geunese, et al. 2002): 25 and 50 mg vs. placebo on four primary and other secondary endpoints zEstrogen Alzheimer trial (Mulnard, et al. 2000): and 1.25 mg vs. placebo on CGIC and other scores zMetformin and rosiglitazone combination trial (Fonseca, et al. 2000): Rosiglitazone 4 and 8 mg vs. placebo on glucose and lipid endpoints

6 Trial Setting (2) Three-dimensional multiplicity adjustment problem: Two doses Two priority categories of endpoints: primary & secondary Multiple endpoints in each priority category zHigher dose-the primary dose, lower dose-the secondary dose (no monotonic order on parameter space for dose response is assumed for any endpoint) zTrial positive only if positive on one or more primary endpoints – except when PAAS is used for multiplicity adjustment zNo further priority order for endpoints within each category

7 Trial Setting (3) zTraditional procedures may be lack of power. For example: for a trial with two primary and two secondary endpoints-a total of 8 active to control comparisons, if a Hochberg procedure is used and if the lower dose shows no effect on the two secondary endpoints, the higher dose on the primary endpoints will be assessed at most at level of /3. zSince no monotonic order in dose response is assumed, stepdown trend tests are not applicable. zMore specific gatekeeping procedures may have higher power.

8 Procedures for a Two-dimensional Problem: Two doses and multiple endpoints H ij is the null hypothesis for comparing the i th dose to the control on the j th endpoint, and p ij is the corresponding p-value, i=1, 2 and j=1, 2, …, g. (Dose 1: the higher and primary dose) Modified Bonferroni-Closed Procedure (2.1) z Step1. Reject all H ij if all p ij, i=1,2, j=1,…, g. Otherwise, z Step 2. Reject H 1j if p 1j /g Reject H 2j if p 1j /g and p 2j /g

9 Modified Hochberg-Bonferroni Procedure (2.2) zStep 1. Reject all H ij if all p ij, i=1,2, j=1,…, g. Otherwise, zStep 2. Use Hochberg procedure for the higher dose at =2 /3. zStep 3. Let l =# of endpoints of no effect for the higher dose. If l=0, use a Hochberg procedure for the lower dose at level. Otherwise, zStep 4: a) Use Hochberg procedure for the lower dose at with all endpoints; b) Assume no effect on the l (>0) endpoints for the lower dose; c) Use Hochberg procedure for the lower dose on the other (g-l) endpoints at level /3; then d) Claim the effect for the lower dose on an endpoint if the effect is significant at both a) and c).

10 Weighted Modified Bonferroni-Closed Procedure (2.3) zStep 1. Reject all H ij if all p ij Otherwise, zStep 2. Reject H 1j if p 1j w j ; reject H 2j if p 1j w j and p 2j w j. where w j 0 and Σw j =1 Applicable to a three-dimensional problem by assigning more weights on the primary and less weights on the secondary endpoints. However, different from PAAS, no need to split the significance level if all p ij.

11 Properties of Procedures (2.n) zReject all the hypotheses if all p-values are. zStep down from the higher dose to the lower dose to form closed procedures. zTheoretically or based on simulation, provide strong control on the FWE rate under appropriate conditions. for (2.1) and (2.2): the condition for the Simes test for (2.3): positive regression dependency condition for the weighted Simes test.

12 Procedures for the Three-dimensional Problem H ij is the null hypothesis for comparing the i th dose to the control on the j th primary endpoint, i=1, 2 and j=1,…, g p S ik is the null hypothesis for comparing the i th dose to the control on the k th secondary endpoint, i=1, 2 and k=1,…, g s. Following Westfall and Krishen (2001), Gatekeeping Procedure (3.1) for g p =1 and g s =1 (a) { H 11 } (b) { H 21, S 11 } (c) { S 21 } level α for each { }.

13 Generalized Gatekeeping Procedure (3.2) (a) { H 1j } (b) {[ H 2j ], [ S 1k ]} One-dimensional (c) { S ik } Two-dimensional One-dimensional procedure at (a) and (b), and two-dimensional procedure at (c). z Reject H 1j ( H 2j ) if it should be rejected at (a) ((b)) z Reject S 1k if it should be rejected at both (b) and (c). z Reject S 2k if all H 2j s have been rejected, S 1k has been rejected at (b) and (c), and S 2k should be rejected at (c).

14 Other Procedures for the Three-dimensional Problem z PAAS: { H ij } two-dimensional at level α p and { S ik } two-dimensional at level α s (α p+ α s= α) z Primary-Secondary Gatekeeping: (a) { H ij } two-dimensional (b) { S ik } two-dimensional each at level α z Procedure (2.2) or others for the two-dimensional Problems

15 Numerical Examples (R=Rejection, NR=No Rejection) 4 Procedures are compared to the regular Hochberg Procedure P1=PAAS with P(2.2) for primary and secondary endpoints P2=P(2.3) with weight wp for primary and ws for secondary P3=P(3.2) with P(2.1) for Step (c) P4=P(3.2) with P(2.2) for Step (c)

16 Numerical Example 1 g p =1, g s =3, αp=.0375, αs=.0125, wp=3/6 and ws=1/6 H 11 H 12 S 11 S 12 S 13 S 21 S 22 S 23 P-value P1RRNR P2RRRNR P3RRRRNRR P4RRRRNR HBRNRR

17 Numerical Example 2 g p =2, g s =2, αp=.0375, αs=.0125, wp=3/8 and ws=1/8 H 11 H 12 H 21 H 22 S 11 S 12 S 21 S 22 P-value P1RRRNR P2RRRNRR P3RRRRRNRR P4RRRRRRRNR HBRNR R

18 Real Data Example zA randomized trial to compare mitoxantrone 12 mg/m 2 (n=64) or 5 mg/m 2 (n=64) to placebo (n=60) on patients with worsening relapsing remitting or secondary progressive multiple sclerosis (Hartung et al. 2002). zHere, two primary and two secondary endpoints for which p-values for both doses are available: EDSS, ambulation index, with relapses and admitted to hospital.

19 Real Data Example (2) g p =2, g s =2, αp=.040, αs=.010, wp=4/10 and ws=1/10 H 11 H 12 H 21 H 22 S 11 S 12 S 21 S 22 P-value > P1NR R P2RNRR R P3RRRNRRR P4RRRNRRR HBNR R

20 Procedures without Overall Strong Control zStrong control is theoretically appealing. However, the procedures are generally conservative. zStrong control in certain key sub-families of hypotheses may be sufficient. Procedure (*) (a) {H 1j H 2j } two-dimensional at level α (b) {S 1k S 2k } two-dimensional at level α Procedure (*) strongly controls the FWE rate in the sub- families of individual doses, individual endpoints, within the families of primary endpoints and secondary endpoints. Also, it weakly controls the overall FWE rate.

21 Procedures without Overall Strong Control (2) Procedure (*) does not strongly control the overall FWE rate. When g p =1 and g s =1, Procedure (*) becomes {H 11 } {H 21 } each { } at level α {S 11 } and {S 21 } (p ij is the p-value for H ij and q ik is the p-value for S ik ) The probability of rejecting H 21 S 11 when H 21 S 11 is true is Sup Pr [(p 11 α)(p 21 α) or (p 11 α)(q 11 α) | H 21 S 11 ] =Sup Pr[(p 21α) or (q 11α) | H 21 S 11 ]2α-α 2

22 Other Procedures without Strong Control For example: Step-down Closed Procedure: zStep a. Apply a procedure for each dose vs. the control separately at level for all primary and secondary endpoints. zStep b. Reject H 1j (or S 1k ) if it should be rejected at Step a. Reject H 2j (or S 2k ) if both H 1j (or S 1k ) & H 2j (or S 2k ) should be rejected at Step a. It controls the FWE rates in the strong sense within sub-families of individual doses and endpoints, but not the family of the primary endpoints nor the family of the secondary endpoints. It controls the overall FWE rate in the weak sense.

23 Discussion zAfter Phase IIb dose finding study, Phase III confirmatory trials may still have two doses of the active treatment. The setting considered here should have very broad application. zAs shown by numerical examples (here and in our manuscript), no procedure is uniformly more powerful than the others. zAll procedures need only individual p-values -- no need to calculate the adjusted p-values based on different configurations of intersection hypotheses. zOne should think hard whether the overall strong control is necessary for a particular trial.

24 Major References zBenjamini Y and Hochberg T. Multiple hypotheses testing with weights. Scandinavian Journal of Statistics 1997; 24: zCPMP Points to Consider on Multiplicity Issues in Clinical Trials zDAgostino RB. Controlling alpha in a clinical trial: the case for secondary endpoints. Statistics in Medicine 2000; 19, zDmitrienko A, Offen WW and Westfall PH. Getekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine 2003; 22: zHartung HP, Gonsette R, Konig N, Kwiecinski H, Guseo A, Morrissey SP, Krapf H, Zwingers T and the Mitoxantrone in Multiple Sclerosis Study Group. Mitoxantrone in progressive multiple sclerosis: a placebo- controlled, double-blind, randomized, multicentre trial. Lancet 2002; 360, zHochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75, zMacus R, Peritz E and Gabriel KR. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 64, zMoye LA. Alpha calculus in clinical trials: considerations and commentary for the new millennium. Statistics in Medicine 2000; 19, zNeer RM, Arnaud CD, Zanchetta JR, Prince R, Gaich GA, Reginster JY, Hodsman AB, Eriksen EF, Ish-Shalom S, Genant HK, Wang O and Mitlak BH. Effect of parathyroid hormone (1-34) on fractures and bone mineral density in postmenopausal women with osteoporosis. The New England Journal of Medicine 2001; 344, 19, Quan H, Luo E, Capizzi T. Xun Chen, Lynn Wei, and Bruce Binkowitz, Multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active control. JSM Proceedings zSamuel-Cahn, E. Is the Simes improved Bonferroni procedure conservative? Biometrika 1996; 83, zSimes RJ. An improved Bonferroni procedure for multiple tests of significance. Biometrika, 1986; 63: zTreanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, Singh S, Kinnersley N, Ward P and Mills RG. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza. JAMA 2000; 283, 8, zWestfall PH and Krishen A. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference 2001; 99,