1. Generalized pairwise comparisons of prioritized outcomes Marc Buyse, ScD marc.buyse@iddi.com

2. The Wilcoxon test, and generalizations Generalized pairwise comparisons Universal measure of treatment effect An example Conclusions Outline

3. General Setup Let X i be the continuous outcome of the i th subject in T (i = 1, …, n ) R Eligible subjects Control (C )Treatment (T ) Let Y j be the continuous outcome of the j th subject in C (j = 1, …, m )

4. The Wilcoxon test statistic can be derived from all possible pairs of subjects, one from T and one from C. Let Wilcoxon-Mann-Whitney test statisticW The Mann-Whitney form of the Wilcoxon test

5. The Wilcoxon test can be generalized to the case of censored outcomes. Letting and denote censored observations, the pairwise comparison indicator is now Gehan generalized the Wilcoxon test

6. Now let X i and Y j be observed outcomes for ANY outcome measure (continuous, time to event, binary, categorical, …) First, generalize the test further for a single outcome measure YjYj XiXi favors T (favorable) favors C (unfavorable) neutral uninformative pairwise comparison

7. Binary outcome measure Pairwise comparisonPair is X i = 1, Y j = 0favorable X i = 1, Y j = 1 or X i = 0, Y j = 0neutral X i = 0, Y j = 1unfavorable X i orY j missinguninformative

8. Pairwise comparisonPair is X i  Y j >  * favorable  X i  Y j  ≤  * neutral X i  Y j <   * unfavorable X i orY j missinguninformative *  chosen to reflect clinical relevance;  = 0 is Wilcoxon test Continuous outcome measure

9. Time to event outcome measure Pairwise comparisonPair is X i  Y j >  * or  Y j >  * favorable  X i  Y j  ≤  * neutral X i  Y j <   * or X i  <   * unfavorable otherwiseuninformative *  chosen to reflect clinical relevance;  = 0 is Gehan test

10. Let X i and Y j be VECTORS of observed outcomes for any number of occasions of a single outcome measure, or any number of outcome measures. We assume that the occasions and/or the outcome measures can be prioritized. Generalized pairwise comparisons

11. Next, generalize the test to prioritized repeated observations of a single outcome measure… Occasion with higher priority Occasion with lower priority Pair is favorableignoredfavorable unfavorableignoredunfavorable neutralignoredneutral uninformativefavorable uninformativeunfavorable uninformativeneutral uninformative

12. Last, generalize the test to several prioritized outcome measures… Outcome with higher priority Outcome with lower priority Pair is favorableignoredfavorable unfavorableignoredunfavorable neutralignoredneutral uninformativefavorable uninformativeunfavorable uninformativeneutral uninformative

13. Extend the previous definition of U ij U is the difference between the proportion of favorable pairs and the proportion of unfavorable pairs. We call this general measure of treatment effect the « proportion in favor of treatment » (  ). A general measure of treatment effect

14.  is a linear transformation of the probabilistic index, P (X > Y ) : The proportion in favor of treatment (  ) SituationP (X > Y )  T uniformly worse than C0 11 T no different from C0.50 T uniformly better than C1+1

15. For a binary variable,  is equal to the difference in proportions For a continuous variable,  is related to the effect size d For a time-to-event variable,  is related to the hazard ratio and the proportion of informative pairs f The proportion in favor of treatment (  )

16. A re-randomization test for  The test statistic U (or  ) no longer has known expectation and variance. An empirical distribution of  can be obtained through re- randomization. Tests of significance and confidence intervals follow suit.

17. The proportion in favor of treatment for the l th prioritized outcome (l = 1,..., L ) is given by and the cumulative proportion is Cumulative proportions for prioritized outcomes

18. Early breast cancer two combination chemotherapies plus herceptin R 3,222 patients after curative resection of HER2+ breast cancer Adriamycin Cyclophosphamide Taxotere (ACT) Taxotere Carboplatin Herceptin (TCH) standard chemotherapy 1,0731,075 main efficacy endpoints disease recurrence or death main safety endpoint congestive heart failure Adriamycin Cyclophosphamide Taxotere Herceptin (ACTH) 1,074

19. 87% 81% 78% 75% 92% 87% 84% 81% 93% 88% 86% 84% Disease-free survival

20. Prioritized outcomes PriorityOutcomes 1Time to death from any cause 2Time to second malignancy 3Time to distant metastases 4Time to locoregional relapse 5Time to congestive heart failure

21. Difference in ACTH better ACT better Cumulative  P-value * Time to death4.97%2.87%2.09%0.006 Time to second tumor1.20%1.21%2.08%0.022 Time to distant mets7.03%3.46%5.66%< 0.001 Time to relapse1.82%1.01%6.47%< 0.001 Time to CHF0.62%1.83%5.25%< 0.001 Prioritized outcomes GENERALIZED PAIRWISE COMPARISONS ACTH vs. ACT * Unadjusted for multiplicity

22. Difference in TCH better ACT better Cumulative  P-value * Time to death5.05%3.49%1.56%0.059 Time to second tumor1.22%0.72%2.05%0.029 Time to distant mets7.18%3.96%5.26%< 0.001 Time to relapse1.75%1.47%5.55%< 0.001 Time to CHF0.63%0.71%5.47%< 0.001 Prioritized outcomes GENERALIZED PAIRWISE COMPARISONS TCH vs. ACT * Unadjusted for multiplicity

23. Difference in TCH better ACTH better Cumulative  P-value * Time to death3.04%3.57%-0.53%0.46 Time to second tumor1.29%0.74%0.02%0.98 Time to distant mets3.84%4.36%-0.50%0.68 Time to relapse1.04%1.63%-1.09%0.40 Time to CHF1.97%0.74%0.14%0.93 Prioritized outcomes GENERALIZED PAIRWISE COMPARISONS TCH vs. ACTH * Unadjusted for multiplicity

24. 1.are equivalent to well-known non-parametric tests in simple cases 2.allow testing for differences thought to be clinically relevant 3.allow any number of prioritized outcomes of any type to be analyzed simultaneously 4.naturally lead to a universal measure of treatment effect, , which is directly related to classical measures of treatment effect (difference in proportions, effect size or hazard ratio) Generalized Pairwise Comparisons

25. References Buyse M. Generalized pairwise comparisons for prioritized outcomes in the two-sample problem. Statistics in Medicine 29:3245-57, 2010. Buyse M. Reformulating the hazard ratio to enhance communication with clinical investigators. Clinical Trials 5:641-2, 2008.