1. What’s the win ratio? Victoria Chang, AbbVie Gaohong Dong, iStats Inc.
Junshan Qiu, US FDA
Roland Matsouaka, Duke University
David Hoaglin, Univ. Massachusetts Medical School
JSM – July 29, 2018

2. Recent evelopment Methods Applications
Pocock et al., 2012: Original proposal
Luo et al., 2015: Closed-form variance estimator for one particular way of defining a win/loss/tie
Bebu and Lachin, 2016; Dong, Li, et al., 2016: Closed-form variance estimator for any way of defining a win/loss/tie
Wang and Pocock, 2016: Win ratio for non-normal outcomes
Oakes, 2016: Win and loss probabilities in a limited time horizon
Luo et al., 2017: Weighted win and loss
Dong, Qiu, et al., 2018: The stratified win ratio
Applications
Post-hoc analyses: Rogers et al., 2014; Prakash et al., 2014; Bakal et al., 2015; Abdalla et al., 2016; Dong et al., 2016; Milojevic et al., 2017; Fergusson et al., 2018.
Study design: Montgomery et al., 2014; Henry et al., 2016.
| July 29, 2018 | What's the win ratio I JSM

3. Outline Motivation ( with two examples) Application of the win ratio
Interpretation and connections to HR and OR
Summary
| July 29, 2018 | What's the win ratio I JSM

4. Example 1: CHARM program
Three randomized trials comparing treatment with placebo in subjects with chronic heart failure (CHF) on three patient types:
Completed in 2003.
7599 subjects with median follow-up 3.14 years
Primary endpoint: Composite of cardiovascular (CV) death or hospitalization for CHF
Trials
Type of patients
CHARM Added
ejection fraction <40%
on ACE-inhibitor
CHARM Alternative
Intolerant to ACE-inhibitor
CHARM Preserved
ejection fraction ≥40%
Source: Pocock et al. (2012)
| July 29, 2018 | What's the win ratio I JSM

5. Example 1: CHARM program (cont.)
Added
Alternative
Preserved
Adjusted HR
0.85
0.70
0.86
95% CI
p-value
0.010
<0.0001
0.051 C Pl
No. of patients
1276
1272
1913
1015
1514
1509
No. with primary composite event
483
538
334
406
333
366
No. of CV death used
174
182
127
120 92 90
Total no. of CV death
302
347
219
252
170
Only 54% of CV deaths contributed to the composite, because it happened later than hospitalization Q: Could all CV deaths be considered for the analysis?
| July 29, 2018 | What's the win ratio I JSM
Source: Pocock et al. (2012)

6. Example 2: Phase III Liver Transplant Trial
Randomized
Treatment: EVR + reduced TAC exposure (243 patients)
Control: standard TAC exposure (245 patients)
Completed in 2013
Using the results at Month 12 in the example
Primary endpoint: a composite of
Treated biopsy-proven acute rejection (tBPAR)
Graft loss (GL)
Death (D)
Data Source: De Simone et al. (2012)
| July 29, 2018 | What's the win ratio I JSM

7. Example 2: Phase III Liver Transplant Trial (Cont.)
Endpoints_______ _Favors Treatment____ _Favors control_____ Risk difference (%) (97.5%CI) BPAR (-23.4,-0.1) tBPAR/GL/D/LTFUP -5.0 (-11.9, 1.8) tBPAR -4.2 (-8.7, 0.3) tBPAR/GL/D -3.0 (-8.7, 2.6) Death (D) 1.2 (-2.4, 4.8) Graft Loss (GL) 1.3 (-1.6, 4.1) GL/D 2.0 (-2.0, 6.1)
GL & Death were in the opposite direction from tBPAR/GL/D.
Q: Was the numerical difference in GL/D a chance finding?
Risk difference (%)
| July 29, 2018 | What's the win ratio I JSM

8. Who wins? Imagine two patients... Control Treat Time Hosp Death Hosp
| July 29, 2018 | What's the win ratio I JSM

9. Who wins? Imagine two patients... Control Treat Time Hosp Hosp Death
| July 29, 2018 | What's the win ratio I JSM

10. Who wins? Imagine two patients... Control Treat Time Hosp Death Hosp
| July 29, 2018 | What's the win ratio I JSM

11. Win ratio (unmatched trials)
Based on pairwise comparisons: each patient in the Treatment group is compared with every patient in the Control group.
TRT win Con win Tied
Control Patient 1
Control Patient 2
Treatment Patient 1
… …
Control Patient Nc
… …
… …
Control Patient 1
Control Patient 2
Last Treatment Patient Nt
… …
Control Patient Nc
| July 29, 2018 | What's the win ratio I JSM

12. Win ratio (cont.)
For each pair,
Most important endpoint
Win (loss)
Can not determine
less important endpoint
Win (loss)
W𝑖𝑛 𝑟𝑎𝑡𝑖𝑜= 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑤𝑖𝑛𝑠 𝑓𝑜𝑟 𝑇𝑟𝑒𝑎𝑡𝑒𝑚𝑒𝑛𝑡 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑤𝑖𝑛𝑠 𝑓𝑜𝑟 𝐶𝑜𝑛𝑡𝑟𝑜𝑙
| July 29, 2018 | What's the win ratio I JSM

13. Example 2: Phase III Liver Transplant Trial (Cont.)
Importance order: Death, graft loss, tBPAR
243 patients in Treatment vs 245 patients in Control
Pairs: 243 x 245 = 59535,
Wins for Treatment, per death =
Wins for Control, per death =
Wins for Treatment, per graft loss =
Wins for Control, per graft loss =
Wins for Treatment, per tBPAR =
Wins for Control, per tBPAR =
Ties = 50381
| July 29, 2018 | What's the win ratio I JSM

14. Example 2: Phase III Liver Transplant Trial (Cont.)
Importance order: Death, graft loss, tBPAR
243 patients in Treatment vs 245 patients in Control
Pairs: 243 x 245 = 59535,
Wins for Treatment, per death =
Wins for Control, per death =
Wins for Treatment, per graft loss =
Wins for Control, per graft loss =
Wins for Treatment, per tBPAR =
Wins for Control, per tBPAR =
Ties = 50381
| July 29, 2018 | What's the win ratio I JSM

15. Example 2: Phase III Liver Transplant Trial (Cont.)
Importance order: Death, graft loss, tBPAR
243 patients in Treatment vs 245 patients in Control
Pairs: 243 x 245 = 59535,
Wins for Treatment, per death =
Wins for Control, per death =
Wins for Treatment, per graft loss =
Wins for Control, per graft loss =
Wins for Treatment, per tBPAR =
Wins for Control, per tBPAR =
Ties = 50381
Win ratio: WR = nt /nc = 5393/3761 = 1.43
95% CI: (0.73, 2.75)

16. Win ratio: ratio of win proportions
Another way to interpret the result
Win proportions of two groups:
Treatment group: Pt = nt / NtNc = 9.1%
Control group: Pc = nc / NtNc = 6.3%
Ties: 84.6% (= 100% – 9.1% - 6.3%)
W𝑖𝑛 𝑟𝑎𝑡𝑖𝑜= 𝑛 𝑡 𝑛 𝑐 = 𝑛 𝑡 / 𝑁 𝑡 𝑁 𝑐 𝑛 𝑐 / 𝑁 𝑡 𝑁 𝑐 = 𝑃 𝑡 𝑃 𝑐
| July 29, 2018 | What's the win ratio I JSM

17. Win ratio: Connection to OR and HR
Win ratio = odds ratio in favor of treatment (Dong et al., 2018)
for a binary endpoint ignoring time-to-event information.
When the importance order among the components and time-to-event information are ignored.
Win ratio = Hazard ratio (HR) in favor of treatment, when there is no censoring for a single time-to-event outcome.
Group
Number of patients
Without event
With event
Total
Treatment
𝒏 𝟏𝟏
𝒏 𝟏𝟐
𝑵 𝒕
Control
𝒏 𝟐𝟏
𝒏 𝟐𝟐
𝑵 𝒄
W𝑖𝑛 𝑟𝑎𝑡𝑖𝑜= 𝑛 𝑡 𝑛 𝑐 = 𝑛 11 𝑛 22 𝑛 12 𝑛 21 =𝑂𝑑𝑑𝑠 𝑟𝑎𝑡𝑖𝑜
| July 29, 2018 | What's the win ratio I JSM

18. The stratified win ratio
Stratified randomization is usually performed for clinical trials.
Patients within a stratum is comparable in terms of the prognostic factors.
The stratified win ratio can avoid biases caused by an unstratified analysis.
SAS and R programs can be downloaded from the publisher’s website
| June 16, 2018 | What's the win ratio

19. A Generalized Analytic Approach
Rules determining winners/losers/ties are critical, however, there is no a set of rules that can fit all disease indications.
The paper proposed a way to perform a win ratio analysis regardless what rules are used, so users can use their own rules fitting their projects.
SAS and R programs can be downloaded from the publisher’s website
2018 ICSA Applied Statistics Symposium | A Generalized Analytic Solution to the Win Ratio

20. Summary
Conventional analysis for composite endpoint uses first event, which may not be the most important outcome
Win ratio prioritize the most major component and more of these major events are included in the analysis
In some cases, the result using win ratio may give a stronger signal than hazard ratio because more critical events were included.
It provides an alternative way to analyze composite endpoints.
Easy to interpret: the ratio of numbers of wins for two groups
It is also the ratio of win proportions for two groups
It has a connection to the odds ratio and the hazard ratio
| July 29, 2018 | What's the win ratio I JSM

21. Acknowledgements L.J. Wei for our early publications
Stuart J. Pocock for our recent research development
Anonymous referees of our published papers and paper(s) currently under review
| July 29, 2018 | What's the win ratio I JSM

22. Reference
Abdalla S, Montez-Rath ME, Parfrey PS, Chertow GM. (2016). The win ratio approach to analyzing composite outcomes: An application to the EVOLVE trial. Contemp Clin Trials 48:
Agresti, A. and Min, Y. (2004). Effects and non‐effects of paired identical observations in comparing proportions with binary matched‐pairs data. Statistics in medicine, 23(1):
Bakal JA, Roe MT, Ohman EM, Goodman SG, Fox KA, Zheng Y, Westerhout CM, Hochman JS, Lokhnygina Y, Brown EB and Armstrong PW. (2015). Applying novel methods to assess clinical outcomes: insights from the TRILOGY ACS trial. Eur Heart J. 36(6): a.
Bakal JA, Westerhout CM, Armstrong PW. (2015). Impact of weighted composite compared to traditional composite endpoints for the design of randomized controlled trials. Stat Methods in Medical Research 24(6):
Bebu I, Lachin JM. (2016). Large sample inference for a win ratio analysis of a composite outcome based on prioritized components. Biostatistics 17(1):
Buyse M. Reformulating the hazard ratio to enhance communication with clinical investigators. Clin Trials ;5(6):
Buyse M. (2010). Generalized pairwise comparisons of prioritized outcomes in the two-sample problem. Stat Med. 29(30):
Cochran, W.G., The comparison of percentages in matched samples. Biometrika, 37(3/4):
de Wreede, L. C., Fiocco, M., & Putter, H. (2011). mstate: an R package for the analysis of competing risks and multi- state models. Journal of Statistical Software, 38:7.
De Simone, P et al. “Everolimus With Reduced Tacrolimus Improves Renal Function in De Novo Liver Transplant Recipients: A Randomized Controlled Trial.” American Journal of Transplantation 12.11 (2012): 3008–3020. PMC. Web. 25 June 2018.
Dong G, Li D, Ballerstedt S, Vandemeulebroecke M. (2016). A generalized analytic solution to the win ratio to analyze a composite endpoint considering the clinical importance order among components. Pharmaceutical Statistics 15(5): 430–437.
| June 16, 2018 | What's the win ratio

23. Reference
Dong G, Guo M, Hoaglin DC, Vandemeulebroecke M. (2016a). “Is the asymptotic variance estimator of the logarithm of the win ratio under the null hypothesis sound? – A note on the generalized analytic solution to the win ratio by Dong et al. (2016)”. Supporting information #7 (a post-publishing note) to Dong et al. (2016) ( /store/ /pst.1763/asset/supinfo/pst1763-sup-0007-supplementary.pdf?v=1&s= cd757e23cb40556b3a1511b53c73747fe06fae89). Pharmaceutical Statistics 15(5): 430–437.
Dong G, Qiu J, Wang D, Vandemeulebroecke M. (2017). The stratified win ratio. Journal of Biopharmaceutical Statistics, DOI: /
Duc AN, Wolbers M. (2017). Weighted analysis of composite endpoints with simultaneous inference for flexible weight constraints. Stat Med. 10;36(3):
Finkelstein DM, Schoenfeld DA. Combining mortality and longitudinal measures in clinical trials. Statistics in Medicine 1999 Jun 15;18(11):
Gehan, E. A. (1965). A generalized wilcoxon test for comparing arbitrarily singly-censored samples. Biometrika 52, 203–223
Henry TD, Schaer GL, DeMaria A, Recker D, Remmers AE, Goodrich J, Patel A. (2016). The iXCELL-DCM Trial: Rationale and Design. Cell Transplant 25(9):
Lu, Y., (2010). A revised version of McNemar's test for paired binary data. Communications in Statistics—Theory and Methods, 39(19):
Lu, Y., Wang, M. and Zhang, G., (2017). A new revised version of McNemar's test for paired binary data. Communications in Statistics-Theory and Methods, 46 (20):
Luo X, Tian H, Mohanty S, Tsai WY. (2015). An alternative approach to confidence interval estimation for the win ratio statistic. Biometrics 71(1):
| June 16, 2018 | What's the win ratio

24. Reference
Luo X, Qiu J, Bai S, Tian H. (2017). Weighted win loss approach for analyzing prioritized outcomes. Stat Med. 36(15):
Mann H, Whitney D. (1947). On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Annals of Mathematical Statistics. 18 (1): 50–60.
McNemar, Q. (1947). Note on the Sampling Error of the Difference between Correlated Proportions or Percentages. Psychometrika 12:153–157
Milojevic M, Head SJ, Andrinopoulou ER, Serruys PW, Mohr FW, Tijssen JG, Kappetein AP. (2017). Hierarchical testing of composite endpoints: applying the win ratio to percutaneous coronary intervention versus coronary artery bypass grafting in the SYNTAX trial. EuroIntervention 15;13(1):
Montgomery A, Abuan T, and Kollef M. (2014). The win ratio method: a novel hierarchical endpoint for pneumonia trials in patients on mechanical ventilation. 34th International Symposium on Intensive Care and Emergency Medicine March 2014, Brussels, Belgium.
Moser BK, McCann MH. (2008). Reformulating the hazard ratio to enhance communication with clinical investigators. Clin Trials 5(3):
Moye L. A., Davis B. R. and Hawkins, C. M. (1992). Analysis of a clinical trial involving a combined mortality and adherence dependent interval censored endpoint. Statistics in Medicine 11(13):
Oakes D. (2016). On the win-ratio statistic in clinical trials with multiple types of event. Biometrika 103(3): 742–745.
O’Brien, R. G., and Castelloe, J. M. Exploiting the Link between the Wilcoxon-Mann-Whitney Test and a Simple Odds Statistic. In Proceedings of the Thirty-First Annual SAS Users Group International Conference. Cary, NC: SAS Institute Inc
PEACE Trial Investigators. (2004). Angiotensin-converting-enzyme inhibition in stable coronary artery disease. New England Journal of Medicine 351(20):2058–2068.
| June 16, 2018 | What's the win ratio

25. Reference
Péron J, Roy P, Ozenne B, Roche L, Buyse M. (2016). The Net Chance of a Longer Survival as a Patient-Oriented Measure of Treatment Benefit in Randomized Clinical Trials. JAMA Oncology 2(7):
Péron J, Buyse M, Ozenne B, Roche L, Roy P. An extension of generalized pairwise comparisons for prioritized outcomes in the presence of censoring. Stat Methods Med Res Aug 2. pii:
Pfeffer MA, Swedberg K, Granger CB, Held P, McMurray JJV, Michelson EL, Olofsson B, Ostergren J, Yusuf S for the CHARM Investigators and Committees. (2003). Effects of candesartan on mortality and morbidity in patients with chronic heart failure: the CHARM-Overall programme. Lancet 362:759–766.
Pocock SJ, Ariti CA, Collier TJ, Wang D. (2012). The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. Eur Heart J. 33(2):
Prakash R, Horsfall M, Markwick A, Pumar M, Lee L, Sinhal A, Joseph MX and Chew DP. (2014). Prognostic impact of moderate or severe mitral regurgitation (MR) irrespective of concomitant comorbidities: a retrospective matched cohort study. BMJ Open 4(7):e
Pratt J. W., Gibbons J. D. (1981), Concepts of Nonparametric Theory, Springer-Verlag, New York
Ramchandani R, Schoenfeld DA, Finkelstein DM. (2016). Global rank tests for multiple, possibly censored, outcomes. Biometrics 72(3):
Randles, R. H. (2001), On neutral responses (zeros) in the sign test and ties in the Wilcoxon-Mann-Whitney test, The American Statistician, 55, 96–101.
Rauch, G., Jahn-Eimermacher, A., Brannath, W. and Kieser, M. (2014), Opportunities and challenges of combined effect measures based on prioritized outcomes. Statist. Med., 33: 1104–1120.
Rogers JK, Pocock SJ, McMurray JJ, Granger CB, Michelson EL, Östergren J, Pfeffer MA, Solomon SD, Swedberg K, Yusuf S. (2014). Analysing recurrent hospitalizations in heart failure: a review of statistical methodology, with application to CHARM-Preserved. Eur J Heart Fail 16(1):33-40.
| June 16, 2018 | What's the win ratio

26. Reference
Sun H, Kawaguchi A, Koch G. Analyzing multiple endpoints in a confirmatory randomized clinical trial-an approach that addresses stratification, missing values, baseline imbalance and multiplicity for strictly ordinal outcomes. Pharm Stat Mar;16(2):
Wang D, Pocock S. (2016). A win ratio approach to comparing continuous non-normal outcomes in clinical trials. Pharmaceutical Statistics 15(3):
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1: 80‐83.
| June 16, 2018 | What's the win ratio