A consideration of potential implications on health economic analyses of using adaptive designs Sue Todd Department of Mathematics and Statistics University of Reading, UK ICTMC/SCT Conference May 2017

Outline Types of adaptive designs Benefits and limitations Approaches to economic evaluation Consequences for economic evaluation of adaptive designs Conclusions ICTMC/SCT Conference May 2017

Types of adaptive designs DEFINITION: A trial which incorporates interim analyses where data are examined in order to make a decision concerning the rest of the trial’s conduct Can incorporate a wide range of procedures, for example in the confirmatory setting may see Sample size reassessment Group sequential tests Flexible adaptive designs Response adaptive designs ICTMC/SCT Conference May 2017

Use of adaptive designs Use of adaptive designs has grown due to the influence of ETHICS ECONOMICS ENTHUSIASM But there are issues with implementation surrounding ADMINISTRATION APATHY ANALYSIS ICTMC/SCT Conference May 2017

Practical issues with implementation ADMINISTRATION Timing of interim analyses Ensuring interim data quality How to budget appropriately for costs and people’s time APATHY More complex to design More complicated to organise More time involved in data collection and processing ICTMC/SCT Conference May 2017

Statistical issues with implementation ANALYSIS Conventional P-values may be inaccurate. Conventionally calculated point estimates may be biased and confidence intervals may have incorrect coverage (depending on type of AD) Although the meaning and interpretation of these traditional data summaries remain as for standard conventional designs, they may require alternative calculation in the setting of adaptive designs ICTMC/SCT Conference May 2017

Statistical issues with implementation Example bias plot (group sequential design) ICTMC/SCT Conference May 2017

Statistical issues with implementation P-values: E.g. Whitehead (2000); Bauer & Köhne (1994) Estimates / CIs - primary endpoint literature: E.g. Whitehead (1986a); Pocock & Hughes (1989); Todd et al. (1996); Emerson & Kittelson (1997); Shen (2001); Stallard et al. (2008); Posch et al. (2005); Bauer et al. (2009); Kimani et al. (2013, 2015) Estimates / CIs - secondary endpoint literature: E.g. Whitehead (1986b); Hall & Yakir (2000); Whitehead et al. (2000) Software exists E.g. EaSt; ADDPLAN; RCTdesign ICTMC/SCT Conference May 2017

Statistical issues with implementation Issues with analysis are therefore well documented, but in practice, are often not considered “Many trials which are terminated at an interim analysis are reported with analyses which take no statistical account of the interim analysis … Most authors present naïve estimates of the treatment effect, but these are known to be biased ... The analyst should also consider the impact of interim analyses on the analysis of secondary variables which are correlated with the primary variable assessed during interim analyses” Facey & Lewis (1998) “Of the 46 RCTs that were stopped early, only 3 (7%) reported the use of appropriate statistical methods for bias correction of point estimates of the intervention effect, and associated CIs and P-values” Stevely et al. (2015) ICTMC/SCT Conference May 2017

Consequences for economic evaluation How might the above practical and statistical issues with implementing adaptive designs impact on health economic evaluation? In short, it depends on what economic evaluation is being conducted... ... and when ICTMC/SCT Conference May 2017

Types of economic evaluation DEFINITION: “Comparative analysis of alternative courses of action in terms of both their costs and consequences” Drummond et al. (2015) Incorporates a wide range of approaches within the broad categories of Cost-effectiveness analysis Value of information analysis ICTMC/SCT Conference May 2017

Types of economic evaluation Cost information comes from a range of sources e.g. the Personal Social Services Research Unit document of unit costs for a range of health and social care services in England and the British National Formularly for drug costs Consequences are measured using a health related utility such as quality adjusted life years(QALY) Sometimes summary measures are used to compare costs/consequences. Alternatively, decision analytic models may be developed ICTMC/SCT Conference May 2017

Economic evaluation at the end of an adaptive trial Economic evaluations are often ‘added on’ to existing trials. They are not separately powered. If a trial stops early, will have even less information with which to carry out the evaluation If use is made of the treatment effect relating to the primary endpoint in a decision analytic model, then bias in the estimate may influence the economic model If calculate a secondary economic endpoint e.g. QALY, it is likely to be biased as correlated with the primary endpoint ICTMC/SCT Conference May 2017

Economic evaluation alongside clinical evaluation in an adaptive trial Could consider economic evaluation informally during an adaptive design (e.g. In the context of a flexible adaptive design). Practicalities of conducting interims must then be considered such as timing and data required Could consider formally monitoring measures of clinical and cost effectiveness at each interim analysis. This leads to bivariate monitoring. Previous literature in the context of efficacy and safety monitoring E.g. Jennison & Turnbull (1993); Cook & Farewell (1994) and in the case of two endpoints more generally E.g. Todd (1999; 2003) ICTMC/SCT Conference May 2017

Conclusions Adaptive design methodology and health economic analyses have evolved largely independently, yet if more adaptive designs are going to be used and health economic analyses are going to become even more embedded in trials then the interface between the two must be explored Research to this effect has been initiated Practical implementation is also developing ICTMC/SCT Conference May 2017