Presentation on theme: "Controversy? What controversy? An attempt to structure the debate on adaptive designs Marc Vandemeulebroecke MCP, Vienna, 11.7.2007."— Presentation transcript:
Controversy? What controversy? An attempt to structure the debate on adaptive designs Marc Vandemeulebroecke MCP, Vienna, 11.7.2007
Vandemeulebroecke Controversy? What controversy? 1
Vandemeulebroecke Controversy? What controversy? 2 A. Wald (1945): „The National Defense Research Committee considered these developments sufficiently useful for the war effort to make it desirable to keep the results out of the reach of the enemy.“ Examples (1): Sequential analysis F. J. Anscombe (1963): „‘Sequential analysis‘ is a hoax.“ P. Armitage (1989): „If [...] ‘sequential analysis is a hoax‘, then its proponents are amongst the most persuasive swindlers, and their clients amongst the most gullible consumers, in the history of scientific endeavour.“
Vandemeulebroecke Controversy? What controversy? 3 Example (2): Group sequential designs such that for all Repeated confidence intervals (C. Jennison, B. Turnbull 1989) P. Armitage: „ingenious derivation“ S. J. Pocock: „a minor change of presentation style“
Example (3): Adaptive/flexible designs Vandemeulebroecke Controversy? What controversy? 4 C.-F. Burman, C. Sonesson (2006): Are flexible designs sound? M. Proschan: „Such methods, if extremely abused, produce illogical conclusions, but that is no more a condemnation of adaptive methods than Jack the Ripper is a condemnation of cutlery.“ P. Bauer: „Is wine sound? It is the way to drink it that matters.“ „The weighted test violates inference principles [...] and may lead to unreasonable results.“ Test H 0 : with ; interim: Change to observe
Example (4): Terminology Vandemeulebroecke Controversy? What controversy? 5 A. J. Phillips, O. N. Keene (2006): „The terms ‘flexible design’ and ‘adaptive design’ are used interchangeably.” C. Jennison, B. Turnbull (2006): „The adjectives ‘flexible’ and ‘adaptive’ are not synonymous. Indeed adaptive designs can range from being totally inflexible to very flexible.”
Vandemeulebroecke Controversy? What controversy? 6 Issues Imagine a study protocol: Double-blind multicenter trial in Crohn‘s Disease Active vs placebo 1:1 Endpoint: CDAI after 4 weeks IA at half of the sample size, switch to 2:1 if encouraging Trial conduct: IA disappointing, but new budget available Continue 1:1, ad hoc increase of sample size Final analysis correctly adjusted and significant In stage 2, patients are more severe, and placebo is doing much worse
Issue 1: Feasibility Vandemeulebroecke Controversy? What controversy? 7 Practical, logistical or operational issues regarding the conduct of the study Speed of recruitment vs. speed of response Availability of specialized software „Supply chain management“: Data capture/cleaning/ processing; real-time randomization and drug supply Training/SOPs/“charter“ Investigate behavior of procedure by simulation, play through scenarios with decision makers / IDMC
Vandemeulebroecke Controversy? What controversy? 8 Statistical/mathematical property. Correct type I error Point estimates available and sensible (e.g., mean/median unbiased) Confidence intervals available, sensible and correct Issue 2: Validity
Vandemeulebroecke Controversy? What controversy? 9 Hard to quantify. Concern: „operational bias“ Consistency/homogeneity across trial stages (A. Koch: identification problem) MCP: T. Friede, A. Faldum... Jeopardizing integrity means jeopardizing credibility. Measures to maintain trial integrity: avoid unblinding if possible control dissemination of results control scope and amount of adaptations Issue 3: Integrity
Vandemeulebroecke Controversy? What controversy? 10 V. Dragalin (2006, PhRMA): „To maintain study validity means providing correct statistical inference [...], ensuring consistency between different stages of the study, and minimizing operational bias. To maintain study integrity means providing convincing results to a broader scientific community; preplanning, as much as possible, based on intended adaptations; and maintaining the blind of interim analysis results.“ Issues 2/3: Validity/Integrity H. Schäfer (2006): internal validity - external validity
Vandemeulebroecke Controversy? What controversy? 11 Only interpretable results are communicable. Intregrity and interpretability have the main impact of the persuasiveness of the study results. Endpoints meaningful Point estimates and confidence intervals available and sensible Jeopardizing integrity (e.g., across-stage consistency or amount of adaptations) will also reduce interpretability. Issue 4: Interpretability
Vandemeulebroecke Controversy? What controversy? 12 Issue 5: Efficiency Often, adaptation comes at the cost of efficiency....in terms of power / expected information Reason: non-sufficient statistics Any adaptive design can be outperformed by a nonadaptive design - and vice versa! „Statistical efficiency is not always the priority“ (Koch 2006) „Response-dependent choice of group sizes can itself be a source of improved efficiency“ (Schmitz 1993, Jennison/Turnbull 2006)
Vandemeulebroecke Controversy? What controversy? 13 Two main questions: Should there be an a priori selection of study features that may be adapted? If so, now many and which? Should the design changes follow prespecified rules or may they be done ad hoc? Should they depend on information internal or external to the trial? Issue 6: Flexibility stopping + stage sizes + other Adaptable features Prespeficied rule Not dep. on trial-internal data Completely free Nature of change Group sequential Spending functions p-value combinations „Schmitz designs“
Vandemeulebroecke Controversy? What controversy? 14 Issues Feasibility:„Only“ operational issues Validity: „Only“ mathematical issues Integrity:Requirements? „Operational bias“? Homogeneity across stages? Interpret‘y: Requirements? Efficiency:Weigh against flexibility (sponsor‘s choice) Flexibility: Summary/conclusions A priori selection of adaptable features? How many/which? Adaptations ad hoc or following prespecified rules? Depending on internal or external information ?
Biom J 48 (2006, special issue), in particular: O‘Neil, Hung et al., Koch, and discussants: Section on „Regulatory perspectives“, pp. 559-622 Burmann, Sonesson (2006): Are flexible designs sound? Biometrics 62, pp. 664-683 (with discussion) EMEA (2006): Reflection paper on methodological issues in confirmatory clinical trials with flexible design and analysis plan. CHMP/EWP/2459/02 Fleming, DeMets (1993): Monitoring of clinical trials: Issues and Recommendations. Contr Clin Trials 14, pp. 183-197 Gallo et al. (PhRMA, 2006): Adaptive designs in clinical drug development - an executive summary of the PhRMA working group. J Biopharm Stat 16, pp. 275-283 (with discussion) Jennison, Turnbull (2006): Adaptive and nonadaptive group sequential tests. Biometrika 93, pp. 1-21 Phillips, Keene (PSI, 2006): Adaptive designs for pivotal trials: discussion points from the PSI adaptive design expert group. Pharm Stat 5, pp. 61-66 PhRMA (2006): Full white paper. DIJ 40, pp. 421-484 Posch, Bauer, Brannath (2003): Issues in designing flexible trials. Stat in Med 22, pp. 953-969 Stat in Med 25 (2006, special issue), pp. 3229-3408 Some references Vandemeulebroecke Controversy? What controversy? 16...and many thanks to Norbert Benda!
Appendix: Terminology Vandemeulebroecke Controversy? What controversy? A1 PhRMA: Definition of “adaptive design” P. Gallo et al (2006): “a clinical study design that uses accumu- lating data to decide how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial. [...] In such trials, changes are made ‘by design’, and not on an ad hoc basis.” V. Dragalin (2006): “a multistage study design that uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. [...] preplanning, as much as possible, based on intended adaptations.”
Appendix: Terminology Vandemeulebroecke Controversy? What controversy? A2 Other definitions of “adaptive design” EMEA (2006): “A study is called ‘adaptive’ if statistical methodology allows the modification of a design element [...] at an interim analysis with full control of the type I error.” Novartis slide deck: “Adaptive designs allow for initial uncertainties in trial design to be confirmed/adapted during the trial. The integrity of the trial is maintained and the evidence for the same hypothesis before and after the adaptation is combined.”
Appendix: Terminology Vandemeulebroecke Controversy? What controversy? A3 Other terms flexible adaptive combination flexible group sequential flexible adaptive adaptive group sequential internal pilot self designing seamless adaptive design automates.... (List stolen from J. Röhmel‘s talk at BfArM Discussion on Adaptive Designs, 2006)
Vandemeulebroecke Controversy? What controversy? A4 Jennison, Turnbull (2006): „The strength of adaptive redesign lies in coping with the unexpected, in particular responding to external information that could not have been anticipated at the start of a study.“ Appendix: Flexibility P. Gallo et al. (2006): „Should or should it not be viewed that adaptive designs might remedy inadequate planning?“ M. Frisén (2006): „One should use planned adaptive designs when one expects that it will be necessary to adapt the design. If one unexpectedly has strong reasons to change the plans, one should be very careful.“
Your consent to our cookies if you continue to use this website.