DATA MONITORING COMMITTEES: COMMENTS ON PRESENTATIONS Susan S. Ellenberg, Ph.D. Department of Biostatistics and Epidemiology University of Pennsylvania School of Medicine FDA/Industry Workshop Washington, DC September 28, 2006

CHANGING ROLE OF DMC Increasing structure –Policies of funders (NIH, industry) and regulatory agencies More transparency of process –Study protocol –Statistical analysis plan –DMC Charter –Publications describing standard and/or recommended practices

3 CATEGORIES OF DMC ACTIVITIES AND PROCESSES Governed by well-established and widely accepted principles –Maintaining confidentiality of interim data –Minimizing conflicts of interest Subject to controversy regarding optimal approaches –Review of coded vs unblinded interim data –Presentations by “independent statistician” Not yet discussed to any great extent— anything goes (?)

RALPH D’AGOSTINO: ISSUES FACED BY DMCs Recommending or approving changes in endpoints Mortality as a safety or efficacy endpoint Timing of reporting unexpected results Balancing safety and efficacy DMC authorship on study paper

RALPH D’AGOSTINO ISSUES FACED BY DMCs Recommending or approving changes in endpoints (2) Mortality as a safety or efficacy endpoint (1) Timing of reporting unexpected results (3) Balancing safety and efficacy (3) DMC authorship on study paper (3)

CHANGING ROLE OF STATISTICIANS ON DMCs? Maybe no longer the only DMC member with prior DMC experience Involvement of “independent statisticians” may put increased pressure on experienced DMC statistician Increased use of DMCs may make DMC decisions of more variable quality, as those who serve may be less experienced

BRAM ZUCKERMAN: DMCs FOR DEVICE TRIALS Traditional mantra of device industry: DEVICES ARE NOT DRUGS! Still, generally recognized that trials of devices intended to save lives or impact on health in a major way require the same type of interim monitoring as drug trials FDA guidance on DMCs developed with full input and support of CDRH

WHAT DEVICES REQUIRE STUDY IN TRIAL WITH DMC? Novel use requiring invasive procedures –Implantable devices –Stents Use has implications for diagnosis or treatment of serious disease and requires evaluation in controlled trial –Assays –Imaging

SPECIFIC ISSUES DMC involved in modifying sample size –Inadvisable if DMC has already seen unblinded interim comparison –Could affect trial conduct, interpretation of final comparisons DMC viewing unblinded data in closed session –Should be routine, rather than “not uncommon” Disputes between DMC and FDA –If happening commonly, might be helpful to convene a workshop specifically on data monitoring in device trials

PAUL GALLO: ADAPTIVE DESIGNS AND DMCs Much has been written about the statistical properties of adaptive designs involving sample size re- estimation Little has been written about how these designs would actually be implemented –Prespecified plans, with automatic sample size changes effected if boundaries are crossed? –Up to judgment of DMC or statistician doing interim analysis for DMC?

IMPLICATIONS FOR CONFIDENTIALITY Clinical trial initiated Protocol known to sponsors, investigators, FDA, others Interim data known only by DMC and statistician analyzing interim data Partway through the trial, sample size is increased What information does this provide to investigators, sponsors, external sources (eg, Wall Street)?

UNBLINDING Sample size increases will have to be made known to sponsor, others involved in trial If procedures are pre-specified, sample size increases can be used to back- calculate interim estimate of treatment effect If procedures are not pre-specified, sponsor is essentially committing to open sequential design; seems impractical

IS THIS REALLY A PROBLEM? Seattle Times, August 2005 –Investment firms interviewing physicians involved in clinical trials as investigators or DMC members –Firms have been able to glean sufficient knowledge to make highly accurate guesses about trial outcomes –Major effects on stock prices –All without this extra information

PROBLEM RECOGNIZED Schäfer and Müller: “[A]… problem is that knowledgeable observers can draw conclusions about the current observed treatment differences from the action determined.” Chen, DeMets, Lan: “Further research is still needed to investigate the impact…on the conduct of the trial and to find ways to protect the integrity of the study.”

HOW MUCH INFORMATION EMERGES FROM OTHER DESIGNS? Group sequential design –If DMC stops study, full information but study is over; doesn’t matter –If DMC continues study, know results are between stopping bounds—pretty minimal information –If futility analyses also done, narrower bound on possible outcome as study approaches planned size Triangular design may provide more information Depends on frequency of interim looks

SEAMLESS DESIGNS Designs permitting “seamless” transition from phase 2 to phase 3 have potentially great efficiency –Avoids stopping and restarting enrollment –Avoids one IRB approval process Major issue is whether one can anticipate to a sufficient extent everything that might happen in phase 2 that could affect approach to phase 3 –Sponsor will not have opportunity to review data and incorporate new information in design of next phase –May place unreasonably heavy reliance on DMC judgment

WHAT IS THE DIFFERENCE …between a study designed for 500 subjects that could get bumped up to 1000, and a study designed for 1000 subjects that could stop at 500?

WHAT IS THE DIFFERENCE …between a study designed for 500 subjects that could get bumped up to 1000, and a study designed for 1000 subjects that could stop at 500? –Big difference –Enlarging the sample size gives out information that could affect the further conduct and interpretation of the study –Stopping early gives out information, but study is over