A Benefit-Risk Model to Facilitate DMC-Sponsor Communication and Decision Making Andreas Sashegyi PhD ELI LILLY AND COMPANY 1

Picture the Following Scenario… DMC of a landmark Phase 3 clinical trial meets to review accumulating data of the study, just past the half-way point – Committee agrees that emerging benefit looks real and promising – Risk profile of the compound is more severe than expected – DMC reviews ICD and DMC Charter to determine what degree of risk tolerance was specified and concludes that it is not unethical to continue trial; no additional guidance specified – DMC believes the study could continue, but does not know whether sponsor would agree The committee’s deliberations are stalled in absence of knowing how sponsor would assess benefit-risk balance Is there a way of communicating emerging benefit-risk balance to the sponsor without unblinding study results? 2

The Dilemma Independent DMCs are intentionally engaged for pivotal trials to monitor patient safety, allowing sponsor to remain blinded to data DMC makes recommendations based on emerging data, but… Sponsor called to make decisions without recourse to these data Is such decision making always feasible? 3

Requirements for a Resolution At times a sponsor should be informed of certain key features of the emerging trial data (beyond a terse DMC recommendation) in order to make a good decision concerning the continuing conduct of the study The information conveyed may not reveal any specifics about trial endpoints or other study measurements, so as to protect study blinding The information conveyed should allow an assessment of whether emerging risks for patients are tolerable in view of accruing benefits 4

A Benefit-Risk Assessment Model (BRAM) Consider application of a multi-attribute model for evaluating benefit-risk profiles (Felli, Noel & Cavazzoni, Medical Decision Making, 2009) BRAM graphically presents contextual beliefs about benefits and risks in a framework conducive to focused discussion Requires definition of an analytic frame – Disease State = Disease being studied by the trial in question – Population of Interest = Study participants – Perspective for Assessments and Tradeoffs = DMC, focused on patient safety 5

Model Structure V RiskBenefit Safety Safety Measure 1 Safety Measure n S Tolerability Tolerability Measure 1 Tolerability Measure n T Improper Use : : Efficacy Efficacy Measure 1 Efficacy Measure n E Life Effects Life Effect 1 Life Effect n L Convenience : : Treatment Administration Treatment Burden Monitoring Acquisition Storage Assessed with other efficacy measures… Along with other Benefit attributes… In the context of key safety and other Risk measures Primary endpoint 6

Evaluation Weights are chosen – to reflect relative importance of each benefit and risk attribute (sum to 1 within each dimension) – to reflect relative importance of each measure within an attribute (sum to 1 within each attribute) Attributes are scored, based on accumulating interim data – Normalized to lie between 0 and 1 – Convention: more advantageous benefit and more detrimental risk ratings correspond to higher values (i.e. would like to see benefit scores as close to 1 and risk scores as close to 0 as possible) 7

Evaluation Aggregate scores are computed – for the 2 risk and 3 benefit attributes (weighted averages of the measures within the various attributes) – for the overall benefit and risk dimensions of the model (weighted averages of the various attributes) Result is a “generic” (x, y) pair depicting a position on a benefit (y) – risk (x) plane P = [0,1]x[0,1] Of interest is the position of the evaluation of the experimental therapy relative to control 8

Model Structure V RiskBenefit Safety Score 1 Score n S Tolerability Score 1 Score n T Improper Use : : Efficacy Score 1 Life Effects Score 1 Score n L Convenience : : Score 2 Score 3 Score 4 Score 5 Score n E w1w1 w2w2 w3w3 w1w1 w2w2 w1w1 w nE w1w1 w nL Score 1 w1w1 w2w2 w3w3 w4w4 w5w5 w1w1 w nS w1w1 w nT Sum to 1.0 Є [0, 1] 9

E.g. Diabetes Compound Weight Loss Benefit Efficacy HbA1c Fasting Glucose Insulin Sensitivity Beta Cell Function Lipid Profile Life Effects Convenience Benefit Score = V Safety Tolerability Improper Use Risk Risk Score =

BRAM Output Risk Benefit Is the increase in risk implied by active therapy over control justified in view of the anticipated benefit? Active Control 11

In Practice… Each DMC member – assigns own scores (ideally a range based on uncertainty given data collected!) – assigns own weights? – evaluates active therapy and control Resampling from distribution of weights and scores observed across DMC members allows a reflection of the uncertainty in the benefit/risk assessment – generates an BR “cloud” 12

Active treatment Placebo Stop study due to futility/ safety concerns Continue study Is the benefit/risk trade-off acceptable? 13

Active treatment Placebo Benefit/risk trade-off acceptable? DMC members may not all agree May put DMC Chair in a difficult position Could involve sponsor in making decision Need not unblind sponsor

Benefit Risk Benefit Risk Benefit Risk Benefit Risk Archetypes Control Active Promising Treatment Fully Aligned DMC Promising Treatment Varying positions on B/R assessment among DMC members Difficult decision! Fully Aligned DMC Involve sponsor? 15 Benefit Risk More Difficult decision! DMC not aligned Involve sponsor?

Some Thoughts… Could this kind of tool be useful in helping a DMC form a recommendation (say, in the face of disagreement) even if the BRAM output is not shared with the sponsor? In cases where the assessment is shared with the sponsor, should the links between the individual DMC members’ assessment of control and active therapy be revealed? – Shows extent to which DMC members consistently rate high or low, regardless of therapy group – Would highlight variation among DMC members’ assessments of the relative difference between active therapy and control 16

Reducing this to Practice… At the DMC Organizational Meeting – Sponsor discusses BRAM structure with DMC – the attributes, including scoring protocols which objectively transform observed data to the unit interval – Sponsor shares weights of the study team – Sponsor discusses scenarios highlighting changes in benefit/risk scores with which the team would be comfortable or uncomfortable – Agreement reached on how to incorporate new findings (e.g. unexpected side effect) BRAM structure and intended use captured in DMC Charter BRAM structure should not be altered by DMC during the conduct of the study – With exception of incorporating new findings – agreed upon process to be followed 17

Reducing this to Practice… DMC statistician brings BRAM evaluation tool to meetings – responsible for collecting scores and producing model output in real time Ultimately, we need DMC and sponsor to be comfortable with this approach Input from regulators Pilot the model… 18