1. Safety Monitoring During Clinical Development
Greg Ball and Bill Wang Merck & Co., Inc., Kenilworth, NJ, USA

2. Disclaimer
The authors are employed by Merck & Co., Inc. (known as MSD outside of the US and Canada)
The opinions expressed are their own and do not necessarily reflect those of their employer

3. Agenda
High level summary of safety monitoring during clinical development (from ASA Biopharm Safety Monitoring working group)
Overview of new guidance on FDA IND safety reporting final rule
Specific new method for blinded reviews of aggregate safety data that is being implemented in clinical trial programs

4. Overview ASA Safety Monitoring Working Group
Established in 2015 by the ASA Biopharm section Safety Statistics WG
Goal
To empower the biostatistics community to play a more proactive role and better enable quantification in safety monitoring
Key activities
Review safety regulations, survey industry, and interview thought leaders
Review statistical methodologies
2016 deliverables
August: JSM Biopharm Section, DIA China Quantitative Science Forum
December: Deming Conference (tutorial)

5. ASA Safety Monitoring Working Group
WS1: Industry Practice Regulation
Faiz Ahmad (Galderma)
Greg Ball (Co-lead, Merck)
Michael Colopy (UCB)
Susan Duke (Co-lead, AbbVie)
Robert (Mac) Gordon (Janssen)
Qi Jiang (Amgen)
Wenquan Wang (Morphotek)
William Wang (Chair, Merck)
WS2: Methodology
Michael Fries (Behring)
Karolyn Kracht (AbbVie)
Judy Li (Co-lead, FDA)
Melvin Munsaka (Co-lead, Takeda)
Matilde Sanchez (Arena)
Krishan Singh (GSK)
Ed Whalen (Pfizer)
William Wang (Chair, Merck)
Kefei Zhou (Amgen)
Safety Monitoring Statistical Advisors
Aloka Chakravarty (FDA)
Brenda Crowe (Lilly)
Larry Gould (Merck)
Qi Jiang (Amgen)
Olga Marchenko (Quintiles)
Amy Xia (Amgen)
Janet Wittes (Statistics Collaborative)
We are indebted to the 18 thought leaders who each spent at least an hour with us discussing their views on quantitative assessment of safety monitoring
Interviewed by Greg Ball, Susan Duke, Mac Gordon, and Bill Wang

6. Thought Leaders Aloka Chakravarty (FDA) Bob Temple (FDA)
Brenda Crowe (Lilly)
Christy Chuang-Stein (Pfizer)
Conny Berlin (Novartis)
Dave DeMets (UW)
Frank Rockhold (GSK, now Duke)
Frank Shen (AbbVie)
Janet Wittes (Statistics Collaborative)
Jose Vega (Merck)
Juergen Kuebler (CSL Behring)
Lily Krasulja (Janssen)
Mark Levenson (FDA)
Mondira Bhattacharya (AbbVie)
Olga Marchenko (Quintiles)
Steve Snapinn (Amgen)
Valerie Simmons (Eli Lilly)
Walter Offen (AbbVie)

7. Four Pillars of Safety Statistics
Cross-disciplinary scientific engagement
Intelligent data architecture
Visual and analytic methods/tools
Effective, efficient operational process
Regulatory landscape:
Protecting the public good
Making good business decisions
Shift from efficacy to benefit-risk

8. Thought Leader Interviews: Cross-Disciplinary Scientific Engagement
“Safety is the new efficacy” - a public health issue
No longer just PV and spontaneous reports
Requires experienced statisticians to interact with other departments
Safety physicians need to rely heavily on quantitative expertise for aggregate data analysis and interpretation
Siloed discussions of safety and benefit are not in the patients’ best interest
Statisticians need a safety mindset and need to closely engage other disciplines (eg, safety physicians) to increase our impact
Statisticians needs to understand about “why” before jumping into “how”
Reference: FDA Draft guidance on Safety Assessment Committees (SAC). Dec 2015.

9. Agenda
High level summary of safety monitoring during clinical development (from ASA Biopharm Safety Monitoring working group)
Overview of new guidance on FDA IND safety reporting final rule
Specific new method for blinded reviews of aggregate safety data that is being implemented in clinical trial programs

10. Regulatory Motivation: CIOMS Working Group on Safety
Since 1986, CIOMS working groups on drug safety have been recognized as “think tanks” for advancing international pharmacovigilance practices
The initiatives over the years have resulted in several major published reports
Many of these CIOMS recommendations have become part of regulatory guidance by ICH, EMA, FDA, etc
CIOMS=Council for International Organization of Medical Sciences; ICH=International Conference on Harmonization; EMA=European Medicines Association; FDA: Food and Drug Administration.

11. Regulatory Motivation: CIOMS VI: Close Linkage With Clinical Trial Safety
Introduces proposals for enhancing the collection, analysis, evaluation, reporting, and overall management of safety information from all safety data sources (special focus on clinical trials)
A shift from the management of postmarketing safety information (spontaneous reports) to the management of clinical trial information

12. Regulations Vary Across Regional Jurisdictions
Japan, PMDA: 3 pillar system
Safety
Risk mitigation
Relief
Health damage
Review
Risk reduction
Europe, EMA:
EudraVigilance GVP Module IX for postmarketing signal detection
USA, FDA:
IND safety reporting final rule
Safety Assessment committee
Safety Surveillance Plan
Planned unblinding of safety data
China, CFDA:
Minimal sample size requirement (Provision for Drug Registration 2007); guidance on postmarketing commitment studies (2013 draft)
Provisions for nationalized monitoring of ADRs (2011); postmarketing intensive safety monitoring guidance (2013 draft)

13. FDA IND Safety Reporting Final Rule
September 2010: Final rule
Investigational New Drug Safety Reporting Requirements for Human Drug and Biological Products and Safety Reporting Requirements for Bioavailability and Bioequivalence Studies in Humans (21 CFR Parts 312 and 320; Federal Register, Vol 75, No 188)
December 2012: Final guidance
Safety Reporting Requirements for INDs and BA/BE Studies
December 2015: Draft guidance
Safety Assessment for IND Safety Reporting Guidance for Industry

14. Safety Reporting Requirements for INDs: Guidance for Industry (December 2012)
To improve the overall quality of safety reporting and to comply with requirements for IND safety reports based on data in the aggregate, “the sponsor should have in place a systematic approach for evaluating the accumulating safety data”
“Reasonable possibility” for IND safety reporting:
“A single occurrence of an event that is uncommon and known to be strongly associated with drug exposure”
“One or more occurrences of an event that is not commonly associated with drug exposure, but is otherwise uncommon in the population exposed to the drug”
“An aggregate analysis of specific events observed in a clinical trial that indicates those events occur more frequently in the drug treatment group than in a concurrent or historical control group”

15. Safety Assessment for IND Safety Reporting: Draft Guidance for Industry (December 2015)
FDA recommends that sponsors develop a Safety Assessment Committee and a Safety Surveillance Plan as “key elements of a systematic approach to safety surveillance”
Sponsors should periodically review accumulating safety data
Integrate across multiple studies (completed and ongoing)
Provide a quantitative framework for measuring the evidence of an association (for unexpected events) or a clinically important increase (for expected events)
Make a judgment about “reasonable possibility” for IND safety reporting
“It is critical for sponsors to detect and report, as early as possible, serious and unexpected suspected adverse reactions and clinically important increased rates of previously recognized serious adverse reactions”
Opportunity for sponsors to partner with FDA to focus on important safety issues

16. Aggregate Analyses for Comparison of Adverse Event Rates Across Treatment Groups
Preferred approach
SAC should regularly perform unblinded comparisons across treatment groups to detect numerical imbalances
Anticipated SAEs prespecified in the SSP (anticipated events)
Previously recognized SARs listed in the IB (expected events)
Appropriate steps should be taken to maintain overall study blinding
Alternative approach
Only perform unblinded comparison of event rates across treatment groups if the overall rate for all treatment groups of a specific SAE is substantially higher than a predicted rate
Considerable uncertainty of predicted rate in patient population
Substantial challenges for specifying a predicted rate for all events
Sponsors should prespecify, in the Safety Surveillance Plan
Predicted rates of anticipated events and expected events
Guidelines for determining when an observed rate had exceeded the predicted rate

17. Agenda
High level summary of safety monitoring during clinical development (from ASA Biopharm Safety Monitoring working group)
Overview of new guidance on FDA IND safety reporting final rule
Specific new method for blinded reviews of aggregate safety data that is being implemented in clinical trial programs

18. Key Questions to Answer
Have more events occurred than were expected (yes or no)?
What is the magnitude of this relative risk elevation (how much)?
Quantitative framework is intended to stimulate SMT discussions and improve conversations about safety monitoring of accumulating blinded data; these are not decision rules

19. Collaborative Process and Quantitative Framework: Pilot Study
Evaluation, Not Implementation
Based on SMT discussion, two AESI were selected for evaluation
Rash: Common, but not serious
Syncope: Serious, but not common
Extensive literature search for background rates was conducted

20. Collaborative Process: Characterize Background Rates
Study Incidence: Not Annualized
(ADNI is 2 years, and other studies are 1.5 years)
Study
Description N
Year
Age
Female
MMSE
Range
Syncope
Semagacestat
76-week, phase 3 study (stopped early)
501
2008
73.2 (8.2) 53
20.8 (3.5)
16-26
1.4†
ADNI
2-year natural history, nontreatment study
190
2004
75.2 (7.5)
47.9
23.3 (2.0)
20-26
4.2‡
Bapineuzumab
18-month, published trial
110
2005
67.9 (9.4)
59.8
20.7 (3.1)
1.8
78-week, phase 3 study
524
2007
71.9 (10.1)
50.3
21.2 (3.2)
2.5
Solanezumab
Two 18-month, phase 3 studies
1025
2009 
73.4 (7.9)
55.9
21 (3)
2.1
MMSE=Mini Mental State Examination is used to test for complaints of problems with memory or other mental abilities, with higher scores indicating better cognitive function. †Stopped early; ‡2-year study of different patient population.
Henley DB, Sundell KL, Sethuraman G, Dowsett SA, May PC. Safety profile of semagacestat, a gamma-secretase inhibitor: IDENTITY trial findings. Curr Med Res Opin. 2014;30(10):
Henley DB, Sundell KL, Sethuraman G, Siemers ER. Safety profile of Alzheimer’s disease populations in Alzheimer’s Disease Neuroimaging Initiative and other 18-month studies. Alzheimers Dement. 2012;8(5):
Salloway S, Sperling R, Fox NC, et al. Two phase 3 trials of bapineuzumab in mild-to-moderate Alzheimer’s disease. N Engl J Med. 2014;370(4):
Doody RS, Thomas RG, Farlow M, et al. Phase 3 trials of solanezumab for mild-to-moderate Alzheimer’s disease. N Engl J Med. 2014;370(4):

21. Collaborative Process: Characterize Background Rates (continued)
Estimates for Background Rates Incorporating Clinical and Epidemiological Considerations
Estimate (%)
Syncope
Lower Background Rate
1.5
Upper Background Rate
3.0
Confidence Level 95
Expected Background Rate
2.0

22. Collaborative Process: Characterize Treatment Rates
Estimates for Treatment Rates Incorporating Clinical and Epidemiological Considerations
Estimate (%)
Syncope
Lower Treatment Rate
1.5
Upper Treatment Rate
3.0
Confidence Level 85
Critical Rate

23. Quantitative Framework: Bayesian Posterior Probabilities of Risk Elevation for AESI
Safety Monitoring Requires Flexibility
Bayesian approach
Accommodates uncertainty
Natural for learning and decision-making
Leverage prior information from earlier trials and related treatments
Unified framework for continuous safety monitoring using all of the available data
Probability statements that are easy to interpret
Operating characteristics can be used to tune the probability threshold boundaries

24. Quantitative Framework: Probability Threshold Boundaries
Have more events occurred than were expected (yes or no)?

25. Quantitative Framework: Probability Threshold Boundaries
Probability (Pooled Rate > Critical Rate / Data)
≥ Probability Threshold
Parameters
Critical rate
Probability threshold
Data
Overall number of events = x
Overall number of patients = n
Pooled rate = x/n

26. Quantitative Framework: Probability Threshold Boundaries (continued)
Operating Characteristics of Probability Threshold Boundaries for Syncope With a Critical Treatment Rate of 3.0%
True Control Rate
True Treatment Rate
True Pooled Rate
Probability Threshold Boundary
(Percent of Trials Crossing the Boundary)
70%
80%
90%
2.0%
2.00%
9.6%
4.0%
0.9%
3.0%
2.67%
63.7%
47.9%
26.8%
3.33%
98.9%
96.6%
91.7%
5.0%
4.00%
100.0%
2.33%
30.1%
16.7%
5.8%
3.00%
90.6%
82.2%
64.3%
3.67%
99.8%
99.0%
4.33%

27. Quantitative Framework: Probability Threshold Boundaries (Mock Data)
Probability Threshold Boundaries for Syncope: Pooled Rate 2.67%
(Critical treatment rate 3.0% and background rate 2.0%)
Ball G, Piller LB, Silverman MH. Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Contemp Clin Trials. 2011;32(Suppl 1):S2-S10.

28. Quantitative Framework: Estimates of Relative Risk Elevation
What is the magnitude of this relative risk elevation (how much)?

29. Quantitative Framework: Estimates of Relative Risk Elevation (Mock Data)
Assessment of Relative Risk Elevation for Syncope (n = N)
Gould AL, Wang W. Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine. Stat Med (In press.)

30. Key Questions to Answer
Have more events occurred than were expected (yes or no)?
Probability threshold boundaries
What is the magnitude of this relative risk elevation (how much)?
Estimates of relative risk elevation

31. Summary of Aggregate Safety Monitoring With Ongoing Blinded Studies
An Alternative Approach for Expected Events
Collaborative process facilitates engagement with clinical safety, clinical development, epidemiology, and statistics
Characterize background event rates
Tune probability threshold boundaries
Quantitative framework helps guide medical review and safety monitoring of the accumulating blinded data
General summary of aggregate safety profile
Bayesian posterior probabilities of risk elevation
SMT uses medical judgment to decide on next actions
Have more events occurred than were expected?
What is the magnitude of this relative risk elevation?

32. References
Crowe BJ, Xia HA, Berlin JA, et al. Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clin Trials. 2009;6(5):
US Department of Health and Human Services, Food and Drug Administration. Investigational new drug safety reporting requirements for human drug and biological products and safety reporting requirements for bioavailability and bioequivalence studies in humans. CFR
Ball G, Piller LB, Silverman MH. Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Contemp Clin Trials. 2011;32(Suppl 1):S2- S10.
US Department of Health and Human Services, Food and Drug Administration. Safety reporting requirements for INDs and BA/BE studies. Guidance for industry and investigators
Wittes J, Crowe B, Chuang-Stein C, et al. The FDA's final rule on expedited safety reporting: statistical considerations. Stat Biopharm Res. 2015;7(3):
US Department of Health and Human Services, Food and Drug Administration. Safety assessment for IND safety reporting. Guidance for industry (draft)
Schnell PM, Ball G. A Bayesian exposure-time method for clinical trial safety monitoring with blinded data. Ther Innov Regul Sci (In press.)
Gould AL, Wang W. Monitoring potential adverse event rate differences using data from blinded trials: the canary in the coal mine. Stat Med (In press.)