1. DIA Virtual Journal Club Statistical Considerations on the Evaluation of Imbalances of Adverse Events in Randomized Clinical Trials Discussant Susan Duke, Safety Statistics AbbVie
12 December 2016

2. Points I will cover from the paper
Risk difference, confidence intervals, graphics
Time to first event
Volcano plot
Dropouts and their predictive capacity post approval
Baseline risk factors and confounding
Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc

3. Setup A bit of history Safety analysis – what matters to patients?
Safety Statistics is emerging as a new sub-discipline of statistics

4. Some interesting history
1962 Thalidomide, a new sleeping pill, caused birth defects in thousands of babies born in western Europe. Dr. Frances Kelsey, FDA medical officer, keeps the drug off the U.S. market, arousing public support for stronger drug regulation. Kefauver-Harris Drug Amendments passed to ensure drug efficacy and greater drug safety. Drug manufacturers are required to prove to FDA the effectiveness of their products before marketing.

5. Some interesting history, continued
2005 Formation of the Drug Safety Board, consisting of FDA, NIH and VA staff. The Board will advise CDER on drug safety issues and work with the agency in communicating safety information to health professionals and patients.
2007 report, Institute of Medicine

6. The beginnings of Safety Statistics
George Rochester, FDA statistician
Served as FDA’s Expert Lead Statistician for Quantitative Safety in the Office of Biostatistics, developed the Quantitative Safety Program, renamed the Division of Biometrics VII.
SPERT and Program Safety Analysis Plan
Amy Xia, Amgen
Brenda Crowe, Lilly

7. SPERT Recommendations
A proactive approach:
Create a Program Safety Analysis Plan early in development
Planning of repeated, cumulative meta-analyses of the safety data
3-tier system for signal detection and analysis of adverse events
Pre-specified hypothesis testing – AESIs
Signal detection of common events
Descriptive analysis (rare AESIs & others)
Sponsors review aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission

8. Regulatory Motivation: CIOMS Working Groups on Safety
CIOMS WG
Descriptions
Resulting Regulatory Guidance VI
Management of Safety Information from Clinical Trials (2005)
IND Safety Reporting X
Considerations for applying good meta-analysis practices to clinical safety data within the biopharmaceutical regulatory process (2016)

9. Setup A bit of history Safety analysis – what matters to patients?
Safety Statistics is emerging as a new sub-discipline of statistics

10. What matters to patients?
Both efficacy and safety matter
It may be argued that safety matters more
Do no harm

11. How is analysis of safety different from that of efficacy?
Scientific engagement
Tools to analyze
Process and data architecture needs
The 4 Pillars of Safety Statistics
From: ASA Safety Monitoring Working Group Thought Leader Interviews

12. 1. Scientific engagement
Safety events/questions often change as drug exposure increases. Efficacy questions are much less dynamic – the science of efficacy can be planned for in a way that the science of drug safety cannot
The interchange between clinician and statistician to understand the science is vital to understanding how best to analyze
Drug Safety and Statistics groups need each other to do this well
This requires mutual respect, trust, frequent interaction
Tools of hypothesis testing & power that are the backbone of efficacy are less applicable for safety

13. 2. Tools to analyze safety
Generally not in a hypothesis testing situation with safety data
Study is powered for efficacy. Safety comes along for the ride
What does a non-significant p-value mean in this setting?
Underpowered or no association?
What does a p-value for descriptive purposes mean?
How does it relate to clinical relevance?
False positives & negatives, sensitivity, specificity may be more relevant
Risk difference, confidence intervals, graphics

14. 2. Tools to analyze safety
Risk difference, confidence intervals, graphics
Graphics, well-designed to answer the question, are rigorous statistical evaluations too
From Jiang and Xia, Quantitative Evaluation of Safety in Drug Development

15. 2. Tools to analyze safety
From the paper being discussed, Use of distributions of time to first event to better understand risks Weibull, generalized gamma shape parameters help to better understand whether these may be constant, increasing or decreasing risks

16. 2. Tools to analyze safety
In one picture, the entire story is clearly and transparently described
Volcano plot
From Jiang and Xia, Quantitative Evaluation of Safety in Drug Development

17. 2. Tools to analyze safety
Graphs can elucidate important statistical properties about safety
Subgroup effects aid in identifying We need to clearly understand risk mitigation outliers. When are point estimates relevant?
From Jiang and Xia, Quantitative Evaluation of Safety in Drug Development
From Duke, et al, Seeing is Believing

18. 2. Tools to analyze safety
Apply statistical rigor to graphs just as we do to methods
From Duke, et al, Seeing is Believing

19. 3a. Well-considered process – dynamic and static evaluation
Static evaluations happen a few times with a clearly defined end time. Dynamic evaluations occur repeatedly as information accrues. Control of type I error can be an important consideration in both.
Static Evaluation
Safety evaluation during pre-marketing at pre-determined timepoint, e.g., end of clinical trial, end of phase 2, aggregated data for regulatory filing
Evaluations at pre-specified time in observational studies, claims data, pharmacovigilance studies
Dynamic Evaluation
Sequential safety monitoring of cumulative data over time for known or pre-specified adverse events of interest
Evaluation after each event, or at successive duration of time intervals
Dynamic evaluation includes a “process” of detecting an unknown issue and triggering closer monitoring
Blinded review– companies are actively working on this methodology.
From Gordon, et al, Quantitative Sciences for Safety Monitoring in Clinical Development. Deming conference, Dec 6, 2016

20. 3b. Intelligent data architecture
Nimble process/infrastructure at end of study is good business practice
Nimble infrastructure for on-going safety monitoring is needed for appropriate safety patient management

21. Setup A bit of history Safety analysis – what matters to patients?
Safety Statistics is emerging as a new sub-discipline of statistics
A few thoughts to consider…
Dropouts and their predictive capacity post approval
Baseline risk factors and confounding
Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc

22. How is analysis of safety different from that of efficacy?
Scientific engagement
Tools to analyze
Process and data architecture needs
The 4 Pillars of Safety Statistics
From: ASA Safety Monitoring Working Group Thought Leader Interviews

23. Two pillars statisticians don’t necessarily consider
If safety statisticians don’t play a role in guiding smart process and data architecture for nimble safety monitoring, what functional role will provide the steer?

24. Multifactorial problems
Concomitant medications, subgroups will benefit from better methods, possibly network visualizations such as this one for rotavirus vaccines
Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc
From Duke, et al, Seeing is Believing

25. Role of meta-analysis in assessment of clinical safety data
- Dropouts and their predictive capacity post approval - Baseline risk factors and confounding - Integrating trials for safety, considering multifactorial aspects eg, concomitant meds, subgroups, etc
Time of Approval
Information about benefit
Premarket Phases
Post Market Real World use
B-R Balance
B-R Balance
Information about risk
Meta Analysis
Meta Analysis
From CIOMS X, Described in Gordon, et al, Quantitative Sciences for Safety Monitoring in Clinical Development. Deming conference, Dec 6, 2016

26. References
H. Ma, C Ke, Q Jiang, S. Snapinn, Statistical considerations on the evaluation of imbalances of adverse events in randomized clinical trials. Therapeutic Innovation & Regulatory Science 49(6):
M. Colopy, S. Duke, G. Ball, R. Gordon, F. Ahmad, Q. Jiang, W. Wang, and W. Wang ASA Biopharm’s Safety Monitoring Work Group: Survey of statisticians, thought leaders, and regulatory guidance. In JSM Proceedings, Biopharmaceutical Section. Alexandria, VA: American Statistical Association. pp
Qi Jiang, Amy Xia (eds) Quantitative evaluation of safety in drug development: design, analysis and reporting. Chapman & Hall/biostatistics series. Chapter 11, Safety Graphics. (SP Duke, Q Jiang, L Huang, M Banach, M Cherny).
SP Duke, F Bancken, B Crowe, M Soukup, T Botsis, R Forshee Seeing is believing: Good graphic design principles for medical research. Statistics in Medicine 34(22):
R. Gordon, J. Li, G. Ball, W. Wang. Dec 6, Short course: Quantitative Sciences for Safety Monitoring in Clinical Development. 72nd Annual Deming Conference on Applied Statistics.
CIOMS X Evidence Synthesis and Meta-Analysis for Drug Safety: Report of CIOMS Working Group X. Council for International Organizations of Medical Sciences (CIOMS).