Data Mining AERS FDA’s (Spontaneous) Adverse Event Reporting System Division of Drug Risk Evaluation Office of Drug Safety Carolyn McCloskey, M.D., M.P.H. Drug Safety and Risk Management Advisory Committee May 18, 2005

Outline Brief History of data mining (DM) activities at the FDA Current Use in the Division of Drug Risk Evaluation (DDRE) – CRADA Application Development - WebVDME Pilot - Examples and Selected Conclusions Other CRADA Activities Future Directions in DDRE Pharmacovigilance

Historical Overview February 1998: Office of Women’s Health Grant (Ana Szarfman) March 2003 – July 2005: Cooperative Research & Development Agreement (CRADA) Research in more advanced methodology (Drug-drug interaction & logistic regression modeling)

CRADA - Cooperative Research and Development Agreement Web Visual Data Mining Environment (WebVDME) With Lincoln Technologies, Inc. March 2003 – July 2005

CRADA Objectives User-friendly application Web-based environment Performance Evaluations by User Groups Training Continued development and refinement

Empirical Bayes Geometric Mean (EBGM) An observed/expected score Adjusts for sampling variation (e.g., sample size) No adjustment for reporting bias Allows data stratification in DM software Standard stratification: gender, age group, year

EB05 – EB95 Interval Interval in which the EBGM score could be found because of sampling variability EB05 is the lower bound EB95 is the upper bound 90% probability of EBGM occurring between EB05 and EB95

Example of Sampling Variability Adjustment (for small numbers) Adverse Event (AE) Observed Count (N) Expected Count (E) Obs/Exp* N/E (RR) EBGM** EB05 EB95 Myalgia 1,665 334 4.99 4.97 4.78 5.18 Spinal Osteoarthritis 17 3 6.16 4.54 3.03 6.60 * Obs = Observed; Exp = expected ** EBGM = Empirical Bayes Geometric Mean - adjusts for sampling variability

CRADA Pre-Pilot Performance Evaluations May 2003 – October 2003 WebVDME record retrieval validation with AERS case retrieval Multiple trade & ingredient nomenclature Drug assignment allocations (suspect & concomitant) Duplicate removal logic OIT system performance evaluations

CRADA Pilot Medical Safety Evaluators Evaluated data mining scores for drugs and biologics Indication vs. new signal Labeled vs. unlabeled Innocent bystanders or concomitant medications Drug names Safety Evaluators’ ease of use

CRADA Pilot Epidemiologists Evaluated Temporal trends Drug name selections Suspect & Concomitant selections Stratification Signal strengths Epidemiologists’ ease of use

Pilot Examples New vs. Old Drug EBGM Rankings & Confidence Intervals

New Drug (1 Year) EBGM (EB05-EB95)

OLDER DRUG (>10 Years) EBGM (EB05-EB95)

Selected Pilot Conclusions - 1 WebVDME DM - Statistical tool assists in identifying unusual patterns with AERS data but ! Patterns Need Interpretation!

Selected Pilot Conclusions - 2 Knowledge of data in database imperative to interpret Clinical & pharmacologic activities of drug Other - reporting disproportionalities which also reflect limitations of AERS data

AERS DATABASE LIMITATIONS

Continuing CRADA Activities - DDRE March 2004 - Present Access by interested Reviewers to WebVDME Training Application Refinements addressing Technical problems identified Customization by user needs

Summary – 1 DM Signals in DDRE Assist in prioritizing evaluations of case series Identify associations, NOT a cause or degree of risk Reflect limitations of data

Summary – 2 DM Signals in DDRE Threshold a compromise between sensitivity and specificity (false positives & negatives) Absence of a DM signal ≠ absence of a drug-event association Magnitude of DM scores ≠ magnitude of risk Always require clinical case report and reporting bias evaluation

Future Directions of DM Prospective signal detection Parallel use with traditional pharmacovigilance methods in DDRE Continued research in more advanced methodology (Drug-drug interaction & logistic regression modeling)

Acknowledgments Division of Drug Risk Evaluation Rita Ouellet-Hellstrom, Ph.D., M.P.H. Mary Willy, Ph.D. Mark Avigan, M.D., C.M.

THANK YOU