Presentation is loading. Please wait.

Presentation is loading. Please wait.

© 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 1 Methodologies and Automated Applications for Post-Marketing Outcomes.

Similar presentations


Presentation on theme: "© 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 1 Methodologies and Automated Applications for Post-Marketing Outcomes."— Presentation transcript:

1 © 2003 By Default! A Free sample background from Slide 1 Methodologies and Automated Applications for Post-Marketing Outcomes Surveillance of Medical Devices and Medications Michael E. Matheny, MD, MS NLM Biomedical Informatics Fellow Decision Systems Group, Department of Radiology Brigham & Women’s Hospital, Boston, MA

2 © 2003 By Default! A Free sample background from Slide 2 Outline Post-Marketing Surveillance Background Post-Marketing Surveillance Background Statistical Methodology Development Statistical Methodology Development Computer Application Development Computer Application Development Clinical Examples Clinical Examples Future Directions Future Directions

3 © 2003 By Default! A Free sample background from Slide 3 Background Surveillance Rationale Phase 3 Trials insufficient to ensure adequate safety of medications and devices Phase 3 Trials insufficient to ensure adequate safety of medications and devices –Low frequency events are not detected –Protected populations (pregnant women, children) and more ill populations not represented –Complications delayed by a number of years cannot be detected

4 © 2003 By Default! A Free sample background from Slide 4 Background FDA Medical Devices 1,700 types of devices 1,700 types of devices 500,000 device models 500,000 device models 23,000 manufacturers 23,000 manufacturers

5 © 2003 By Default! A Free sample background from Slide 5 Background FDA New Drug Applications (NDA)

6 © 2003 By Default! A Free sample background from Slide 6 Background Current Post-Marketing Surveillance Combination of mandatory and voluntary adverse event reporting Combination of mandatory and voluntary adverse event reporting –Mandatory reporting by manufacturers and health facilities –Voluntary MedWatch / MAUDE reports by providers and patients 2004 Drug-Related Adverse Event Reports Total422,889 Manufacturer & Facility Reports Manufacturer & Facility Reports401,396 MedWatch MedWatch21,493

7 © 2003 By Default! A Free sample background from Slide 7 Background MAUDE Cypher Reporting Rate FDA WarningFDA Warning Cancelled

8 © 2003 By Default! A Free sample background from Slide 8 Background Current Post-Marketing Surveillance ‘Phase 4’ Trials ‘Phase 4’ Trials –Poor Compliance As of March 2006 report, 797 of 1231 (65%) agreed- upon trials had yet to be startedAs of March 2006 report, 797 of 1231 (65%) agreed- upon trials had yet to be started –Barriers Lack of manufacturer incentivesLack of manufacturer incentives –Expensive –Drug already on the market Lack of regulatory enforcementLack of regulatory enforcement

9 © 2003 By Default! A Free sample background from Slide 9 Background Medical Device Recalls Boston Scientific cardiac stent (1998) Boston Scientific cardiac stent (1998) –Balloon rupture at low pressures Guidant cardio-defibrillator (2005) Guidant cardio-defibrillator (2005) –Malfunction due to electrical short Vioxx (2004) Vioxx (2004) –cardiovascular complications Tequin (2006) Tequin (2006) –Hypoglycemia and hyperglycemia

10 © 2003 By Default! A Free sample background from Slide 10 Background FDA Response Increasing demands for Phase 4 trials Increasing demands for Phase 4 trials Legislation to increase quality of adverse event reporting Legislation to increase quality of adverse event reporting Emphasizing trial registries (clinicaltrials.gov) as way to prevent omission of results Emphasizing trial registries (clinicaltrials.gov) as way to prevent omission of results Commissioned IOM report “The Future of Drug Safety” Commissioned IOM report “The Future of Drug Safety”

11 © 2003 By Default! A Free sample background from Slide 11 Background Adverse Event Data Continuum Phase 3 Trials MAUDE / MedWatch Voluntary Registry Mandatory Registry Internal Validity Generalizibility+/ Breadth Immediacy Lack of Bias

12 © 2003 By Default! A Free sample background from Slide 12 Statistical Methods Medical Outcomes Monitoring Using registry data that tracks all patients allows different types of analysis than used in the FDA’s adverse event reporting systems Using registry data that tracks all patients allows different types of analysis than used in the FDA’s adverse event reporting systems No generally accepted methods for monitoring registry data for adverse events No generally accepted methods for monitoring registry data for adverse events –Lack of sufficient discrete electronic data sources to construct registries –Some outcomes are challenging or expensive to track for an entire population

13 © 2003 By Default! A Free sample background from Slide 13 Objective Develop methodologies and implement an automated computer monitoring system to perform outcomes surveillance of registry data for new medical devices and medications Develop methodologies and implement an automated computer monitoring system to perform outcomes surveillance of registry data for new medical devices and medications

14 © 2003 By Default! A Free sample background from Slide 14 Statistical Methods Statistical Process Control

15 © 2003 By Default! A Free sample background from Slide 15 Statistical Methods Bayesian Updating Statistics

16 © 2003 By Default! A Free sample background from Slide 16 Statistical Methods Establishing Baseline Data Primary Data Sources Primary Data Sources –Phase 3 trial data –Post-Marketing data from a closely related medication/device Alternative Data Sources Alternative Data Sources

17 © 2003 By Default! A Free sample background from Slide 17 Statistical Methods Establishing Alerting Thresholds Use number of events and sample size to calculate proportion with confidence intervals Use number of events and sample size to calculate proportion with confidence intervals Typically, medical domains use 95% CI or 1.96 sigma from the point estimate Typically, medical domains use 95% CI or 1.96 sigma from the point estimate

18 © 2003 By Default! A Free sample background from Slide 18 Statistical Methods Establishing Alerting Thresholds SPC BUS

19 © 2003 By Default! A Free sample background from Slide 19 Statistical Methods Establishing Alerting Thresholds

20 © 2003 By Default! A Free sample background from Slide 20 Statistical Methods Establishing Alerting Thresholds Wilson’s method of comparison between two proportions Wilson’s method of comparison between two proportions

21 © 2003 By Default! A Free sample background from Slide 21 Statistical Methods Risk Stratification Allows creating subgroups for separate analyses Allows creating subgroups for separate analyses Single variable Single variable Logistic regression model with scoring thresholds Logistic regression model with scoring thresholds

22 © 2003 By Default! A Free sample background from Slide 22 Application Development DELTA Data Extraction and Longitudinal Time Analysis (DELTA) Data Extraction and Longitudinal Time Analysis (DELTA) Design Goals Design Goals –Generic data import format –Allow both prospective and retrospective analyses –Modular framework to allow sequential addition of statistical methodologies –Multiple alerting methods –Any number of concurrent ongoing analyses

23 © 2003 By Default! A Free sample background from Slide 23 Application Development DELTA Data Dictionary Statistical Modules DELTA Database Clinical Data Entry Source Database Source IT Manager SPC BUS VPN Intranet DELTA Users Web Server

24 © 2003 By Default! A Free sample background from Slide 24 Application Test Data Cypher Drug-Eluting Stent (DES) Setting: Setting: –Brigham & Women’s Hospital (07/2003 – 12/2004) Population: Population: –All patients receiving a drug-eluting stent (2270) Outcome: Outcome: –Post-procedural in-hospital mortality (27) Baseline: Baseline: –University of Michigan Data ( )

25 © 2003 By Default! A Free sample background from Slide 25 Application Development DELTA

26 © 2003 By Default! A Free sample background from Slide 26 Application Development DELTA

27 © 2003 By Default! A Free sample background from Slide 27 Application Development DELTA

28 © 2003 By Default! A Free sample background from Slide 28 Application Development DELTA

29 © 2003 By Default! A Free sample background from Slide 29 Application Development DELTA

30 © 2003 By Default! A Free sample background from Slide 30 Application Development DELTA

31 © 2003 By Default! A Free sample background from Slide 31 Application Development DELTA

32 © 2003 By Default! A Free sample background from Slide 32 Application Development DELTA

33 © 2003 By Default! A Free sample background from Slide 33 Application Development DELTA

34 © 2003 By Default! A Free sample background from Slide 34 Risk Stratification Potential Solution Incorporate individual risk prediction models in order to adjust for case mix and illness severity Incorporate individual risk prediction models in order to adjust for case mix and illness severity

35 © 2003 By Default! A Free sample background from Slide 35 Possible Risk Prediction Methods Linear / Logistic Regression Linear / Logistic Regression Artificial Neural Networks Artificial Neural Networks Bayesian Networks Bayesian Networks Support Vector Machines Support Vector Machines

36 © 2003 By Default! A Free sample background from Slide 36 LR External Validation Models Model DatesLocationSample NY NY5827 NY – 1994NY62670 CC – 1994Cleveland, OH12985 NNE – 1996NH, ME, MA, VT15331 MI – 2000Detroit, MI10796 BWH – 1999Boston, MA 2804 ACC – 2000National100253

37 © 2003 By Default! A Free sample background from Slide 37 LR External Validation Setting: Setting: –Brigham & Women’s Hospital (01/2002 – 09/2004) Population: Population: –All patients undergoing percutaneous coronary intervention (5216) Outcome: Outcome: –Post-procedural in-hospital mortality (71)

38 © 2003 By Default! A Free sample background from Slide 38 LR External Validation Results CurveDeathsAUCHL χ 2 HL (p) NY <0.001 NY <0.001 CC <0.001 NNE <0.001 MI <0.001 BWH <0.001 ACC <0.001 BWH

39 © 2003 By Default! A Free sample background from Slide 39 LR External Validation Conclusions Excellent discrimination across all models Excellent discrimination across all models Calibration (Hosmer-Lemeshow) poor for all models but recent local one Calibration (Hosmer-Lemeshow) poor for all models but recent local one Addressed categorical risk stratification by keeping all records in one stratum Addressed categorical risk stratification by keeping all records in one stratum Calibration problems over time limit application, and require exploration of recalibration methods Calibration problems over time limit application, and require exploration of recalibration methods

40 © 2003 By Default! A Free sample background from Slide 40 OPUS (TIMI-16) Setting: Setting: –888 Hospitals in 27 Countries Intervention: Intervention: –Oral IIb-IIIa Inhibitor vs Placebo Population: Population: –Intervention Arm [Both arms identical at 30 days] (6867) Outcome: Outcome: –30 day mortality –Trial stopped early due to elevation in intervention arm Baseline: Baseline: –Control Arm (3421)

41 © 2003 By Default! A Free sample background from Slide 41 OPUS (TIMI-16) 30 Day Mortality Control Intervention PeriodEventsPatientsEvent Rate (%)EventsPatientsEvent Rate (%)p * * , , , , , , ,6102.2< , ,4102.1< , , , , , ,

42 © 2003 By Default! A Free sample background from Slide 42 OPUS (TIMI-16) Alert Summary

43 © 2003 By Default! A Free sample background from Slide 43 CLARITY (TIMI-28) Setting: Setting: –313 Hospitals in 23 Countries Intervention: Intervention: –Oral Anti-Platelet Agent vs Placebo Population Population –Intervention Arm (1751) Outcome: Outcome: –Major Bleeding –DSMB concerned, but trial did not stop early Baseline Baseline –Control Arm (1739)

44 © 2003 By Default! A Free sample background from Slide 44 CLARITY (TIMI-28) Major Bleeding ControlIntervention PeriodEventsPatientsEvent Rate (%)EventsPatientsEvent Rate (%)p * *

45 © 2003 By Default! A Free sample background from Slide 45 CLARITY (TIMI-28) Major Bleeding ControlIntervention PeriodEventsPatientsEvent Rate (%)EventsPatientsEvent Rate (%)p , , , , , , , , , , , ,

46 © 2003 By Default! A Free sample background from Slide 46 CLARITY (TIMI-28) Alert Summary

47 © 2003 By Default! A Free sample background from Slide 47 OPUS /CLARITY Conclusions SPC performed well in the positive study, but did have some false positive alerts in the negative study SPC performed well in the positive study, but did have some false positive alerts in the negative study LR stratified SPC failed to alert early in the positive study, but performed well in the negative study LR stratified SPC failed to alert early in the positive study, but performed well in the negative study BUS was more specific than SPC in both studies BUS was more specific than SPC in both studies

48 © 2003 By Default! A Free sample background from Slide 48 Sensitivity Analysis Setting: Setting: –Brigham & Women’s Hospital (01/2002 – 12/2004) Population: Population: –All patients undergoing percutaneous coronary intervention (6175) Outcome: Outcome: –Post-procedural major adverse cardiac events (403) DeathDeath Post-Procedural Myocardial InfarctionPost-Procedural Myocardial Infarction Repeat VascularizationRepeat Vascularization Baseline: Baseline: –Arbitrarily set event rates and sample sizes

49 © 2003 By Default! A Free sample background from Slide 49 Sensitivity Analysis Results

50 © 2003 By Default! A Free sample background from Slide 50 Clinical Alert Setting: Setting: –Brigham & Women’s Hospital (01/2002 – 12/2004) Population: Population: –All patients receiving a vascular closure device after percutaneous coronary intervention (3947) Outcome: Outcome: –Retroperitoneal Hemorrhage (25) Baseline: Baseline: –Stanford University Data (2000 – 2004)

51 © 2003 By Default! A Free sample background from Slide 51 Event Rate Elevation

52 © 2003 By Default! A Free sample background from Slide 52 Manual Review Triggered root cause analysis Triggered root cause analysis Manual chart review and multivariable analsysis Manual chart review and multivariable analsysis Final Result: Not related, confounded by indication Final Result: Not related, confounded by indication

53 © 2003 By Default! A Free sample background from Slide 53 Future Work Methodology Address Calibration Concerns Address Calibration Concerns –Recalibration of Logistic Regression models –Development of Machine Learning Risk Prediction Models Address BUS Insensitivity Address BUS Insensitivity –Incorporate data weight decay over time

54 © 2003 By Default! A Free sample background from Slide 54 Future Work Application Outcomes DB DELTA Agent Brigham and Women’s Hospital DELTA reports DELTA Server Massachusetts Data Analysis Center (Mass-DAC) SMTP Server alert Outcomes DB DELTA Agent Massachusetts General Hospital Outcomes DB DELTA Agent St. Elizabeth’s Medical Center MA Claims DB MA Death Index

55 © 2003 By Default! A Free sample background from Slide 55 Future Work New Medication Outcomes Surveillance New Medication Outcomes Surveillance Inpatient (versus Outpatient) Inpatient (versus Outpatient) –More frequent monitoring –Higher quality source data –Outcomes easier to capture

56 © 2003 By Default! A Free sample background from Slide 56 Future Work Develop Data Repository Patient Demographics Local Institution DELTA Server Centralized Data Repository Computerized Order Entry Laboratory Progress Notes (NLP) Hospital Billing Medication Administration Radiology IS State Death Index Other Outcomes

57 © 2003 By Default! A Free sample background from Slide 57 Future Work Initial Framework New medication laboratory monitoring protocol New medication laboratory monitoring protocol Standard measures that are most commonly affected in new medications Standard measures that are most commonly affected in new medications –AST, ALT, Creatinine, WBC, Platelets Establish reasonable baselines Establish reasonable baselines –Closely Related medication lab results –Unrelated medication lab results –Expert Panel Estimation

58 © 2003 By Default! A Free sample background from Slide 58 Acknowledgements Mentors Mentors –Lucila Ohno-Machado, MD, PhD –Frederic S. Resnic, MD, MS Collaborators Collaborators –Nipun Arora, MD –Sharon Lise-Normand, PhD –Ewout Steyerberg, PhD Programming Team Programming Team –Richard Cope –Barry Coflan –Atul Tatke Funding Funding –NLM R01-LM –NLM T15-LM-07092

59 © 2003 By Default! A Free sample background from Slide 59 Michael Matheny, MD MS Brigham & Women’s Hospital Thorn Francis Street Boston, MA Michael Matheny, MD MS Brigham & Women’s Hospital Thorn Francis Street Boston, MA The End


Download ppt "© 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 1 Methodologies and Automated Applications for Post-Marketing Outcomes."

Similar presentations


Ads by Google