Presentation on theme: "Vaccine Safety Datalink (VSD) Project and Monitoring of Pandemic Influenza Vaccines Aug. 21, 2008 Pandemic Influenza Vaccine: Doses Administered and Safety."— Presentation transcript:
Vaccine Safety Datalink (VSD) Project and Monitoring of Pandemic Influenza Vaccines Aug. 21, 2008 Pandemic Influenza Vaccine: Doses Administered and Safety Training Conference Eric Weintraub, MPH James Baggs, PhD CDC/OCSO/ISO
Goals of Talk Background of the Vaccine Safety Datalink (VSD) Project Background on VSD’s Rapid Cycle Analysis (RCA) Project Plans for monitoring safety of pandemic influenza vaccines within the VSD
Vaccine Safety Datalink (VSD): Background Established in 1990 A collaborative project among CDC and 8 managed care organizations (MCOs) Allows for planned immunization safety studies as well as timely investigations arising from –hypotheses from medical literature and pre-licensure –reports to the Vaccine Adverse Event Reporting System (VAERS),Vaccine Adverse Event Reporting System (VAERS), –changes in immunization schedules, or the introduction of new vaccines.
VSD Population Collects medical care and vaccination data on more than 8.8 million members annually (3% of the US population) As of 12/31/2006 : 2,365,978 children (<18) enrolled –3.2% of US population 6,450,704 adults (≥18) enrolled –2.9% of US population Average yearly birth cohort = 94,701
VSD Sites: 2008 Group Health Cooperative Northwest Kaiser Permanente No. CA Kaiser Permanente So. CA Kaiser Permanente Kaiser Permanente Colorado Health Partners Marshfield Clinic Harvard Pilgrim CDC
VSD Project Utilizes administrative data sources from health plans –Provides medical and immunization histories on >8.8 million people annually Data: –Demographic and enrollment –Vaccination (vaccine type, date of vaccination) –Medical outcomes (outpatient, inpatient, ER, Procedures) –Birth data –Geocoding and census –Pregnancy (developing)
The VSD Distributed Data Model CDC Hub “Direct” “Indirect” SAS Programs, Logs, Output, & Analytical Datasets
VSD History of Success France EK, Safety of the trivalent inactivated influenza vaccine among children: a population- based study.* Archives of Pediatrics & Adolescent Medicine 2004;158(11):1031–1036. Hambidge SJ,. Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old. Journal of the American Medical Association 2006;296(16):1990–1997.
VSD Rapid Cycle Analysis A new approach to surveillance that takes advantage of VSD’s strengths Alternative to traditional post-licensure vaccine safety study methods, which generally take years to complete VSD now updates data on all vaccines and all outcomes every week We conduct updated analyses every week
Basics of VSD Rapid Cycle Analysis Choose specific outcomes to monitor Each week, evaluate the number of events in vaccinated persons Compare it to the expected number of events based on a comparison group –Historical, concurrent Adjust statistically for multiple looks
Sequential Analysis Methods Used in RCA Each week, our analysis includes data from all previous weeks Problem: Repeated testing of the same data increases the chance of false-positive results Need to adjust for this statistically Solution: Two types of methods used: 1.Poisson MaxSPRT Uses a fixed rate as a comparison 2. Flexible Exact Sequential Analysis Uses concurrent comparison group Allows for matching for the entire population
VSD and Monitoring Pandemic Influenza for Vaccine Safety 1.Developing capabilities: Incorporating data from state and local registries to health plan vaccine registries 2.Identification of cohorts: Pre-event: Individuals most likely to be vaccinated (first responders and health care workers) Event: Individuals who received vaccine 3.Analytical Methods and Data Collection: Establish alternative methods to conduct ad-hoc case control methodologies Phone, web, other survey methods Develop chart review instruments for potential vaccine-related outcomes Calculate background rates for potential vaccine-related outcomes Develop modifications to existing seasonal influenza RCA study
Option A: Vaccine distributed outside MCOs –Estimate background rates for potential adverse events –Use alternative data collection methods to identify MCO members who receive vaccines –Monitor rates of identified adverse events in its Dynamic Data Files. –Conduct ad-hoc case control studies –Conduct other ad-hoc studies (e.g. Diary card type studies) Option B: Vaccine distributed within MCOs –Conduct RCA on pandemic influenza vaccine –Monitor uptake and compliance of two-dose vaccine series VSD and Pandemic Influenza Vaccine
VSD Rapid Cycle Analysis for Seasonal Influenza Vaccine Simulated real-time monitoring of adverse events after 1 st dose TIV –2005-06 season –2006-07 season Piloted during 2007-08 season and next influenza season Monitoring 12 adverse events after TIV: –Neurologic: GBS, seizures, meningoencephalitides, Bell’s palsy, other cranial nerve disorders, demyelinating disease, peripheral nervous system disorders, ataxia, hemorrhage stroke, ischemic stroke –Allergic: anaphylaxis, allergic reactions
VSD Investigators At CDC and MCOs Centers For Disease Control and Prevention Julianne Gee Kaiser Permanente of No. California (NCK), Oakland CA Roger Baxter, MD Nicky Klein, MD, PhD Ned Lewis Group Health Cooperative (GHC), Seattle WA Lisa Jackson, MD, MPH Darren Malais Northwest Kaiser Permanente (NWK), Portland OR Allison Naleway, PhD John Mullooly, PhD Karen Riedlinger Lois Drew Harvard Pilgrim /Harv. Vanguard (HAR) Boston MA Tracy Lieu, MD, MPH Richard Platt, MD, MSc Richard Fox Marshfield Clinic Rsch. Foundation (MFC) Marshfield WI Edward Belongia, MD James Donahue, MD Jeremy McCauley Health Partners Rsch Foundation (HPM) Minneapolis MN Jim Nordin, MD Amy Butani Kaiser Permanente of Colorado (KPC) Denver, CO Simon Hambidge, MD, PhD Jason Glanz, MS, PhD David McClure, PhD Christina Clarke So. California Kaiser Permanente (SCK), CA Los Angeles, CA Steven Jacobson, MD, PhD Wansu Chen, MS Sites include > 125 staff working on VSD
Acknowledgements We thank the principal investigators of participating VSD sites, members of the VSD Rapid Cycle Analysis working group, and members of the VSD project for their contributions to this study. *The findings and conclusions in this presentation are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
VSD Acronyms Vaccine Safety DatalinkVSD Distributed Data ModelDDM Dynamic Data FilesDDF Rapid Cycle AnalysisRCA Group Health Cooperative, WAGHC Harvard Pilgrim Health Care, MAHAR HealthPartners Research Foundation, MNHPM Marshfield Clinic, WIMFC Kaiser Permanente of Colorado, COKPC Northern California Kaiser Permanente, CANCK Northwest Kaiser Permanente, ORNWK Southern California Kaiser Permanente SCK
VSD: Strategic Priorities 1.Evaluate the safety of newly licensed vaccines 2.Evaluate the safety of new vaccine recommendations for existing vaccines 3.Evaluate clinical disorders following immunizations 4.Assess vaccine safety in special high risk populations 5.Develop and evaluate methodologies for vaccine safety assessment
RCA Collaborators – partial list James Baggs, CDC Roger Baxter, NCK Bob Davis, CDC Bruce Fireman, NCK Rich Fox, HAR Paul Gargiullo, CDC Julianne Gee, CDC Jason Glanz, KPC Sharon Greene, HAR Nicky Klein, NCK Margarette Kolczak, CDC Martin Kulldorff, HAR Ned Lewis, Kaiser Renny Li, HAR Dave McClure, KPC Jennifer Nelson, GHC Rich Platt, HAR Irene Shui, HAR Eric Weintraub, CDC Katherine Yih, HAR Ruihua Yin, HAR GHC, Group Health Cooperative; HAR, Harvard; KPC, Kaiser Permanente Colorado; NCK, Northern California Kaiser
Constant File Study ID VSD Site Birth Date Sex Enrollment File Study ID Start Date End Date Vaccine File Study ID Vaccination Date Vaccine Facility Site of Introduction Manufacturer Vaccine Lot # Inpatient File Study ID Care Date Length of Stay ICD-9 Code - DX Type - Primary/Secondary Facility Location Outpatient File Study ID Care Date Department ICD-9 Code - DX Type - Primary/Secondary Facility Location Rule-out DX Birth File Study ID Mother ID Facility Gest Age Birth Weight Apgar1 Apgar5 Race Birth State Birth County Mother's Year of Birth Mortality File Study ID Death Date Exact Date Mort Cause of Death File Study ID Cause of Death - ICD9/10 Cause of Death Type Geocode File Study ID Date of Address Census Tract County State Medicaid File Study ID Monthly Medicaid Status VSD Data Dictionary
Setting Up A Rapid Cycle Analysis Choose outcomes to monitor Choose comparison method(s) – e.g., historical, concurrent Set the upper limit for when to stop
Why We Need Early Detection Systems in Vaccine Safety Rare adverse events may be impossible to detect in pre- licensure studies Reports to passive surveillance systems (e.g., the Vaccine Adverse Event Reporting System) often need rapid follow- up Follow-up studies can take months to years using traditional approaches
Maximized Sequential Probability Ratio Testing (maxSPRT) (Kulldorff et al., 2004) A refinement of a classical statistical method (Wald, 1945) Null hypothesis – No excess risk Alternative hypothesis – Increase in risk The test statistic is the log likelihood ratio -- depends on the observed vs. expected number of events
Historical Comparison Method Uses incidence rates from historical data –Advantage: Knowing the historical rate of rare events allows earlier recognition that a small number of cases is unusual –Example: 4 cases of Guillain-Barre syndrome in vaccinees, 0 expected –Limitation: Background rates may vary over time (secular trends)
Concurrent Comparison Method Uses matched controls, e.g., patients making preventive visits –Advantage: Avoids false signaling or missed signals due to secular trends –Limitations: Need to define an appropriate control group – not simple! Vaccines may be adopted rapidly, leaving few controls
Example: Rotashield ® vaccine and intussusception (historical analysis) Log likelihood ratio 1999 Vaccine licensed Aug 98 15 VAERs reports through Jul 99 Vaccine suspended Withdrawn Critical value = 3.3 MaxSPRT analysis would have signaled in May 1999
What Happens When a Signal Occurs? Rapid cycle analysis methods detect signals – values above specified statistical thresholds Not all signals represent a true increase in risk When a signal occurs, we conduct a series of evaluations using traditional epidemiologic methods
How We Evaluate Signals – 1 1.Check data quality 2.Check whether comparison groups are defined appropriately 3.Conduct the analysis using a different control group (e.g., concurrent vs. historical) or different vaccine
How We Evaluate Signals – 2 4.Conduct a temporal scan to see if outcomes cluster during a post-vaccination time window 5.Conduct a definitive study using logistic regression analysis 6.Review charts to confirm or exclude cases as true cases
References Davis RL, Kolczak M, Lewis E, et al. Active surveillance of vaccine safety: a system to detect early signs of adverse events. Epidemiology 2005;16:336-41 Lieu TA, Kulldorff M, Davis RL, et al. Real-time vaccine safety surveillance for the early detection of adverse events. Med Care 2007;45:S89-95 Kulldorff M, Davis RL, Kolczak M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Unpublished data being submitted for publication.