1. Vaccine Safety Datalink (VSD) Project and Monitoring of Pandemic Influenza Vaccines Aug. 21, 2008 Pandemic Influenza Vaccine: Doses Administered and Safety Training Conference Eric Weintraub, MPH James Baggs, PhD CDC/OCSO/ISO

2. Goals of Talk  Background of the Vaccine Safety Datalink (VSD) Project  Background on VSD’s Rapid Cycle Analysis (RCA) Project  Plans for monitoring safety of pandemic influenza vaccines within the VSD

3. Vaccine Safety Datalink (VSD): Background  Established in 1990  A collaborative project among CDC and 8 managed care organizations (MCOs)  Allows for planned immunization safety studies as well as timely investigations arising from –hypotheses from medical literature and pre-licensure –reports to the Vaccine Adverse Event Reporting System (VAERS),Vaccine Adverse Event Reporting System (VAERS), –changes in immunization schedules, or the introduction of new vaccines.

4. VSD Population  Collects medical care and vaccination data on more than 8.8 million members annually (3% of the US population)  As of 12/31/2006 :  2,365,978 children (<18) enrolled –3.2% of US population  6,450,704 adults (≥18) enrolled –2.9% of US population  Average yearly birth cohort = 94,701

5. VSD Sites: 2008 Group Health Cooperative Northwest Kaiser Permanente No. CA Kaiser Permanente So. CA Kaiser Permanente Kaiser Permanente Colorado Health Partners Marshfield Clinic Harvard Pilgrim CDC

6. VSD Project  Utilizes administrative data sources from health plans –Provides medical and immunization histories on >8.8 million people annually  Data: –Demographic and enrollment –Vaccination (vaccine type, date of vaccination) –Medical outcomes (outpatient, inpatient, ER, Procedures) –Birth data –Geocoding and census –Pregnancy (developing)

7. The VSD Distributed Data Model CDC Hub “Direct” “Indirect” SAS Programs, Logs, Output, & Analytical Datasets

8. VSD History of Success  France EK, Safety of the trivalent inactivated influenza vaccine among children: a population- based study.* Archives of Pediatrics & Adolescent Medicine 2004;158(11):1031–1036.  Hambidge SJ,. Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old. Journal of the American Medical Association 2006;296(16):1990–1997.

9. VSD Rapid Cycle Analysis  A new approach to surveillance that takes advantage of VSD’s strengths  Alternative to traditional post-licensure vaccine safety study methods, which generally take years to complete  VSD now updates data on all vaccines and all outcomes every week  We conduct updated analyses every week

10. Basics of VSD Rapid Cycle Analysis  Choose specific outcomes to monitor  Each week, evaluate the number of events in vaccinated persons  Compare it to the expected number of events based on a comparison group –Historical, concurrent  Adjust statistically for multiple looks

11. Sequential Analysis Methods Used in RCA  Each week, our analysis includes data from all previous weeks  Problem: Repeated testing of the same data increases the chance of false-positive results  Need to adjust for this statistically  Solution: Two types of methods used: 1.Poisson MaxSPRT Uses a fixed rate as a comparison 2. Flexible Exact Sequential Analysis Uses concurrent comparison group Allows for matching for the entire population

12. VSD and Monitoring Pandemic Influenza for Vaccine Safety 1.Developing capabilities: Incorporating data from state and local registries to health plan vaccine registries 2.Identification of cohorts: Pre-event: Individuals most likely to be vaccinated (first responders and health care workers) Event: Individuals who received vaccine 3.Analytical Methods and Data Collection: Establish alternative methods to conduct ad-hoc case control methodologies Phone, web, other survey methods Develop chart review instruments for potential vaccine-related outcomes Calculate background rates for potential vaccine-related outcomes Develop modifications to existing seasonal influenza RCA study

13.  Option A: Vaccine distributed outside MCOs –Estimate background rates for potential adverse events –Use alternative data collection methods to identify MCO members who receive vaccines –Monitor rates of identified adverse events in its Dynamic Data Files. –Conduct ad-hoc case control studies –Conduct other ad-hoc studies (e.g. Diary card type studies)  Option B: Vaccine distributed within MCOs –Conduct RCA on pandemic influenza vaccine –Monitor uptake and compliance of two-dose vaccine series VSD and Pandemic Influenza Vaccine

14. VSD Rapid Cycle Analysis for Seasonal Influenza Vaccine  Simulated real-time monitoring of adverse events after 1 st dose TIV –2005-06 season –2006-07 season  Piloted during 2007-08 season and next influenza season  Monitoring 12 adverse events after TIV: –Neurologic: GBS, seizures, meningoencephalitides, Bell’s palsy, other cranial nerve disorders, demyelinating disease, peripheral nervous system disorders, ataxia, hemorrhage stroke, ischemic stroke –Allergic: anaphylaxis, allergic reactions

15. VSD Investigators At CDC and MCOs Centers For Disease Control and Prevention  Julianne Gee Kaiser Permanente of No. California (NCK), Oakland CA  Roger Baxter, MD  Nicky Klein, MD, PhD  Ned Lewis  Group Health Cooperative (GHC), Seattle WA  Lisa Jackson, MD, MPH  Darren Malais Northwest Kaiser Permanente (NWK), Portland OR  Allison Naleway, PhD  John Mullooly, PhD  Karen Riedlinger  Lois Drew Harvard Pilgrim /Harv. Vanguard (HAR) Boston MA  Tracy Lieu, MD, MPH  Richard Platt, MD, MSc  Richard Fox Marshfield Clinic Rsch. Foundation (MFC) Marshfield WI  Edward Belongia, MD  James Donahue, MD  Jeremy McCauley Health Partners Rsch Foundation (HPM) Minneapolis MN  Jim Nordin, MD  Amy Butani Kaiser Permanente of Colorado (KPC) Denver, CO  Simon Hambidge, MD, PhD  Jason Glanz, MS, PhD  David McClure, PhD  Christina Clarke So. California Kaiser Permanente (SCK), CA Los Angeles, CA  Steven Jacobson, MD, PhD  Wansu Chen, MS Sites include > 125 staff working on VSD

16. Acknowledgements We thank the principal investigators of participating VSD sites, members of the VSD Rapid Cycle Analysis working group, and members of the VSD project for their contributions to this study. *The findings and conclusions in this presentation are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

17. Additional Slides

18. VSD Acronyms  Vaccine Safety DatalinkVSD  Distributed Data ModelDDM  Dynamic Data FilesDDF  Rapid Cycle AnalysisRCA  Group Health Cooperative, WAGHC  Harvard Pilgrim Health Care, MAHAR  HealthPartners Research Foundation, MNHPM  Marshfield Clinic, WIMFC  Kaiser Permanente of Colorado, COKPC  Northern California Kaiser Permanente, CANCK  Northwest Kaiser Permanente, ORNWK  Southern California Kaiser Permanente SCK

19. VSD: Strategic Priorities 1.Evaluate the safety of newly licensed vaccines 2.Evaluate the safety of new vaccine recommendations for existing vaccines 3.Evaluate clinical disorders following immunizations 4.Assess vaccine safety in special high risk populations 5.Develop and evaluate methodologies for vaccine safety assessment

20. VSD Basic Tables

21. RCA Collaborators – partial list  James Baggs, CDC  Roger Baxter, NCK  Bob Davis, CDC  Bruce Fireman, NCK  Rich Fox, HAR  Paul Gargiullo, CDC  Julianne Gee, CDC  Jason Glanz, KPC  Sharon Greene, HAR  Nicky Klein, NCK  Margarette Kolczak, CDC  Martin Kulldorff, HAR  Ned Lewis, Kaiser  Renny Li, HAR  Dave McClure, KPC  Jennifer Nelson, GHC  Rich Platt, HAR  Irene Shui, HAR  Eric Weintraub, CDC  Katherine Yih, HAR  Ruihua Yin, HAR GHC, Group Health Cooperative; HAR, Harvard; KPC, Kaiser Permanente Colorado; NCK, Northern California Kaiser

22. VSD Basic Tables

23. Constant File Study ID VSD Site Birth Date Sex Enrollment File Study ID Start Date End Date Vaccine File Study ID Vaccination Date Vaccine Facility Site of Introduction Manufacturer Vaccine Lot # Inpatient File Study ID Care Date Length of Stay ICD-9 Code - DX Type - Primary/Secondary Facility Location Outpatient File Study ID Care Date Department ICD-9 Code - DX Type - Primary/Secondary Facility Location Rule-out DX Birth File Study ID Mother ID Facility Gest Age Birth Weight Apgar1 Apgar5 Race Birth State Birth County Mother's Year of Birth Mortality File Study ID Death Date Exact Date Mort Cause of Death File Study ID Cause of Death - ICD9/10 Cause of Death Type Geocode File Study ID Date of Address Census Tract County State Medicaid File Study ID Monthly Medicaid Status VSD Data Dictionary

24. Setting Up A Rapid Cycle Analysis  Choose outcomes to monitor  Choose comparison method(s) – e.g., historical, concurrent  Set the upper limit for when to stop

25. Why We Need Early Detection Systems in Vaccine Safety  Rare adverse events may be impossible to detect in pre- licensure studies  Reports to passive surveillance systems (e.g., the Vaccine Adverse Event Reporting System) often need rapid follow- up  Follow-up studies can take months to years using traditional approaches

26. Maximized Sequential Probability Ratio Testing (maxSPRT) (Kulldorff et al., 2004)  A refinement of a classical statistical method (Wald, 1945)  Null hypothesis – No excess risk  Alternative hypothesis – Increase in risk  The test statistic is the log likelihood ratio -- depends on the observed vs. expected number of events

27. Historical Comparison Method  Uses incidence rates from historical data –Advantage: Knowing the historical rate of rare events allows earlier recognition that a small number of cases is unusual –Example: 4 cases of Guillain-Barre syndrome in vaccinees, 0 expected –Limitation: Background rates may vary over time (secular trends)

28. Concurrent Comparison Method  Uses matched controls, e.g., patients making preventive visits –Advantage: Avoids false signaling or missed signals due to secular trends –Limitations:  Need to define an appropriate control group – not simple!  Vaccines may be adopted rapidly, leaving few controls

29. Example: Rotashield ® vaccine and intussusception (historical analysis) Log likelihood ratio 1999 Vaccine licensed Aug 98 15 VAERs reports through Jul 99 Vaccine suspended Withdrawn Critical value = 3.3 MaxSPRT analysis would have signaled in May 1999

30. What Happens When a Signal Occurs?  Rapid cycle analysis methods detect signals – values above specified statistical thresholds  Not all signals represent a true increase in risk  When a signal occurs, we conduct a series of evaluations using traditional epidemiologic methods

31. How We Evaluate Signals – 1 1.Check data quality 2.Check whether comparison groups are defined appropriately 3.Conduct the analysis using a different control group (e.g., concurrent vs. historical) or different vaccine

32. How We Evaluate Signals – 2 4.Conduct a temporal scan to see if outcomes cluster during a post-vaccination time window 5.Conduct a definitive study using logistic regression analysis 6.Review charts to confirm or exclude cases as true cases

33. References Davis RL, Kolczak M, Lewis E, et al. Active surveillance of vaccine safety: a system to detect early signs of adverse events. Epidemiology 2005;16:336-41 Lieu TA, Kulldorff M, Davis RL, et al. Real-time vaccine safety surveillance for the early detection of adverse events. Med Care 2007;45:S89-95 Kulldorff M, Davis RL, Kolczak M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Unpublished data being submitted for publication.