Barry Rhodes NHSN Innovations: Creation of Synthetic Data Sets For Ventilator – Associated Events Funded Through the CHIIC Awards 2014

What is the National Healthcare Safety Network (NHSN)?  A secure, internet-based system for monitoring healthcare-associated adverse events and practices  Participating healthcare facilities use the NHSN web application to enter, analyze, and share data  Data entry is predominantly manual with some electronic submission via Clinical Document Architecture (CDA) documents  States get HAI data from NHSN based on state mandates  CMS gets aggregated data for quality monitoring and payment purposes

NHSN Supports  Over 36,000 users representing 14,000+ facilities  3,400 facilities ~ 25% reporting at least some data electronically via CDA  National-level public health surveillance (CDC)  State reporting mandates (30 states and DC)  CMS reimbursement programs (Inpatient Prospective Payment System)

NHSN Electronic Data Flow Diagram Healthcare Facility NHSN State Public Health CDC Analysts CDA Messages Identified Data Sets (SAS, CSV, Excel) De-identified Data Sets CMS Quality Measurement Data

Public Health Electronic Reporting Food Chain Public Health or Facility Action Validation (State or Facility Based) Electronic Reporting CDA creation Event Determination (algorithmic or human) Data aggregation (EHR) Data normalization (vocab mapping) We are here !

So What’s the Problem?  How do we know the definitions are being applied properly?  Working with data providers to create conformant CDA messages is a good thing.  The CDA represents the provider’s assessment that there is something to report based on NHSN definitions  There is a validation step but that is manual, labor intensive and can be limited in scope

What’s Needed?  Need a way to electronically validate that the data provider is applying the NHSN definitions properly  So that when a CDA message is sent to NHSN, we can have some level of comfort that it accurately represents an HAI event.  Two approaches to this problem:  Web Services  Synthetic Data Sets

Funded CHIIC Project  Synthetic Patient Dataset for the Determination of Ventilator-Associated Events in Electronic Health Record Systems   Barry Rhodes, PhD  Shelley Magill, MD, PhD  Cindy Gross, MT (ASCP)

Procedure 1. Working with NHSN SMEs and a healthcare facility, create synthetic EHR patient data seeded with Ventilator-Associated Events (VAEs)  We used a de-identified data set from a partner facility to make the data as real as possible.  We tried to think up all the “normal” possibilities for VAEs and as many “edge cases” as seems appropriate.  Created the data set in XML 2. Also created a result set with gives the VAE determination for each patient

Procedure (continued) 3. Provide the data set and result set to vendors and other data providers to import into their systems 4. Allow them to self-certify that their local implementations of the VAE definitions are valid.

CEFOX AMK ……..

Current Status  Sent dataset to vendor community  Received feedback from EPIC (they found an error actually!)  Working with Partners Healthcare in Boston to validate their VAE efforts  Waiting for more feedback to publish the dataset with instructions to the NHSN web site.

Where is This Pilot Going?  Repeat the process for other HAIs  Extend this scenario to a more formal EHR certification process. 1.Provide synthetic dataset to vendor 2.Vendor submits CDAs extracted from the data set to an NSHN certification site 3.NHSN certification checks the CDAs against the known result set and certifies the vendor.

NHSN Innovations Philosophy and Why CHIIC is such a Good Thing for CDC  This project is one of several that are ongoing in NHSN under the heading of Informatics Innovations  I believe there are many low cost, low technology, quick turn around projects that may have a broad impact on very difficult issues.  Projects like this are good because by virtue of their low cost, they have the “privilege of failure”, i.e. If we do ten small projects and 8 fail, we still have two successful ones and we’ve learned 8 new things!

Comments/ Questions? Please send accolades to