Ami Shah, MPH Alicia Shugart, MA Priti Patel, MD Christi Lines, MPH November 19, 2014 Disclaimer: The findings and conclusions in this report/presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. The Past, Present, and Future: NHSN Analysis Resources and How to Make Them Work for You National Center for Emerging and Zoonotic Infectious Diseases Division of Heatlhcare Quallity Promtoion

Analysis in NHSN - Outline  A brief history of NHSN  Value of analysis  What Analysis tools are presently available?  Finding and using Analysis tools  Tailoring reports to your needs  Future expectations for NHSN Analysis

A BRIEF HISTORY OF NHSN

Dialysis Surveillance Early On  Via annual survey, CDC conducted surveillance of hemodialysis associated hepatitis since the early 1970s  1999: CDC established the Dialysis Surveillance Network (DSN)  A voluntary national surveillance system that monitored: IV Antimicrobial Starts Positive Blood Cultures Hospitalization  DSN was designed for dialysis center personnel, NOT infection control professionals

Dialysis Surveillance Early On  2005: Providers using DSN transitioned to using the National Healthcare Safety Network (NHSN)  Approximately 100 dialysis facilities voluntarily participated in the early years of NHSN  Most were hospital-affiliated dialysis units  2008: First publication of NHSN outpatient dialysis facility data  Dialysis Surveillance Report: National Healthcare Safety Network (NHSN)—Data Summary for Seminars in Dialysis—Vol 21, No 1 (January–February) 2008 pp. 24–28  2009: CDC Dialysis BSI Prevention Collaborative established  Facilities used NHSN for prevention initiatives

NHSN Changes and QIP  2011: Dialysis Event Reporting Changed  “Hospitalization” event type was discontinued  New dialysis event type introduced: Pus, redness, and increased swelling at the vascular access site (PRS)  Hospitalization and death were included as outcomes related to dialysis events  End of 2011: Centers for Medicare and Medicaid Services (CMS) published the End Stage Renal Disease (ESRD) Quality Incentive Program (QIP) Rule Calendar Year 2012 (Payment Year 2014)  QIP incentivized NHSN enrollment and reporting  Anticipated a dramatic increase in NHSN enrollment

Outpatient Hemodialysis Facility Enrollment in NHSN, Present First CMS ESRD QIP final rule published in November 2011 for participation in CY ,027 5,694

NHSN Growth  2012: First year of CMS ESRD QIP incentivized participation in NHSN  Over 5,500 additional outpatient dialysis facilities enrolled  2014: CDC implemented the NHSN Dialysis Component to tailor the user interface for dialysis facility users  NHSN continues to improve with updates a few times per year  Addition of new surveillance options  Improvements to the user interface in response to user feedback  Introduction of new and improved analytical tools

Development of Analytical Tools for Dialysis Users  Dialysis reports initially based upon hospital reports  Reports were mixed among hospital reports  CDC developed the Centers for Medicaid and Medicare Services End Stage Renal Disease Quality Incentive Program Report (CMS ESRD QIP) with dialysis users in mind  Facilities needed help to ensure that criteria were met for reporting requirements mandated by CMS  2012 – Present: CDC has observed a large uptake in the use of the QIP report and other analysis tools 2012: 100’s of QIP reports run After Present: 45,000+ QIP reports run

Finding Dialysis Analysis Tools Early On  NHSN Dialysis Event Reporting and Analysis was housed in the Patient Safety Component  Dialysis reports were mixed in with hospital reports in the “Device- Associated Module” folder

Establishing NHSN Aggregate Rates  Current NHSN aggregate data are from facilities that entered data between January 2007 — April  CDC’s intention is to update and publish new aggregate rates once a clean and complete year of data becomes available. Location Access Type Summary Yr/QtrMonths Number Bloodstream Infections Patient- months Bloodstream Infection Rate/100 patient-months NHSN Bloodstream Infection Pooled Mean Rate/100 patient-months Incidence Density p-value Incidence Density Percentile All2014Q Fistula2014Q Graft2014Q Other Access2014Q Tunneled2014Q Nontunneled2014Q Any CVC2014Q Most Dialysis Rate Tables provide aggregate data from all of NHSN. This information can be used to compare each facility to the rest of NHSN.

NHSN and Analysis Beginnings  To summarize…  The Dialysis Surveillance Network preceded the introduction of NHSN and monitored different dialysis event types  NHSN was created in 2005 In 2012 as a result of QIP, enrollment increased exponentially All analytical resources were found under the Patient Safety Component o A single analysis tree view was used to address the needs of both hospitals and dialysis facilities o CDC developed the QIP report with dialysis users in mind o CDC’s ability to update aggregate rates annually depends on data quality of NHSN  NHSN continues to grow and improve

NHSN ANALYSIS: TODAY Finding and using available NHSN tools…

NHSN Analysis: Today  In August 2014, the new Dialysis Component was launched and all Analysis options related to Dialysis moved out of the Patient Safety Component!  Analysis output options are presented in a streamlined and user-friendly layout  Reports are separated into pertinent categories  Reports can be run as-they-are or modified to better suit your needs

NHSN Analysis: Today  The report type determines how data are displayed  Report types include:  Line Listings  Frequency Tables  Pie Charts  Rate Tables  Run Charts

CREATE A REPORT IN 3 STEPS

Creating Reports in NHSN  Experiment with the Analysis function – You won’t break anything!  NHSN does the work for you!

Create a Report in 3 Steps 1. Generate Data Sets 2. Select a Report  Modifying the report is optional 3. ‘Run’ the Report

Step 1 - Generate Data Sets  Data sets are the files NHSN uses to run reports  Generating new data sets captures all of your facility’s NHSN data so that reports are created using complete, up-to-date information  Each user has their own analysis data sets  May take several minutes to generate

Step 1 - Generate Data Sets  From the navigation bar, select ‘Analysis,’ then ‘Generate Data Sets’  If data sets exist, the date generated is shown Only information in NHSN before the “Date Last Generated” will be included in the reports.

Step 1 - Generate Data Sets  Click “Generate New” and then select ‘OK’ to replace existing data sets  Wait for update

Step 2 – Select a Report  Once data sets are generated, select ‘Output Options’ from the navigation bar  “Expand All” or select the appropriate folder to find the relevant report i.e., Output Options > Dialysis Events > Numerators > CDC Defined Output > “Line Listing – Frequency of Dialysis Events”

Step 3 – ‘Run’ the Report  Press the “Run” button next to the report you want

Step 3 – ‘Run’ the Report  The report will open in a separate window ALLOW POP-UPS!

OPTIONAL REPORT MODIFICATIONS (OPTIONAL)

Modifying Reports is Optional  Some suggestions to modify reports:  Restrict the report to a certain time period  Choose what variables appear and how they are organized in reports you run  Click the ‘Modify’ button next to the template you’d like to change

The Modify Screen The modify screen has several components that users can experiment with. A couple of easy modification options: 1.Filter by date 2.Specify variables that appear and adjusting the order in which they appear in the output.

Modifying Reports: Filtering by Date  Filter by time period  Try “eventDate” for a report that includes all dialysis events that occurred during a specific time interval  Different reports have differing filtering options

Modifying Reports: Filtering by Date  Filter by “eventDate”  Use MM/DD/YYYY date format  In the example below, the report will include all dialysis events that occurred on or between October 1, 2011 and October 31, 2011

Modifying Reports: Filtering by Date  Common date variable is SummaryYM  SummaryYM = Summary of data by Year and Month  Enter date(s) in MM/YYYY format E.g., the report will include data from Oct 1, 2013 to Dec 31, 2013

Modifying Reports – Filtering by Date  Another common date variable is SummaryYQ  SummaryYQ = Summary of data by Year and Quarter  Enter date(s) in YYYYQ# format (e.g. 2014Q1 = the 1 st quarter of 2014)  E.g., the report will include data from the 3 rd quarter of 2013 through the 2 nd quarter of 2014 (or July 2013 – June 2014)

Modifying Reports – Changing Variable Display and Output Order  The bottom of the modify screen allows you to specify what data will be displayed in the output and the order in which they will appear  Click the link next to the “Modify Variables to Display by Clicking” option. Note: Modification and display options vary by report

Modifying Reports – Changing Variable Display and Output Order

 To modify which variables are included in the report output, select a variable from the “Available variables” column and press the to move it to the “Selected variables” column.

Modifying Reports – Changing Variable Display and Output Order  Click the ‘Up’ and ‘Down’ buttons to change the display order in the “Selected variables” column  Click ‘Save’ and run the report when done

Modifying Reports – Changing Variable Display and Output Order The report will pop-up in a new dialogue box with the variable added in the position you assigned.

READING NHSN REPORTS

Understanding Basic NHSN Terminology  In-Plan vs. Off-Plan Reporting: Selecting the checkbox next to a surveillance option on the “Monthly Reporting Plan” indicates the facility will report data in-plan, according the corresponding NHSN protocol  Numerator = number of dialysis events  Information from “Dialysis Event” form  Numerator = 0 if the “Report No Events” box is checked on the “Denominators for Outpatient Dialysis” form  The top number in a rate calculation  Denominator = number of at-risk patient-months  Information from “Denominators for Outpatient Dialysis” form  The bottom number in a rate calculation

HOW TO READ NHSN REPORTS Example 1: CMS ESRD QIP Line Listing

Line Listing - CMS ESRD QIP Rule Report  Aim of the report is to show if minimum QIP NHSN reporting requirements have been met for a given month  Have data been reported in-plan?  Has a complete numerator been reported?  Has a complete denominator been reported?

Line Listing - CMS ESRD QIP Rule Report  Generate Data Sets  Locate the report under Output Options in the “CMS Reports” folder  Click “Run”

Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL2014M01YNYN Dialysis Test FacilityOPDIAL2014M02YYYY Dialysis Test FacilityOPDIAL2014M03NNNN Dialysis Test FacilityOPDIAL2014M04YYNN Example: Line Listing - CMS ESRD QIP Rule Data are reported to CMS by CCN. Verify that a CCN is listed and that it is correct.  CCN = CMS Certification Number  CCN can be added or edited on the Facility Info screen

Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL2014M01YNYN Dialysis Test FacilityOPDIAL2014M02YYYY Dialysis Test FacilityOPDIAL2014M03NNNN Dialysis Test FacilityOPDIAL2014M04YYNN Example: Line Listing - CMS ESRD QIP Rule  Summary Year/Month column indicates which month is represented by the row  Looking down the column, you can determine if consecutive months are represented

Example: Line Listing - CMS ESRD QIP Rule Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL2014M01YNYN Dialysis Test FacilityOPDIAL2014M02YYYY Dialysis Test FacilityOPDIAL2014M03NNNN Dialysis Test FacilityOPDIAL2014M04YYNN Y = Reporting Plan saved with “DE” selected for the month  Dialysis Events will be reported “in-plan”

Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL2014M01YNYN Dialysis Test FacilityOPDIAL2014M02YYYY Dialysis Test FacilityOPDIAL2014M03NNNN Dialysis Test FacilityOPDIAL2014M04YYNN Example: Line Listing - CMS ESRD QIP Rule  Did the facility report the number of at-risk patient- months (denominator) in January and February 2014? Y = Denominators for Outpatient Dialysis form was completed for the month

Example: Line Listing - CMS ESRD QIP Rule  Report the number of highest risk vascular access types  Check off the “Report No Events” boxes on the Denominator Form as necessary.

Reporting a Numerator  Each month, each dialysis event type needs to be accounted for.  This can be done by : 1.Reporting an event via the Dialysis Event form, or… 2.Checking off the “report no events” box for specific event types on the “Denominators for Outpatient Dialysis” form to confirm that no events (i.e., zero events) of that type occurred during the month.  Numerator = 0 when the “report no events” checkbox is checked Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL2014M01YNYN Dialysis Test FacilityOPDIAL2014M02YYYY Dialysis Test FacilityOPDIAL2014M03NNNN Dialysis Test FacilityOPDIAL2014M04YYNN

Adding an Event Can Satisfy the Numerator Requirement  Report dialysis events using the “Dialysis Event” form  Complete all required fields and click “Save”

Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL 2014M01YNYN Dialysis Test FacilityOPDIAL 2014M02YYYY Dialysis Test FacilityOPDIAL 2014M03YNNN Dialysis Test FacilityOPDIAL 2014M04YYNN “Reporting No Events” Can Satisfy the Numerator Requirement The “Report No Events” checkboxes are found on the Denominators Form. Y = No events reported, report no events boxes appropriately checked N = No events reported, report no events boxes have NOT been appropriately checked

Example of Reporting No Events: No IV Antimicrobial Starts in January and February 2012 January 2012: -Numerator Reported = “N – NO” because no IV antimicrobial start events were reported, AND the “Report No Events: No IV Antimicrobial Start Events” was NOT checked off on the Denominator form. February 2012: -Numerator Reported = “Y – YES” because no IV antimicrobial start events were reported, AND the “Report No Events: No IV Antimicrobial Start Events” WAS checked off on the Denominator form. Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month 2014M01YNYN 2014M02YYYY

Org ID CMS Certification Number Facility NameLocation Summary Year/ Month DE on Reporting Plan Dialysis Event Numerator Reported Dialysis Event Denominator Reported Criteria Met this Month Dialysis Test FacilityOPDIAL 2014M01YNYN Dialysis Test FacilityOPDIAL 2014M02YYYY Dialysis Test FacilityOPDIAL 2014M03YNNN Dialysis Test FacilityOPDIAL 2014M04YYNN Example: Line Listing - CMS ESRD QIP Rule Verify NHSN reporting requirements are met for the month, reflected by a “Y” (Yes) in each field  To meet CMS criteria, all other Yes/No fields in the same row must be “Y”  “N” indicates that action is needed

ESRD QIP Resources CMS-QIP-NHSN-report.pdf

HOW TO READ NHSN REPORTS Example 2: Bloodstream Infection (BSI) Rate Table

Components of a Rate  Numerator = number of dialysis events  Information from “Dialysis Event” form  Numerator = 0 if the “Report No Events” box is checked on the Denominators for Outpatient Dialysis form  Denominator = number of at-risk patient-months  Information from “Denominators for Outpatient Dialysis” form  Rate (per 100 patient-months)  NHSN dialysis event rates are calculated per 100 patient-months  Typically rates are stratified by vascular access type = Dialysis Events (numerator) Patient-Months (denominator) x 100

Most Dialysis Rate Tables are interpreted similarly. Aggregate Rates are provided for comparison for the following Rate Table reports:  Rate Table – IV Antimicrobial Start Data  Rate Table – IV Vancomycin Start Data  Rate Table – Bloodstream Infection Data  Rate Table – Access Related Bloodstream Infection Percent Adherence measurements are provided for the following Rate Table reports:  Rate Table for Hand Hygiene Adherence  Rate Table – All Practice Adherence (CLIP)  Rate Table – Flu Vaccine Adherence  Rate Table – Flu Vaccine Declination  In 2015, additional reports will be added for newly introduced surveillance options.

Example: Bloodstream Infection Data Rate Table  Aim of the report is to provide the rate of bloodstream infections over time for the facility and provide NHSN aggregate data for comparison  Bloodstream Infection  Any positive blood culture Note: This example has been modified to specify a distinct time interval: 2 nd quarter of 2012

Example: Bloodstream Infection Data Rate Table  Generate data sets  Locate the report under Output Options: 1.‘Dialysis Events’ folder 2.‘Rates’ folder 3.‘CDC Defined Output’ folder Rate Table – Bloodstream Infection Data  Click “Run”

Location Access Type Summary Yr/Qtr Months Number Bloodstream Infections Patient- Months Bloodstream Infection Rate/100 patient- months NHSN Bloodstream Infection Pooled Mean Rate/100 patient-months Incidence Density p-value Incidence Density Percentile All2012Q Fistula2012Q Graft2012Q Other Access2012Q Tunneled2012Q Nontunneled2012Q Any CVC2012Q Example: Bloodstream Infection Data Rate Table Non-shaded (white) area is the facility data. Shaded (yellow) area is aggregate data from all of NHSN. Use this information to compare each facility to the rest of NHSN.

Location Access Type Summary Yr/Qtr Months Number Bloodstream Infections Patient- Months Bloodstream Infection Rate/100 patient- months NHSN Bloodstream Infection Pooled Mean Rate/100 patient- months Incidence Density p-value Incidence Density Percentile All2012Q Fistula2012Q Graft2012Q Other Access2012Q Tunneled2012Q Nontunneled2012Q Any CVC2012Q Example: Bloodstream Infection Data Rate Table NumeratorDenominatorFacility Rate = 1 45 x 100 Rate = BSI/100 patient-months

Location Access Type Summary Yr/Qtr Months Number Bloodstream Infections Patient- Months Bloodstream Infection Rate/100 patient- months NHSN Bloodstream Infection Pooled Mean Rate/100 patient- months Incidence Density p-value Incidence Density Percentile All2012Q Fistula2012Q Graft2012Q Other Access2012Q Tunneled2012Q Nontunneled2012Q Any CVC2012Q Example: Bloodstream Infection Data Rate Table This column shows the mean or average RATE (per 100 patient-months) for all dialysis facilities reporting to NHSN

Location Access Type Summary Yr/Qtr Months Number Bloodstream Infections Patient- Months Bloodstream Infection Rate/100 patient- months NHSN Bloodstream Infection Pooled Mean Rate/100 patient- months Incidence Density p-value Incidence Density Percentile All2012Q Fistula2012Q Graft2012Q Other Access2012Q Tunneled2012Q Nontunneled2012Q Any CVC2012Q Example: Bloodstream Infection Data Rate Table NHSN Aggregate Rate Facility Rate

Location Access Type Summary Yr/Qtr Months Number Bloodstream Infections Patient- Months Bloodstream Infection Rate/100 patient- months NHSN Bloodstream Infection Pooled Mean Rate/100 patient-months Incidence Density p-value Incidence Density Percentile All2012Q Fistula2012Q Graft2012Q Other Access2012Q Tunneled2012Q Nontunneled2012Q Any CVC2012Q Example: Bloodstream Infection Data Rate Table P-value and Percentile are provided to assist with interpretation of rate comparison  Typically, a p-value of <0.05 is considered a statistically significant difference between rates  The lower the percentile, the better the facility is performing relative to the others in NHSN

Comparing Rates Using Percentiles  The percentile indicates how a facility ranks for the event among all NHSN facilities  A lower the percentile indicates a lower rate of infection. 46% of facilities reported lower BSI rates among patients with tunneled central lines than facility

Analysis: Rate Table Interpretation Examples  Among patients with tunneled central lines in each quarter, how would you interpret this facility’s rates? Access Type Summary Yr/QtrMonths Number Bloodstream Infections Patient- months BSI Rate/100 patient- months NHSN BSI Pooled Mean Rate/100 patient- months Incidence Density p-value Incidence Density Percentile Tunneled 2014Q Tunneled 2014Q Tunneled 2014Q

Quarter Number Bloodstream InfectionsPatient-months BSI Rate/100 patient-months NHSN BSI Pooled Mean Rate/100 patient-months Incidence Density p-value Incidence Density Percentile Quarter 1, facility rate is 12.50, NHSN rate is 3.24  Percentile (96) is high  Conclusion: facility has a higher than average BSI rate 2. Quarter 2, facility rate is 3.33, NHSN rate is 3.24  Percentile (58) is medium  Conclusion: facility has an average BSI rate 3. Quarter 3, facility rate is zero, NHSN rate is 3.24  Percentile is (10) low  Conclusion: facility has a lower than average BSI rate

BSI Resources /dialysis/BSI-cheatsheet.pdf  Guidance for other reports is also available on the NHSN Dialysis homepage.

Interpreting Data  Please keep in mind that data quality is essential for meaningful rates, comparisons, and conclusions  Verify: Is the Protocol being followed correctly?  Verify: Are all Dialysis Events being captured?  Verify: Has all event information been reported to NHSN?  Use all the information available to you, including percentile rank, to interpret your rates  Combine data interpretation with investigative work in the unit and common sense  For evaluation, examining data over longer timeframes is more informative  e.g., draw conclusions based on ≥ 1 data quarter, versus a single month of data

Data Quality and Quantity  When reviewing your facility’s rates, remember the importance of data quality:  High rates may = high event occurrence OR over-reporting  Low rates may = low event occurrence OR under-reporting  NHSN rates could increase if facilities improve the accuracy and completeness of reporting  And data quantity:  Rates may fluctuate over short periods of time  Assessing rates over greater time intervals can increase confidence in the values

Review Your Data  Monthly to:  Ensure all data have been accurately reported  Quarterly to:  Detect problems in your facility  Provide feedback to your staff  Get staff engaged in quality improvement  Prepare for CMS quarterly reporting deadlines  Better understand your facility’s performance by comparing your facility’s rates against NHSN aggregate rates

Resources for Reviewing the Data  The 3 Steps to Review DE Surveillance is a great tool for ensuring that your data are accurate and complete! 3-Steps-to-Review-DE-Data-2014.pdf

WHAT LIES AHEAD FOR NHSN Looking to the future…

NHSN Analysis: Goals for the Future  Update NHSN aggregate data  Important to have improved data quality  Continue streamlining Analysis interface  Updating the Analysis tree view to reflect new options in an organized fashion  Introduce new reports to track surveillance and facility participation  Healthcare Personnel Flu Vaccination  5 Prevention Process Measures Increase the use of Analysis tools by all NHSN users!

Summary—Use NHSN Analysis to Your Advantage  The launch of the Dialysis Component separated Dialysis analytical tools from all other tools  The component is streamlined  Analysis is easier to navigate  Creating and Running Reports  3 step process Generate data sets Modify the report if necessary Run the report  Suggested Report Modifications 1.Filter by date 2.Choose variables and organize them to suit your reporting needs

Summary—Use NHSN Analysis to Your Advantage  Understand reports to see your facility’s performance  The CMS ESRD QIP report is a great tool to help users ensure that they have met minimum CMS reporting requirements Did the facility report in-plan? Was a complete numerator reported? Was a complete denominator reported?  The BSI Data Rate Table (and other rate tables) can inform facility performance and improvement How does the facility’s BSI rate compare to the NHSN rate?

Summary—Use NHSN Analysis to Your Advantage  Review and interpret your data often  Reviewing the data can serve as a learning opportunity  By reviewing the data regularly, facilities can demonstrate progress or need for improvement to frontline staff  Data quality is of utmost importance

For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA Telephone, CDC-INFO ( )/TTY: Web: Thank you!