Validation of Central-Line Associated Bloodstream Infection (CLABSI) Data Reporting, Oregon, 2009 Zintars Beldavs, MS Manager Healthcare-Associated Infections Program Acute and Communicable Disease Prevention Section Office of Disease Prevention and Epidemiology Public Health Division Oregon Health Authority October 13, 2011

Oregon HAI OPHD OHPR OPSC NHSN Validation NHSN Reporting Multi-hospital Collaborative CLABSI SSI HAI EIP Projects Candidemia Surveillance HAI Point Prevalence Denominator Simplification MDRO Surveillance C. difficile Surveillance

Validation for Accurate Data Research indicates surveillance definitions applied differently by different IPs Demonstrated by poor inter-rater reliability (agreement between different people reporting on same case): kappas of .30 to .58 Previous validation studies: potentially more than half of cases not reported If the rationale behind mandatory reporting is to enable the consumer to choose the best hospital, we have the obligation to provide reliable, generalizable data that are internally and externally valid. Oregon hospitals:Lin MY, et al., SHEA 2010. % of 661 positive blood cultures considered CLABSIs: Infection Preventionists: 21% Standardized review: 35% Electronic algorithm: 57 Kappa values 0.43, 0.30, 0.58 Mayer J, et al., SHEA 2010: 120 VAMC records, 18 IPs compared to electronic algorithm Overall kappa 0.42, range 0.30–0.50 Individual IPs varied up to 2-fold in CLABSI assessments

Central Line-Associated Bloodstream Infection (CLABSI) Attributable mortality ~18% 14,000 deaths/ year in ICU patients Estimated cost per episode $3,700 to $29,000 Prolong hospitalization by mean of 7 days Preventable through hand hygiene, barrier precautions, skin antisepsis, catheter site selection Chosen by our HAI advisory committee based on its morbidity, mortality, and costs --- and its preventability A bloodstream infection can occur when pathogens travel down a central catheter - that is, a catheter terminating at or near the heart or one of the great vessels - and enter the blood. Pittet D, Tarara D, Wenzel RP. Nosocomial bloodstream infection in critically ill patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1994;271:1598-1601. Soufir L et al. Infect Control Hosp Epidemiol 1999 Jun;20(6):396-401.

Objectives Evaluate quality of reported data Assess under- and over-reporting Gauge the reliability and consistency of surveillance case definitions Provide feedback and guidance to facilities on surveillance case definitions and reporting methods

Methods Study period: calendar year 2009 Included: 44 acute care hospitals 28 with <50 beds 10 with >200 beds Median central line days 210, range 4-4956 OPHD validation team: Research analyst Epidemiologist EIS Officer/Physician 3 public health nurses Hospitals in OR One major teaching hosp, 25 critical access hospitals Bed size: As you can imagine, there is tremendous variation in the levels of resources available, as well as the patient populations being served, the kinds of procedures being done… Map: Oregon Association of Hospitals and Healthcare Systems, oahhs.org

Methods March 2010 - April 2011: on-site hospital visit for chart review Retrospective record review by 1-4 reviewers At 37 hospitals: all ICU patients blood culture(+) At 7 largest hospitals: all reported CLABSI plus random sample of 60 patients with ICU blood-culture(+) not reported as CLABSI Validators blinded as to whether cases reported as CLABSI

Methods After visit, all cases with discordant CLABSI determinations (suspected false positives or false negatives) adjudicated by phone with hospital staff Participants Hospital IP staff Hospital physician OPHD validators OPHD physician Review of all findings for final CLABSI determination If no consensus reached, case referred to CDC staff This step unique to Oregon’s validation project (not previously attempted by other states)

Results 1199 medical records reviewed 549 at 7 highest-volume facilities (records sampled) 722 at small- and medium- volume facilities 817 record reviews included in final analysis 382 records censored as could not meet ICU CLABSI case definition due to timing rules (positive blood cultures were obtained prior admit or > 48 hours after discharge from ICU)

CLABSI rate before and after validation Validation increased the statewide ICU CLABSI rate from 1.21 (95% CI: 0.95–1.51) to 1.54 (95% CI: 1.25–1.88) CLABSI per 1,000 central-line days Change after validation in CLABSI rate No. hospitals % Rate decreased 0.70 1 2 No change 33a 75 0.01–0.50 higher 5 0.51–1.00 higher >1.00 higher 6b 14 Total 44 a 23/33 had no CLABSI identified either before or after the validation. b 3/6 had no CLABSI before the validation.

Results Validation outcome, unadjusted Validation findings (after chart review and adjudication) Originally reported to NHSN? Yes No CLABSI 70 (True Positives) 16 (False Negatives) Non-CLABS I 6 (False Positives) 712 (True Negatives)

Results Estimated number of CLABSI adjusted for sampling fraction Estimated CLABSIs among All ICU Patients with Positive Blood Cultures, by Initial Hospital Report — Oregon, 2009 CLABSI Final determination Present Absent Total Hospital report 70 (TP) 6 (FP) 76 27a (FN) 1089a (TN) 1116 97 1095 1192 Sensitivity = 0.72 (95% CI: 0.62–0.81); Specificity = 0.99 (95% CI: 0.99–1.00). Positive predictive value = 0.92 (95% CI: 0.83–0.77). Negative predictive value = 0.98 (95% CI: 0.96–0.98). Prevalence = 0.08 (95% CI: 0.07–0.10).

Importance of Inter-Agency Follow-up Discussion Of 27 unreported cases initially identified as possible CLABSI by OPHD, 16 (59%) actual CLABSI Sensitivity of reporting: 72% based on follow-up adjudication vs. 60% based on OPHD review alone (P= 0.07), closer to some previous validation efforts

Reasons for discrepancies 6 CLABSI “just missed”: at some facilities, IP staff had changed since 2009 and current staff unaware of rationale for previous CLABSI reporting/ non-reporting decisions.

CLABSI Pathogens before and after validation Charts indicate primary organism associated with for each (~4 true pos cases had secondary and tertiary orgs). This is statewide aggregate data. “other” = 1 Moraxella (gram neg diplococcus),1 Stenotrophomonas, 1 Veillonella. All but 2 “other yeast” are non- albicans Candida spp. Before validation (n=76) After validation (n= 86)

Conclusions Validating hospital CLABSI reporting improves accuracy of hospital-based CLABSI surveillance Discussing discordant findings improves the quality of validation

Acknowledgments Questions? 971-673-1111 zintars.g.beldavs@state.or.us OPHD HAI program staff and others assisting Paul Cieslak – Public Health Physician Ann Thomas – Public Health Physician Margaret Cunningham – HAI Epidemiologist Diane Roy – HAI Administrative Assistant John Oh – EIS Officer Steve Moore – Public Health Nurse Jennifer Tujo – Infection Preventionist Valerie Ocampo – HAI Public Health Nurse Oregon Patient Safety Commission Office for Oregon Health Policy and Research Association of Professionals in Infection Control, Oregon-SW Washington Chapter Questions? 971-673-1111 zintars.g.beldavs@state.or.us