Presentation is loading. Please wait.

Presentation is loading. Please wait.

What is the Best Way to Track Surgical Complications? Jacques X. Zhang, B.Sc. Diana Song, MD Julie Bedford, RN, MSN Douglas J. Courtemanche, MD, MS, FRCSC.

Similar presentations


Presentation on theme: "What is the Best Way to Track Surgical Complications? Jacques X. Zhang, B.Sc. Diana Song, MD Julie Bedford, RN, MSN Douglas J. Courtemanche, MD, MS, FRCSC."— Presentation transcript:

1 What is the Best Way to Track Surgical Complications? Jacques X. Zhang, B.Sc. Diana Song, MD Julie Bedford, RN, MSN Douglas J. Courtemanche, MD, MS, FRCSC Marija Bucevska, MD Jugpal S. Arneja, MD, MBA, FAAP, FACS, FRCSC Comparing ACS NSQIP versus Traditional M&M Rounds

2 Conflicts of Interests Dr. Courtemanche is a director and shareholder with Resilience Software, which made T-Res T-Res is used to collect data for M&M Rounds at UBC and the database provided some of the data for the research 2

3 What is M&M and NSQIP-P? 3 M&M roundsACS NSQIP Pediatrics What isit?Rounds to discussmedical complications and deaths Discussions are protected under Section 51 of the Evidence Act. 1 - Standardized - Tracks post-op complications - Subset of surgical patients - Allows risk-adjusted benchmarking between hospitals. 2-4 Background 1. bclaws.ca 2. Khuri SF. 2005;138(5):837-43. 3. Khuri SF, Daley J, Henderson W, et al. 1998;228(October):491-507. 4. Ingraham AM, Richards KE, Hall BL, Ko CY. Adv Surg. 2010;44(1):251-267.. How does it work?Complications are entered retrospectively into a database. Some discussed at rounds. Next Slide…

4 NSQIP-P definitions are very strict 4 How NSQIP-P Works Khuri SF. 2005;138(5):837-43. Khuri SF, Daley J, Henderson W, et al. 1998;228(October):491-507. Ingraham AM, Richards KE, Hall BL, Ko CY. Adv Surg. 2010;44(1):251-267.. Background

5 5 M&M roundsACS NSQIP Pediatrics 1-3 Rounds are biweekly or quarterly as organized by the department Risk-adjusted feedback is provided on a semi-annual basis in the form of a odds ratio for the hospital/department Used for decadesNSQIP P pilot in 2008, BCCH joined in 2011 Surgeon is reviewerSurgical Clinical Reviewer (NSQIP) is reviewer Looks at all casesLooks at a subset of all cases through rigorous sampling Cases reviewed quarterly (90 days)Tracks post-op occurrences up to a set 30 days Complication reported only if surgeon awareActive tracking of patient 30 days post-op All relevant complications trackedOnly NSQIP complications are tracked Recall and reporting biasFollows strict NSQIP guidelines and definitions Not risk adjustedRisk-adjusted, allows benchmarking Low inter-rater reliabilityHigh inter-rater reliability 4 Low cost and labourHigh cost ($200,000/yr at BCCH) +labour (2 SCRs/ 2 surgeons champions) Side-by-Side Comparison 1. Khuri SF. 2005;138(5):837-43. 2. Khuri SF, Daley J, Henderson W, et al. 1998;228(October):491-507. 3. Ingraham AM, Richards KE, Hall BL, Ko CY. Adv Surg. 2010;44(1):251-267.. 4. Shiloach M, Frencher SK, Steeger JE, et al. J Am Coll Surg. 2010;210(1):6-16 Background

6 Why this paper? Limited studies on: – Pediatric NSQIP in general – Pediatric M&M vs NSQIP – Plastic surgery specific data PURPOSE: – To determine the best way to track pediatric plastic surgeries by comparing complications tracked by NSQIP-P vs traditional M&M rounds, in 2012-2013 6 Purpose

7 Methods For the first 2 full years (2012 and 2013) of NSQIP data: – Extract complications (numerators) for both M&M and NSQIP – Extract total cases (denominators) NSQIP is a subset of M&M – Go through op logs and classify M&M data into “NSQIP categories”  eligible vs ineligible Stratify data into major and minor complications 7

8 NSQIP and M&M have similar rates 8 … However, M&M data includes cases that NSQIP potentially excludes … Need to remove the M&M data that is NSQIP ineligible NS - Not statistically significant (2-sample Z test) Results

9 9

10 NSQIP vs M&M: adjusted rates 10 NS - Not statistically significant (2-sample Z test) Results

11 11 ~50% of all cases are not tracked by NSQIP! Half of all cases are not tracked by NSQIP! n= 648 or (51.4% of all cases) n=613 or (48.6% or all cases) Results

12 What is the concordance and discordance rate? 12 Concordance rate for M&M Eligible = 13/37 = 35.1% Concordance rate for NSQIP = 13/40 = 32.5% Discordance rate for M&M = 64/77 = 83.1% (MM+, NSQIP-) Discordance rate for NSQIP = 27/40 = 67.5% (MM-, NSQIP+) Results

13 What is the discordance rate between NSQIP and M&M? 13 Discordance rate for M&M = 64/77 = 83.1% Results

14 What is the discordance rate between NSQIP and M&M? 14 Discordance rate for NSQIP = 27/40 = 67.5% Results

15 NSQIP and M&M track different complications 15 * = NSQIP tracked complication Results

16 What about the severity of the complication? Stratified into major vs. minor complication – Major complication that leads to: Death Readmission Re-operation – Minor complication Anything else – Results: we find ~50/50 split in both systems 85% “Major” for matched complications (MM+ NSQIP+) 16 Results

17 7 major complications (2012-2013) missed by M&M 17 Results

18 Best way to track pediatric plastics complications? A combination of both M&M rounds and NSQIP… NSQIP and M&M have similar occurrence rates but each has their benefits over the other. NSQIP provides the strict rate of morbidity whereas M&M provides the description. 18 Discussion MOH Hospital Department Division Surgeons NSQIP M&M Feedback and Purpose:

19 19 M&M roundsACS NSQIP Pediatrics 1-3 Rounds are biweekly or quarterly as organized by the department Risk-adjusted feedback is provided on a semi-annual basis in the form of a odds ratio for the hospital/department Used for decadesNSQIP P pilot in 2008, BCCH joined in 2011 Surgeon is reviewerSCR (nurse) is reviewer Looks at all casesLooks at a subset of all cases through rigorous sampling Cases reviewed quarterly (90 days)Tracks post-op occurrences up to a set 30 days Complication reported only if surgeon awareActive tracking of patient 30 days post-op All relevant complications trackedOnly NSQIP complications are tracked Recall and reporting biasFollows strict NSQIP guidelines and definitions Not risk adjustedRisk-adjusted, allows benchmarking Low inter-rater reliabilityHigh inter-rater reliability 4 Low cost and labourHigh cost ($200,000/yr at BCCH) +labour (2 SCRs/ 2 surgeons champions) Side-by-Side Comparison 1. Khuri SF. 2005;138(5):837-43. 2. Khuri SF, Daley J, Henderson W, et al. 1998;228(October):491-507. 3. Ingraham AM, Richards KE, Hall BL, Ko CY. Adv Surg. 2010;44(1):251-267.. 4. Shiloach M, Frencher SK, Steeger JE, et al. J Am Coll Surg. 2010;210(1):6-16 Conclusions

20 Conclusion 1.NSQIP misses 50% of cases 2.Not all complications are tracked 3.NSQIP overall rate is accurate compared to M&M 4.NSQIP methods are strictly defined and rigorous while M&M is subject to recall and reporting bias 20 We suggest expansion of NSQIP-P to include currently excluded cases and an extension of the NSQIP-P study interval.

21 Limitations Limited to the plastic surgery department at a single institution, and only for a 2-year study period Rates for the M&M complications might be artificially under-reported due to the errors in M&M data Low number of complications combined with a large variety of plastic procedures may increase the variability in rate 21

22 Acknowledgements Dr. Arneja Dr. Courtemanche Dr. Diana Song Marija Bucevska Julie Bedford The plastics team

23 ADDITIONAL SLIDES What is the Best Way to Track Plastic Surgery Outcomes: Comparing ACS NSQIP vs M&M rounds 23

24 QA timeline 18941907191119351940s19911994199920012004200820102011 24 Codman develops anesthesia record Codman helps create the ACS End-Result Concept into practice Anesthesia Mortality Committee Becomes the Anesthesia Study Commission (Prelude to M&Ms) 2000: T-res NSQIP Peds at BCCH NSQIP Peds to 40 hospitals NSQIP Peds initiation (4 hospitals) NSQIP becomes open subscription program VA, ACS Patient Safety in Surgery Study, 14 large non-VA academic hospitals VA study PSI at 3 academic non-VA surgical dpt NSQIP established in all 132 major VA surgical centers NSQIP inception in 44 VA hospitals in the NVASRS

25 However, complication rates are still similar 25 NS - Not statistically significant (2-sample Z test) Results

26 NSQIP and M&M obtain similar severity of complications 26 NS - Not statistically significant (2-sample Z test) Results

27 Matched cases are mostly major complications 27 Results

28 NSQIP and M&M serve different purposes… NSQIP-P returns a hospital and department odds ratio (hospital wide view) Whereas M&M returns a divisional and surgeon level rate (patient level view) 28 MOH Hospital Department Division Surgeons NSQIP M&M Discussion Theoretically:Reality:Feedback and Purpose: All cases M&M NSQIP

29 Conclusions Similar rates, however: 1.NSQIP misses 50% of all cases. 2.NSQIP morbidity rate is confirmed against M&M, even for NSQIP ineligible cases, as well as severity of complications. Validation of the program. 3.Differences in definitions and subjectivity in M&M led to low concordance rate, with NSQIP being a more rigorous system. 4.We suggest expansion of ACS-NSQIP to include currently excluded cases and an extension of the ACS-NSQIP study interval. 29


Download ppt "What is the Best Way to Track Surgical Complications? Jacques X. Zhang, B.Sc. Diana Song, MD Julie Bedford, RN, MSN Douglas J. Courtemanche, MD, MS, FRCSC."

Similar presentations


Ads by Google