Download presentation
1
Are you good enough? Ryutaro Hirose MD
UCSF Transplant QI committee, Chair UCSF Dept of Surgery QI committee UCSF Clinical performance improvement committee, Chair UCSF Patient Safety Committee Former Chair, ASTS Standards and Quality committee
2
How do you know if you’re good?
Look in the mirror You ask people Pt satisfaction surveys – important for CMS…but correlate with quality?? CGCAHPS, HCAHPS National rankings You don’t make errors – pt safety You don’t make errors that hurt people or kill people Failure Mode Effect Analysis (FMEA) RCA Processes and Outcomes Benchmarks and comparators
3
Patients tell you that you are good?
Patients with highest satisfaction had increased mortality, worse outcomes Archives of Internal Medicine, Fenton et al 2012
4
Comparison of Four National Rating Systems
5
National Hospital Rankings
US News and World Report, HealthGrades, Leapfrog group, Consumer Reports Also CMS Hospital Compare NO hospital was ranked a high performer by all 4 rankings ONLY 10% of hospitals (844) ranked high performer by one ranking was ranked high performer by another Different rating methods, different focus, stresses different measures of quality
6
Quality Assessment and Performance Improvement
Institute of Medicine To Err is Human Released 2000 Focused attention on medical errors, preventable deaths Systems approach to patient safety Improving processes, not blaming individuals Individuals must be held accountable and held responsible Crossing the Quality Chasm
7
How do you judge quality? Team metric?
2014 – 2015 U of Kentucky Men’s Basketball Overall 34-0 #1 Seed in NCAA tournament National Championship?
8
Are you good enough? New England Patriots (2014-2015)
Super bowl championships 2001, 2003, 2004, 2014
9
Are you good enough Individual provider metrics are confused with team metrics Baseball – Pitchers are judged by W-L record More appropriate metrics ERA WHIP
10
Individual metrics Michael Jordan Tom Brady 6 time NBA champion
10 x scoring title, PPG: 30.2 (career) Tom Brady 4 Superbowl rings 63.5 completion %, 53,258 yrds, 392 TDs, 143 INTs
11
Effect of environment on quality
Domains of quality Patient safety/safe care Practice consistent with current medical knowledge Customization, ability to meet customer-specific values External forces to drive quality Regulatory/legislative pressures Economic and other incentives
12
Professional societies
Set norms, standards of practice Expectation that delivery of safe, high quality care is standard Promote culture of safety and improvement
13
Lapses in quality Misuse - Avoidable complications
Overuse – excess provision of service, not supported by evidence Underuse - failure to provide a service that would have provided a favorable outcome First addressed by pt safety initiatives Second and third addressed by evidence based practice.
14
Quality Assurance and Performance Improvement
QAPI programs Mandated by CMS Industry models PDSA (plan-do-study-act) Six Sigma (DMAIC) Define/Measure/Analyze/Improve/Control Lean Six Sigma LEAN – preserving value with less work Waste reduction, increase efficiency, improve work flow Production time/costs reduced
15
Cycle of continuous improvement Plan – Do – Study – Act
Identify goal/purpose, define metrics of success Do Implement components of plan Study Monitor outcomes, test validity ID areas for improvement Act Close cycle, adjust goal Change methods, reformulate theory
16
Six Sigma/Total Quality Management
Seeks to improve qulaity by identifying and removing errors and minimizing variability DMAIC Define, Measure, Analyze, Improve, Control Combined with lean manufacturing LEAN Six Sigma Address flow and waste
17
QAPI programs Mandated to have process AND outcome measures – for three phases Pre transplant Transplant Post transplant
18
How is quality measured at liver transplant centers
Mandated data collection and submission to UNOS Transplant Centers Pretransplant – Transplant Candidate Registration form Transplant – Transplant Recipient Registration form Post transplant – Transplant Followup forms Organ Procurement Organizations Deceased donor data Performance data
19
What do we do now? SRTR PSR’s Waitlist mortality Transplant rates 1 yr and 3 yr patient and graft survival Used by UNOS/OPTN to flag programs, used by CMS as well
20
Center specific results
Candidates ‘shot selection’ Referring physicians Nephs, hepatologists Selection committee/selection critieria Surgeon Donors Donor demographics Donor management Affects outcome Donor selection In general For specifc recipient Donor/recip interactions Matching Size Operative course Redo, blood loss Technical performance Anesthesia Post op management ICU Surgeons/medical ID
21
Variation in outcomes UNOS/MPSC CMS National coverage decisions e.g.
Liver if meets survival minimum (1-Yr: 77%, 2-yr: 60%) Two consecutive reporting periods of worse than expected outcomes O-E > 3.0; O/E ratio of events > 1.5, p<0.05 Results in review, possible SIA
25
Center specific results
Integrated signals We play a team sport Wait list mortality Referral patterns Who are referred to us Selection criteria/behavior Who are listed Wait list management PCPs, referring nephs, heps De-listing Workup Accurate documentation of comorbidities
26
Scientific Registry of Transplant Recipients
Current Contract is held by the Chronic Disease Research Group of the Minneapolis Medical Research Foundation (Used to be held by the U of Michigan group) Responsible for designing and carrying out rigorous scientific analyses of data and disseminating information to the transplant community – including transplant programs organ procurement organizations policy makers transplant professionals transplant recipients organ donors and donor families and the general public.
27
SRTR roles Policy development – evidence based
Collaborative efforts between the transplant community, the SRTR, and the OPTN. Policy-making is the OPTN's responsibility, the SRTR plays a critical role in policy development through ongoing data analyses designed to provide policy makers with the information necessary to make informed decisions LSAM modeling to predict changes in allocation policy, and effects on wait list mortality, transplant rates, overall outcomes
28
SRTR – Scientific Registry of Transplant Recipients
Risk stratified results Wait list mortality Transplant rates Post transplant 1 month, 1 year, 3 year Graft survival Patient survival CUSUM charts Real time analysis to trend events over time Signals potentially concerning event rates
29
How is the expected pt and graft survival derived?
Risk stratification model E.g 1 yr graft survival model Multiple factors Donor factors Recipient factors Interaction between donor and recipient
30
Model for 1 year graft survival
ABO compatible Cold ischemia time Local vs Regional Recip factors Diagnosis of recip PVT Previous surgery Alb, INR Functional status Race Cr/dialysis ICU/life support Donor age Donor ht Donor race Donor use of drugs (cocaine) Donor hx of cancer Hx of HTN CDC high risk Donor pressors/ddAVP Split vs whole
31
Liver – adult 1 yr pt and graft survival
Only one center had lower than expected 1 year patient survival on the last SRTR PSR Not a sensitive metric for quality
32
Risk stratification Too little? Risk averse behavior
Inhibits innovation Too much? Risky behavior Futile transplants Not best use of organs?
33
Should there be an absolute non risk stratified standard?
Observed/Expected ratios Limited/scarce resources Lower absolute rates of success Best use of limited resources? Argument that innovation is stifled
34
Quality control Available Statistical Techniques Basic techniques
Average mortality rate Rolling average (last 10 cases) Adjusted average mortality rates # of consecutive failures Cox proportional hazard models Adjusted for clinical characteristics Incorporates risk adjustment Statistical process control Continuous monitoring techniques (e.g. CUSUM)
35
CUSUM technique Statistical control charts developed to study industrial processes Designed to ‘signal’ if there is a deviation from accepted production standards CUmulative SUM – used in quality control in industry to trend events over real time CUSUM techniques recently being used in medicine and to track surgical outcomes
36
CUSUM – cumulative sum Cumulative Sum (CUSUM) is technique of monitoring outcomes Began as a method to monitor industrial processes Produces a graphical output that can be tracked over time. CUSUM monitoring utility in health care recognized in 1970s Limited by poor data collection Lack of risk adjustment techniques Recent innovations let to expanded interest Steiner et al. Incorporated risk adjustment Axelrod et al. Effective in multi-center assessment in a retrospective study Kalbfleisch and Biswas- now utilizes survival analysis rather than a binary (alive or dead) logistic analysis
37
CUSUM charts Graphical representation of outcomes for process
Can be risk-adjusted charts for important donor and recipient characteristics Plot outcomes over time to compare the results with expected outcomes based on a national model of mortality or graph failure 2 types: O - E charts and One-sided charts Trends in the plot line suggest improving or declining outcomes Once the trend line reaches a certain predefined level (one-sided charts) or exceeds a certain slope (O - E charts) the CUSUM signals
38
CUSUM chart Graphical representation of observed vs expected events over time Risk adjusted CUSUM chartgs One-sided CUSUM Control limit is set, defines signaling threshold Two-sided or O-E CUSUM
43
A kidney transplant center
44
CUSUM conclusions CUSUM charting provides a reliable, risk adjusted method of tracking outcomes of a clinical process Can be “tuned” to balance the need for sensitively to detect clinical failures with the requirement to limit the number of false positive signals Graphical output is easily interpretable with a minimal amount of training Providing outcomes are promptly reported the CUSUM can provide real time insight into TC outcomes
45
National Transplant QI program
Why? 1 yr graft and pt survival Gross outcomes Team outcomes Provider specific outcomes Need Benchmarks Identify outliers Identify areas for PI Identify best practices
46
Surgical complications
Identification of surgical complications is poor In most cases, done retrospectively By coders/billing personnel Poor risk adjustment Lack of relevant benchmarks Rate of SSI Rate of thrombosis Rate of biliary/ureteral complications readmissions
47
Provider specific results
Blood loss Donor and recip characteristics Need for reoperation/intervention Vascular complications Bile duct/Ureter complications Readmissions
48
TransQIP Accurate prospective data collection
Risk adjustment models to be developed and tested ACS/NSQIP tests inter-rater reliability and accuracy Establishment of national benchmarks, comparison across centers/providers Identification of best practices
49
The future of quality and reimbursement
Value based purchasing Expansion of PQRS program From 0.5% incentive to 2% penalty 3-8% penalty/incentives Maintenance of competency Quality assurance activities
50
Summary The environment is changing Quality will take center stage
Legislative/Regulatory environment Reimbursement/Financial disincentives and incentives Value = Quality/Cost
51
Future directions Patient safety National TransQIP program
Operative debriefings Preventable errors in the OR – retained FB, errors of omission in the OR National TransQIP program Collaboration between ASTS and ACS Relevant outcomes Benchmarks Identify areas for improvement, identify best practices We can always do better, refine processes, increase pt safety Patient satisfaction – modify Surgical Care CAHPS Improve access, eliminate disparities Composite Pre transplant Metric Waitlist mortality Offer acceptance rate Transplant rate (geographically adjusted)
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.