Presentation is loading. Please wait.

Presentation is loading. Please wait.

Quality Improvement Pitfalls and Opportunities

Similar presentations

Presentation on theme: "Quality Improvement Pitfalls and Opportunities"— Presentation transcript:

1 Quality Improvement Pitfalls and Opportunities
James Hallenbeck, MD ACOS/Extended Care VA Palo Alto HCS

2 Objectives Be able to……. Compare top-down and grassroots strategies for quality improvement in terms of advantages and disadvantages Discuss different uses of quality monitors Comparisons across groups Quality improvement Better respond to queries regarding monitors, especially performance measures, especially MDS

3 How this might be useful to you…
Self defense - protecting you and your program when data does not accurately reflect the quality of care you deliver Promote better use of monitors (MDS and others) for quality improvement

4 Purposes of Quality Improvement Monitors
Comparisons VISNs, facilities, wards, clinicians Stimulus for quality improvement efforts Benchmarking and ongoing evaluation of quality improvement efforts

5 What Makes a Good Monitor? (Top-down or grassroots)
Desired outcome strongly linked to what is measured (i.e. minimally influenced by factors outside the health system’s control) System change, informed by monitor, would likely result in better outcomes Accuracy Minimal false negatives & false positives

Advantages: Disadvantages: Enables comparisons across large systems Establishes organizational priorities Buy-in from top management If successful, results in broad, measurable improvement Blind to local issues By attending to certain priorities, others may be ignored Lack of buy-in from front-line staff Need large numbers Disempowerment at local levels

7 What Makes a Good Top-Down Monitor?
Simple, discrete data Counts of things Minimal variation in local systems and populations Comparing apples to apples Large data numbers – in numerator and denominator Data gathering, measurement and comparison automatic Encourages GOOD clinical practice (not just designed to identify bad practice)

8 Examples Good Top-Down Monitors
Flu vaccine administration Appointment waiting time

9 QI Grassroots Disadvantages Advantages
Comparing outcomes across systems difficult Potential lack of buy-in from top management Harder to disseminate best practices Politically, invisible Tailored to local circumstances Better buy-in from front-line staff Can improve morale through empowerment

10 Examples Grassroots QI
Use of out-of-hospital DNR forms Addressing options of autopsy/organ donation

11 Apples to Apples Problem…
Case-mix issues Severity of illness (Problematic: pressure ulcers, weight loss, dehydration in Hospice) Gender (Problematic: UTI rates) Depression/atypical neuroleptics (Problematic: geropsych units, hospice)

12 Vertical Hierarchy Problem
What is “true” at one level of a vertical hierarchy is not necessarily “true” at another (higher or lower) Example: Newtonian physics versus quantum mechanics 30,000 feet view: reveals certain patterns, otherwise invisible, but obscures other


14 Vertical Hierarchy Nation VISN 1 VISN 2 VISN 3 Facility 2 Facility 1
Division/Ward Division/Ward Clinician + Patient Clinician + Patient Clinician + Patient

15 Connection Between What is Measured and Desired Outcome Problem
Many outcomes (good and bad) minimally dependent on what we as clinicians do Examples: UTI rates Fall rates Important to tease out those aspects of the outcome that are dependent on the healthcare system. Ideally, these aspects should be measured and worked on.

16 Outcomes Good Bad Ugly Environment Luck Healthcare System Patient
Variables Healthcare System Clinician Patient Outcomes Good Bad Ugly

17 Example: Falls Falls related to transfers/ambulation---18
1 month data: 47 falls Falls related to transfers/ambulation---18 Falls related to toileting—13 Falls related to reaching/picking/bending—7 Claimed did not fall/eased self to floor--5 Slipped/tripped-2 Falls related to medical condition---2 Falls related to inappropriate footwear-2 Can you tell from this data how these falls might be related to the healthcare system?

18 Numerator/Denominator Problem
Percentages meaningless in isolation (numerator/denominator not given) Small numbers – 1 – 10 always suspicious Numerator problem: rare events Denominator problem: Restricted population based on: Geography (single ward) Time interval (monthly tracking)

19 Oct ‘04 - Jun ‘05 Total FALLS REPORT

20 Coin-Toss QI 10 Facilities toss a coin 10 times
Heads is Good, Tails is Bad True chance (within infinite tosses) = 50% Let’s compare facilities to “national” data

21 Of 100 tosses 56/100 Heads = “National” average
Coin Toss “Problem” Facilities “Exceptional” Facilities Of 100 tosses 56/100 Heads = “National” average

22 Implications Unadjusted percentages create a bias in which small facilities/programs with small numbers to report for a given indicator are more likely to be classified at extremes (failing or exceptional)

23 Data Validity Problem Ideal: Problematic: Automated
Unambiguous results Problematic: Multiple people/services responsible for data entry Results dependent on subjective judgments Questionable inclusion/exclusion criteria

24 MDS Examples Dehydration monitor: Little or no activity monitor
Bedfast Status monitor

25 Blue = sub acute and hospice wards, Red = national


27 Dehydration So, What’s the Problem Here?

28 MDS Quality Indicators: What does the literature show?
MDS tends to under-report problems Example: depression: Differences in rates often reflect either differences in reporting or case mix – NOT quality Example: High/Low Pressure Ulcer NH did not differ significantly in care processes (Bates-Jensen) Little evidence that MDS indicators per se useful in improving processes of care (Rantz)

29 Prevalence of Depression Without Antidepressant Treatment
VISN 21 National Average Target First Quarter 0.0% 7.9% 4.8% 7.4% 11.0% Second Quarter 12.0% 3.3% 8.8% Third Quarter 3.9% 9.1% 7.3% 10.7% Fourth Quarter 9.2% 7.2% 10.4% Prevalence of Depression Without Antidepressant Treatment Month 3.7% 4.0% 3.6% 4.7% 5.0% 1.3% 3.4% 4.5% 2.9% 2.1% 4.4% 3.5% 2.2% 4.3%

30 Comparison to Published Rates
Percentage of NH depressed on MDS: ~ 10-36% Gold standard comparisons: ~ 46-55% (Schnelle, Simmons) “The MDS depression quality indicator underestimates the prevalence of depressive symptoms in all homes, but in particular, among those reporting low or nonexistent rates” Percentage of depressed patients without antidepressant treatment: 45% (Brown) Note: not validated against pharmacy database

31 Suggestions for Using MDS (and similar) Monitors for QI
Consider the good intent of the monitor It would be bad to have patients starving or thirsty, unnecessarily incontinent, subject to polypharmcy, inappropriately treated with antipsychotics, who never get out of bed and are bored to tears Step Back from the individual and the percentages and ask, “what part of this outcome might we have some influence on? Brainstorm with staff

32 Suggestions for Using MDS (and similar) Monitors for QI
Consider feasibility of possible interventions Benchmark interventions Note changes following intervention Revise intervention

33 Example: Falls Transfers/ambulation: Toileting Slipped/tripped
Wheelchair breaks working and routinely inspected on patients’ wheelchairs? Toileting Nightlights available, sleepers, patient instructions to call for assistance Slipped/tripped Inspect rugs, walkways especially during rainy/snowy seasons Falls related to medical condition Review drug therapy for high-risk patients Inappropriate footwear Inspect patients’ footwear, none-slip socks and slippers

34 Summary Top-down monitors by themselves WILL NOT result in improved care delivery Top-down monitors work best when combined with grass-roots efforts to improve care Where monitors work poorly for a given purpose, it is our obligation to say so – and then work to make things better

Download ppt "Quality Improvement Pitfalls and Opportunities"

Similar presentations

Ads by Google