Presentation is loading. Please wait.

Presentation is loading. Please wait.

Quality Improvement Pitfalls and Opportunities James Hallenbeck, MD ACOS/Extended Care VA Palo Alto HCS.

Similar presentations


Presentation on theme: "Quality Improvement Pitfalls and Opportunities James Hallenbeck, MD ACOS/Extended Care VA Palo Alto HCS."— Presentation transcript:

1 Quality Improvement Pitfalls and Opportunities James Hallenbeck, MD ACOS/Extended Care VA Palo Alto HCS

2 Objectives Compare top-down and grassroots strategies for quality improvement in terms of advantages and disadvantages Compare top-down and grassroots strategies for quality improvement in terms of advantages and disadvantages Discuss different uses of quality monitors Discuss different uses of quality monitors Comparisons across groups Comparisons across groups Quality improvement Quality improvement Better respond to queries regarding monitors, especially performance measures, especially MDS Better respond to queries regarding monitors, especially performance measures, especially MDS Be able to…….

3 How this might be useful to you… Self defense - protecting you and your program when data does not accurately reflect the quality of care you deliver Self defense - protecting you and your program when data does not accurately reflect the quality of care you deliver Promote better use of monitors (MDS and others) for quality improvement Promote better use of monitors (MDS and others) for quality improvement

4 Purposes of Quality Improvement Monitors Comparisons Comparisons VISNs, facilities, wards, clinicians VISNs, facilities, wards, clinicians Stimulus for quality improvement efforts Stimulus for quality improvement efforts Benchmarking and ongoing evaluation of quality improvement efforts Benchmarking and ongoing evaluation of quality improvement efforts

5 What Makes a Good Monitor? (Top-down or grassroots) Desired outcome strongly linked to what is measured (i.e. minimally influenced by factors outside the health systems control) Desired outcome strongly linked to what is measured (i.e. minimally influenced by factors outside the health systems control) System change, informed by monitor, would likely result in better outcomes System change, informed by monitor, would likely result in better outcomes Accuracy Accuracy Minimal false negatives & false positives Minimal false negatives & false positives

6 QI INDICATORS TOP TO BOTTOM Enables comparisons across large systems Enables comparisons across large systems Establishes organizational priorities Establishes organizational priorities Buy-in from top management Buy-in from top management If successful, results in broad, measurable improvement If successful, results in broad, measurable improvement Blind to local issues By attending to certain priorities, others may be ignored Lack of buy-in from front-line staff Need large numbers Disempowerment at local levels Advantages:Disadvantages:

7 What Makes a Good Top-Down Monitor? Simple, discrete data Simple, discrete data Counts of things Counts of things Minimal variation in local systems and populations Minimal variation in local systems and populations Comparing apples to apples Comparing apples to apples Large data numbers – in numerator and denominator Large data numbers – in numerator and denominator Data gathering, measurement and comparison automatic Data gathering, measurement and comparison automatic Encourages GOOD clinical practice (not just designed to identify bad practice) Encourages GOOD clinical practice (not just designed to identify bad practice)

8 Examples Good Top-Down Monitors Flu vaccine administration Flu vaccine administration Appointment waiting time Appointment waiting time

9 QI Grassroots Tailored to local circumstances Tailored to local circumstances Better buy-in from front- line staff Better buy-in from front- line staff Can improve morale through empowerment Can improve morale through empowerment Comparing outcomes across systems difficult Potential lack of buy-in from top management Harder to disseminate best practices Politically, invisible Advantages Disadvantages

10 Examples Grassroots QI Use of out-of-hospital DNR forms Use of out-of-hospital DNR forms Addressing options of autopsy/organ donation Addressing options of autopsy/organ donation

11 Apples to Apples Problem… Case-mix issues Case-mix issues Severity of illness (Problematic: pressure ulcers, weight loss, dehydration in Hospice) Severity of illness (Problematic: pressure ulcers, weight loss, dehydration in Hospice) Gender (Problematic: UTI rates) Gender (Problematic: UTI rates) Depression/atypical neuroleptics (Problematic: geropsych units, hospice) Depression/atypical neuroleptics (Problematic: geropsych units, hospice)

12 Vertical Hierarchy Problem What is true at one level of a vertical hierarchy is not necessarily true at another (higher or lower) What is true at one level of a vertical hierarchy is not necessarily true at another (higher or lower) Example: Newtonian physics versus quantum mechanics Example: Newtonian physics versus quantum mechanics 30,000 feet view: reveals certain patterns, otherwise invisible, but obscures other 30,000 feet view: reveals certain patterns, otherwise invisible, but obscures other

13

14 Vertical Hierarchy Nation VISN 1VISN 3VISN 2 Facility 1 Facility 2 Facility 3Facility 4 Division/Ward Clinician + Patient

15 Connection Between What is Measured and Desired Outcome Problem Many outcomes (good and bad) minimally dependent on what we as clinicians do Many outcomes (good and bad) minimally dependent on what we as clinicians do Examples: Examples: UTI rates UTI rates Fall rates Fall rates Important to tease out those aspects of the outcome that are dependent on the healthcare system. Ideally, these aspects should be measured and worked on.

16 Environment Luck Healthcare System Clinician Patient Patient Variables

17 Example: Falls Falls related to transfers/ambulation---18 Falls related to transfers/ambulation---18 Falls related to toileting13 Falls related to toileting13 Falls related to reaching/picking/bending7 Falls related to reaching/picking/bending7 Claimed did not fall/eased self to floor--5 Claimed did not fall/eased self to floor--5 Slipped/tripped-2 Slipped/tripped-2 Falls related to medical condition---2 Falls related to medical condition---2 Falls related to inappropriate footwear-2 Falls related to inappropriate footwear-2 1 month data: 47 falls Can you tell from this data how these falls might be related to the healthcare system?

18 Numerator/Denominator Problem Percentages meaningless in isolation (numerator/denominator not given) Percentages meaningless in isolation (numerator/denominator not given) Small numbers – 1 – 10 always suspicious Small numbers – 1 – 10 always suspicious Numerator problem: rare events Numerator problem: rare events Denominator problem: Denominator problem: Restricted population based on: Restricted population based on: Geography (single ward) Geography (single ward) Time interval (monthly tracking) Time interval (monthly tracking)

19 Oct 04 - Jun 05 Total FALLS REPORT

20 Coin-Toss QI 10 Facilities toss a coin 10 times 10 Facilities toss a coin 10 times Heads is Good, Tails is Bad Heads is Good, Tails is Bad True chance (within infinite tosses) = 50% True chance (within infinite tosses) = 50% Lets compare facilities to national data Lets compare facilities to national data

21 Coin Toss Of 100 tosses 56/100 Heads = National average Problem Facilities Exceptional Facilities

22 Implications Unadjusted percentages create a bias in which small facilities/programs with small numbers to report for a given indicator are more likely to be classified at extremes (failing or exceptional)

23 Data Validity Problem Ideal: Ideal: Automated Automated Unambiguous results Unambiguous results Problematic: Problematic: Multiple people/services responsible for data entry Multiple people/services responsible for data entry Results dependent on subjective judgments Results dependent on subjective judgments Questionable inclusion/exclusion criteria Questionable inclusion/exclusion criteria

24 MDS Examples Dehydration monitor: Dehydration monitor: Little or no activity monitor Little or no activity monitor Bedfast Status monitor Bedfast Status monitor

25 Blue = sub acute and hospice wards, Red = national

26

27 Dehydration So, Whats the Problem Here?

28 MDS Quality Indicators: What does the literature show? MDS tends to under-report problems MDS tends to under-report problems Example: depression: Example: depression: Differences in rates often reflect either differences in reporting or case mix – NOT quality Differences in rates often reflect either differences in reporting or case mix – NOT quality Example: High/Low Pressure Ulcer NH did not differ significantly in care processes (Bates-Jensen) Example: High/Low Pressure Ulcer NH did not differ significantly in care processes (Bates-Jensen) Little evidence that MDS indicators per se useful in improving processes of care (Rantz) Little evidence that MDS indicators per se useful in improving processes of care (Rantz)

29 VISN 21National AverageTarget First Quarter0.0%7.9%4.8%7.4%11.0% Second Quarter0.0%12.0%3.3%8.8%11.0% Third Quarter3.9%9.1%0.0%7.3%10.7% Fourth Quarter0.0%9.2%0.0%7.2%10.4% Prevalence of Depression Without Antidepressant Treatment MonthVISN 21National AverageTarget First Quarter0.0%3.7%4.0%3.6%4.7% Second Quarter0.0%5.0%1.3%3.4%4.5% Third Quarter0.0%2.9%0.0%2.1%4.4% Fourth Quarter0.0%3.5%0.0%2.2%4.3%

30 Comparison to Published Rates Percentage of NH depressed on MDS: ~ % Percentage of NH depressed on MDS: ~ % Gold standard comparisons: ~ 46-55% (Schnelle, Simmons) The MDS depression quality indicator underestimates the prevalence of depressive symptoms in all homes, but in particular, among those reporting low or nonexistent rates Gold standard comparisons: ~ 46-55% (Schnelle, Simmons) The MDS depression quality indicator underestimates the prevalence of depressive symptoms in all homes, but in particular, among those reporting low or nonexistent rates Percentage of depressed patients without antidepressant treatment: 45% (Brown) Percentage of depressed patients without antidepressant treatment: 45% (Brown) Note: not validated against pharmacy database

31 Suggestions for Using MDS (and similar) Monitors for QI Consider the good intent of the monitor Consider the good intent of the monitor It would be bad to have patients starving or thirsty, unnecessarily incontinent, subject to polypharmcy, inappropriately treated with antipsychotics, who never get out of bed and are bored to tears It would be bad to have patients starving or thirsty, unnecessarily incontinent, subject to polypharmcy, inappropriately treated with antipsychotics, who never get out of bed and are bored to tears Step Back from the individual and the percentages and ask, what part of this outcome might we have some influence on? Step Back from the individual and the percentages and ask, what part of this outcome might we have some influence on? Brainstorm with staff Brainstorm with staff

32 Suggestions for Using MDS (and similar) Monitors for QI Consider feasibility of possible interventions Consider feasibility of possible interventions Benchmark interventions Benchmark interventions Note changes following intervention Note changes following intervention Revise intervention Revise intervention

33 Example: Falls Transfers/ambulation: Transfers/ambulation: Wheelchair breaks working and routinely inspected on patients wheelchairs? Wheelchair breaks working and routinely inspected on patients wheelchairs? Toileting Toileting Nightlights available, sleepers, patient instructions to call for assistance Nightlights available, sleepers, patient instructions to call for assistance Slipped/tripped Slipped/tripped Inspect rugs, walkways especially during rainy/snowy seasons Inspect rugs, walkways especially during rainy/snowy seasons Falls related to medical condition Falls related to medical condition Review drug therapy for high-risk patients Review drug therapy for high-risk patients Inappropriate footwear Inappropriate footwear Inspect patients footwear, none-slip socks and slippers Inspect patients footwear, none-slip socks and slippers

34 Summary Top-down monitors by themselves WILL NOT result in improved care delivery Top-down monitors by themselves WILL NOT result in improved care delivery Top-down monitors work best when combined with grass-roots efforts to improve care Top-down monitors work best when combined with grass-roots efforts to improve care Where monitors work poorly for a given purpose, it is our obligation to say so – and then work to make things better Where monitors work poorly for a given purpose, it is our obligation to say so – and then work to make things better


Download ppt "Quality Improvement Pitfalls and Opportunities James Hallenbeck, MD ACOS/Extended Care VA Palo Alto HCS."

Similar presentations


Ads by Google