1. Do Good Nursing Homes Achieve Good Outcomes? Measurement Considerations Associated with Public Reporting Vincent Mor, Ph.D. Brown University

2. Do Good Nursing Homes Achieve Good Outcomes? 2 Purpose  Review Long Term Care Assessments  Using Nursing Home as Example, Review Long Term Care Outcome Domains and Measures  Present Empirical Data Addressing Select Nursing Home Quality Measurement Issues How Correlated are Quality Measures? How Stable are Quality Measures? How Variable are Quality Measures?  Implications for Public Reporting

3. Do Good Nursing Homes Achieve Good Outcomes? 3 Background  Long Term Care, while relatively deficient in resources, had head start with uniform clinical assessment tools in Rehab Hospitals, nursing homes and home health agencies  FIM for Rehab Hospitals, MDS for nursing homes and OASIS for home health emerged from acknowledged need to improve care quality.  MDS and OASIS implemented and applied to policies ranging from case mix reimbursement to quality review

4. The Nursing Home Resident Assessment Instrument (RAI)  1986 Institute of Medicine Report on Nursing Home Quality Recommended a Uniform Resident Assessment Instrument to Guide Care Planning  OBRA ‘87 Contained Nursing Home Reform Act Including RAI Requirement  A 300 Item, Multi-Dimensional RAI – MDS was Tested for 2 Years  Mandated Implementation in 1991  Fully Computerized in 1998  Case Mix Reimbursement in 1999  Quality Reporting in 2002

5. Do Good Nursing Homes Achieve Good Outcomes? 5 The Outcome and Assessment Information Set (OASIS)  Initially designed as an outcome measurement tool for home health agency staff to track how well patients progressing  Transformed into a multi-purpose tool for: patient assessment and care planning for individual adult patients agency-level case mix reports internal HHA performance improvement Case-Mix Reimbursement mandated by BBA National Home Health Quality Compare 2003

6. Do Good Nursing Homes Achieve Good Outcomes? 6 Functional Impact Measure (FIM)  Initiated as a way to document functional progress during rehabilitation  Widely used in Rehab Hospitals around the country  Measures Mobility, ADL dependence and Cognition  Summary scores created and differences between admission & discharge calculated  Some provider performance profiling underway

7. Do Good Nursing Homes Achieve Good Outcomes? 7 Nursing Home Measurement Issues  What does quality mean and to whom?  Hard to know how much inter-state variation is due to measurement vs. real processes  Differentiating between providers; how much of a difference is important?  Inter-relationship among the domains and measures; implications of composite scores  Stability of measures over time

8. Do Good Nursing Homes Achieve Good Outcomes? 8 Domains of Nursing Home Quality Measures  Regulatory Compliance Deficiencies, Complaints, etc.  Staffing Hours/Resident day, Skill Mix, contractors, etc.  Clinical Processes Restraints, Psychotropics, etc.  Clinical Outcomes ADL decline, Unmet needs, hospitalization  Quality of Life Mood, Food, Activity, Social Interaction  Satisfaction Service meets Expectations? Getting what you want? etc.

9. Do Good Nursing Homes Achieve Good Outcomes? 9 Regulatory Based Quality Measures  State surveyors conduct annual inspections following national protocol to assess compliance with Conditions of Participation  Substantial inter & intra-state variation in number, type, severity and distribution of regulatory infractions  Factors associated with deficiencies also differ somewhat by state  Termination of Medicare/Medicaid due to regulatory violations rare

10. Do Good Nursing Homes Achieve Good Outcomes? 10

11. Do Good Nursing Homes Achieve Good Outcomes? 11

12. Do Good Nursing Homes Achieve Good Outcomes? 12

13. Do Good Nursing Homes Achieve Good Outcomes? 13 Staffing as Structural Quality Measures  Historically, most measures of nursing home quality revolved around staffing levels  The mix of staffing has also been considered, particularly the ratio of RNs to practical nurses and to nursing assistants  The availability of Medical Directors and Nurse Practitioners is a more recent focus  Data Quality poor since reported by home; we apply longitudinal cleaning algorithms

14. Do Good Nursing Homes Achieve Good Outcomes? 14 Trends in Total Staffing Levels

15. Do Good Nursing Homes Achieve Good Outcomes? 15

16. Do Good Nursing Homes Achieve Good Outcomes? 16

17. Do Good Nursing Homes Achieve Good Outcomes? 17 Proportion of facilities using 5% or more contract RNs or LPNs, 1992-2001

18. 18 Note: Difference between unadjusted & adjusted means is statistically significant (at p<.001) by paired T-test for all years 2000-2004.

19. Do Good Nursing Homes Achieve Good Outcomes? 19 MDS Resident Assessment Data Quality Issues  Although inter-rater reliability studies show good results, some studies suggest data MDS not consistently collected across providers.  Inconsistency of data collection undermines comparability of quality measures  Analyses of largest reliability trial done reveals large inter-state differences in the “direction” of disagreements; this is related to inter-state variation in quality indicators

20. Do Good Nursing Homes Achieve Good Outcomes? 20 Process Measures of Quality  Calculating the rate at which treatments that should be delivered to certain patients are and the rate at which certain things are done that shouldn’t be done Restraints Anti-psychotics Advanced Directives in Cognitively Impaired Feeding Tubes in the Cognitively Impaired

21. Do Good Nursing Homes Achieve Good Outcomes? 21

22. Do Good Nursing Homes Achieve Good Outcomes? 22 Changing Rate of restraint use in facilities 1999-2004

23. Do Good Nursing Homes Achieve Good Outcomes? 23 Outcome Measures of Quality  Clinical measures related to “sentinel” health events; rare but should be near “0”  Functional decline rates calculated as changes between assessments  Incidence of clinical events that are not desired but hard to prevent - hospitalization  “Unmet” need for care  Measures of Satisfaction from family or resident

24. Do Good Nursing Homes Achieve Good Outcomes? 24 Double the rate between lowest & highest state.

25. Do Good Nursing Homes Achieve Good Outcomes? 25

26. Do Good Nursing Homes Achieve Good Outcomes? 26 How have clinical quality measures changed over the last half decade?  Pressure Ulcer Prevalence remained stable  Anti-psychotic medication use increased  Restraint use declined  HOWEVER  Lots of variation;  Facilities in the Top Quintile in 1999 converged to the middle; and vice versa  Restraint Improvement resulted from reductions in restraint use among highest users

27. Do Good Nursing Homes Achieve Good Outcomes? 27 Changing Rate of ADL decline in nursing homes 1999-2004

28. Do Good Nursing Homes Achieve Good Outcomes? 28

29. Do Good Nursing Homes Achieve Good Outcomes? 29 Do Nursing Homes that start out with good clinical outcomes remain good?  Split all US nursing homes into quintiles in 1999 based upon their clinical quality  Tracked facilities’ clinical quality measures each quarter over next 5 years  Observed substantial “regression to the mean”  Most homes beginning in the top quintile ended up very similar to those beginning at the bottom

30. Do Good Nursing Homes Achieve Good Outcomes? 30

31. Do Good Nursing Homes Achieve Good Outcomes? 31 Measuring Differences between Providers  Variation is reduced after aggregating measures to the level of the provider  Number of residents per facility means little power in standard statistical tests  Many measures are skewed  Comparing averages versus rates proportion meeting a threshold; basis for setting thresholds?  Are we seeing differences when there are none?

32. Do Good Nursing Homes Achieve Good Outcomes? 32 Facility ADL Decline Rates: 1999-2005

33. Do Good Nursing Homes Achieve Good Outcomes? 33 Facility Restraint Rates: 1999-2005

34. Do Good Nursing Homes Achieve Good Outcomes? 34 60% of customers satisfied In homes in bottom 20%

35. Do Good Nursing Homes Achieve Good Outcomes? 35 Facility level distribution of average resident satisfaction scores: RI 2006.5 range out of 5 points

36. Do Good Nursing Homes Achieve Good Outcomes? 36 Facility level distribution of average family satisfaction scores: RI 2006.5 range out of 5 points

37. Do Good Nursing Homes Achieve Good Outcomes? 37 How do Quality Measures Inter-relate?  Nearly as many providers with low rates of physical restraints have high levels of pressure ulcers as have low levels;  The average correlation among MDS based measures of quality across all US nursing homes is low;  Structural and Regulatory based measures of quality are only minimally related to outcome based measures of quality

38. Do Good Nursing Homes Achieve Good Outcomes? 38 Average within-MSA rank-order correlation coefficients between Clinical & Reputation Quality in 2004 (weighted by # NHs in MSA): OSCAR “Reputational” MeasuresMDS Clinical Measures N (MSAs) [A][B][C][D][E][F]G][H] [A] OCCUPANCY1.000 296 [B] % PRIVATE PAY0.1261.000 296 [C] DEFICIENCIES-0.133-0.1381.000 296 [D] TOTAL NURSE HPPD-0.0790.165-0.1251.000 296 [E] ADL DECLINE-0.0530.0820.0630.0311.000 296 [F] RESTRAINTS-0.021-0.0550.094-0.052-0.0121.000 296 [G] PU WORSENING-0.1200.0450.0970.0250.2130.0451.000 296 [H] PERSISTENT PAIN-0.081-0.0820.104-0.0380.0170.0600.0651.000296

39. Do Good Nursing Homes Achieve Good Outcomes? 39 How well do the “best” homes perform?  “Best” in terms of the fewest regulatory deficiencies, most staff, etc.  Classified facilities into thirds on Deficiencies, Staffing, etc.  Then, compared them on clinical quality measures over multiple years

40. Do Good Nursing Homes Achieve Good Outcomes? 40

41. Do Good Nursing Homes Achieve Good Outcomes? 41

42. Do Good Nursing Homes Achieve Good Outcomes? 42 Does Clinical Quality Predict Nursing Home Termination?  NHs with fewest deficiencies less likely to be terminated  NHs with lowest ADL decline and restraint rate less likely to be terminated  BUT, pressure ulcer worsening or persistent pain performance not related to future termination & staffing barely predictive

43. Do Good Nursing Homes Achieve Good Outcomes? 43 Quality Predicting Nursing Home Closure w/in 2 years Very Rare Event

44. Do Good Nursing Homes Achieve Good Outcomes? 44 Implications for Public Reporting and Pay for Performance  Cross-sectional Comparisons are suspect  Comparisons across state lines suspect  Volatility in measures within facility  Measures aren’t correlated; P4P needs a priori value based weighting  Multi-dimensional quality measures mean consumers will need support in using the data to inform decisions

45. Do Good Nursing Homes Achieve Good Outcomes? 45 Implications for Quality Measurement  Improving Data Quality is essential; applies to regulatory, staffing, MDS based data and patient/family surveys  Addressing Methodological issues essential before applying these data to P4P  Findings are likely applicable to Home Health Agencies and Rehab Hospitals