1. The Effects of Health Plan Performance Measurement on Quality of Care for Medicare Beneficiaries Presented by Kate Bundorf Co-authors: Laurence Baker and Kavita Choudhry AcademyHealth Annual Research Conference June 27, 2006 This project received support from the National Institute on Aging (AG023557 ) and the Agency for Healthcare Research and Quality (KO2 HS11668).

2. Background Health care provider “report cards” are viewed as a tool to improve quality of care. –Allow consumers to make tradeoffs between cost and quality when choosing among providers. –Create incentives for providers to improve quality of care. Medicare has been at the forefront of efforts to disseminate information about provider quality. NCQA has been the key driver of initiatives to measure health plan quality. The Medicare program began requiring Medicare managed care plans to submit Medicare HEDIS data as of January 1997

3. Have Health Plan Report Cards Improved Quality of Care? Do consumers use the information? –Health plan report cards have had a small effect on health plan enrollment in the commercial population (Chernew, et. al. 2001; Beaulieu, 2002; Scanlon et. al, 2002; Wedig and Tai-Seale, 2002; Jin and Sorenson, 2005). –Health plan report cards in the Medicare program have shifted enrollment from lower to higher quality plans; but –Enrollment shifts have been driven by measures of consumer satisfaction rather than measures of processes of care (Dafny and Dranove 2006). Do providers use the information? –Hospitals responded to the implementation of mandatory report cards even in the absence of evidence that patients were using the information (Dranove, Kessler, et. al. 2003). –HEDIS performance indicators are widely used by health plans to implement and monitor quality improvement programs (Perry 2000; Scanlon, Darby, et. al. 2001).

4. Have Health Plan Report Cards Improved Quality of Care? Performance on the indicators among plans participating in HEDIS has improved over time (NCQA 2002; Lied and Sheingold 2001; Trivedi, Zaslavsky, et. al. 2005). These trends are not necessarily evidence of a causal effect: –No control group –Improvements in performance could be driven by improvements in measurement –Changes in utilization could be due to changing plan enrollment After controlling for these issues in the commercial population, performance improvements are less dramatic (Bundorf and Baker 2006).

5. Study Design Compare utilization of measured services between Medicare managed care enrollees and other beneficiaries before and after the implementation of mandatory quality reporting. Use managed care market share in the beneficiaries’ county to control for both selection of beneficiaries across sectors and spillovers from managed care to the FFS sector.

6. Data Sources 1993-1999 Medicare Current Beneficiary Survey (MCBS) –Annual survey of approximately 12,000 Medicare beneficiaries conducted by CMS. –Study sample includes non-institutionalized beneficiaries >=65 residing in a metropolitan area. 1993-1999 Medicare Managed Care Market Penetration Report from CMS.

7. Performance Indicators Study Variable HEDIS Performance IndicatorMCBS Definition Mammogram (Women 65- 69) The percentage of women 52 – 69 who were enrolled in a health plan that had a mammogram during the measurement year or the year prior to the measurement year. Respondent self report of having had a mammogram within the past year among women 65-69. Beta Blocker for Heart Attack Patients The percentage of members age 35 and older hospitalized and discharged from the hospital after surviving a heart attack who received a prescription for a beta- blocker Identification of utilization of a beta blocker during the survey year among those self reporting that they had ever been told they had a myocardial infarction, including men and women 65 and over. Flu shotThe percentage of older adults who received a flu shot. Respondent self report of having received a flu shot. Eye Exam for Diabetics The percentage of diabetics receiving eye exams. Respondent self report of having had an eye exam in the past year among beneficiaries indicating they had ever been told they have diabetes.

8. Utilization of performance indicators among Medicare managed care enrollees

9. Utilization of performance indicators relative to other beneficiaries

10. Model Estimation Model 1: Unadjusted difference-in-difference estimate Model 2: Control for selection based on observable characteristics Controls include age, sex, education, marital status, race, ethnicity, self-reported history of 16 conditions, self reported health status, number of ADLs, BMI indicators, smoking status, and county fixed effects. Model 3: Control for selection and spillover effects using managed care market share variables Compare the difference between beneficiaries enrolled in managed care plans and those enrolled in FFS before and after the implementation of mandatory quality reporting for Medicare managed care plans (controlling for time trends common to all beneficiaries). Spillover and selection effects: managed care share Selection effects: managed care share*HMO enrollee Difference between pre- and post-intervention periods in spillover and selection effects: Interaction of each variable with an indicator of the intervention period

11. Results for Mammogram

12. Results for Beta Blocker

13. Results for Flu Shot

14. Results for Eye Exams for Diabetics

15. Summary of Results Rates of utilization of the performance indicators we studied did not increase more rapidly among Medicare HMO enrollees than beneficiaries enrolled in traditional Medicare subsequent to the implementation of mandatory quality reporting. The lack of a positive effect of quality reporting on utilization of performance measures among HMO enrollees is not sensitive to controlling for either selection of enrollees across the sectors or spillovers from quality reporting activities to the FFS sector. Utilization rates of beta blockers increased more rapidly for FFS beneficiaries in markets with high levels of managed care penetration in the post-reporting period.

16. Implications The implementation of mandatory health plan quality measurement and reporting did not increase utilization of performance measures among Medicare managed care enrollees for the services we studied. –Report cards did not create strong enough incentives for plans to improve their performance. –Quality improvement programs were in place in HMOs serving the commercial population prior to the implementation of mandatory participation by the Medicare program. –Indicators we studied are not those that were likely to have shown improvement. Quality reporting by HMOs may have had positive spillover effects on the Medicare FFS population.

17. Research Question Did the implementation of mandatory health plan participation in standardized performance measurement and reporting improve quality of care among enrollees in Medicare Managed Care plans?

18. Health Plan Performance Measurement in Medicare In 1996, HCFA contracted with NCQA to develop HEDIS measures for Medicare managed care plans. HCFA began requiring plans to submit Medicare HEDIS data as of January 1997. In 1998, CMS launched www.Medicare.gov.www.Medicare.gov In 1999, HEDIS measures became available on the site.

19. Model Estimation Model 1: Unadjusted difference-in-difference estimate where i indexes individuals, m indexes counties, t indexes years (1993-1999) Y is a binary indicator of utilization of service; H is an indicator of enrollment in a Medicare managed care plan; I is an indicator of whether mandatory quality reporting had been implemented during the time period; and Z includes year fixed effects Model 2: Control for selection based on observable characteristics Controls include individuals characteristics (age, sex, education, marital status, race, ethnicity, self- reported history of 16 conditions, self reported health status, number of ADLs, BMI indicators, and smoking status) and county fixed effects. Model 3: Control for selection and spillover effects using market share variables

20. Identifying Selection and Spillover Effects (Model 3) CoefficientVariableEffect B3B3 Managed Care ShareSpillover and Selection in the pre- intervention period B4B4 Managed Care Share*HMO EnrolleeSelection in the pre-intervention period B5B5 Managed Care Share*Intervention Period Spillover and selection due to quality reporting intervention B6B6 Managed Care Share*HMO Enrollee*Intervention Period Selection due to quality reporting intervention B 3 -(- B 4 )Spillover from HMO enrollment onto FFS beneficiaries (pre-intervention) B 5 -(- B 6 )Spillover from HMO enrollment onto FFS beneficiaries due to quality reporting