1. Comparison of Physician Rankings on Performance Quality Composites in the Care of Hypertensive Patients AcademyHealth, June 9, 2008 Washington, DC Weifeng Weng, Gerald K. Arnold, Eric S. Holmboe, Rebecca S. Lipner

2. ABIM and Maintenance of Certification (MOC)  ABIM certifies physicians in internal medicine and its subspecialties  Certification is time-limited: 10 year duration  Renew certificates through MOC program  MOC requires demonstration of –Professionalism –Lifelong Learning –Cognitive Expertise –Practice performance

3. Practice Improvement Module (PIM TM ) Performance Report Improvement Chart audit Patient survey Impact plan do study act Practice survey Based on Picker patient and CAHPS surveys Based on Wagner’s Chronic Care Model & IHI’s Idealized Office Design Evidence-based guidelines

4. Research Question  In P4P programs –Clinical measures dominate –Patient survey and Practice system survey measures used less frequently –Typically, rewards awarded by relative ranking –Typically not all three data streams used  Do physician performance rankings (and rewards) vary considerably when different combinations of the three data streams are used?

5. Methods  Physician database with patient-level data  Standardized composite scores for the three data streams (1) Chart audit (2) Patient survey (3) Practice systems  Super-composite scores: Combine composites  Examine changes in physician rankings

6. Physician and Patient Samples  659 Physicians –Mean Age: 44 (SD = 6.4), 26% female –61% general internists, 39% subspecialists (largely nephrologists and cardiologists) –29% in solo practice  Patients –Chart audit: 13,096 patients, age 18-75, 51% male –Patient survey: 14,913 patients, age 18-75, 53% male

7. Chart Audit Individual Measures Measures Physician levelICC N for R m =0.85 Mean (SD)Rho95% CI Blood pressure control0.52 (0.19)0.120.10 to 0.1442 LDL control0.55 (0.20)0.160.14 to 0.1829 Complete lipid profile0.82 (0.22)0.430.39 to 0.468 Urine protein test0.81 (0.26)0.640.60 to 0.683 Annual serum creatinine test0.88 (0.16)0.360.32 to 0.4010 DM co-morbidity documentation or screen test0.93 (0.15)0.590.54 to 0.644 Aspirin therapy for eligible patients0.74 (0.25)0.400.36 to 0.449 Smoking status and cessation advice and treatment0.96 (0.08)0.360.30 to 0.4210 Counseling for diet and physical activity0.89 (0.18)0.510.47 to 0.565 Outcome variables are risk adjusted for co-morbidity conditions: BP control <130/80 for pts with dm or stroke co-morbidities, <140/90 for rest. LDL control <100 for pts with major risks, <130 for pts with other risks, <160 for the rest.

8. Patient Survey Individual Measures Measures Physician levelICC N for R m =0.85 Mean (SD)Rho95% CI Overall hypertension care0.88 (0.12)0.200.17 to 0.2322 Encouraging/answering questions0.83 (0.14)0.190.16 to 0.2224 Providing information on Medication side effects0.73 (0.17)0.170.14 to 0.1928 Providing information on foods to eat & avoid0.60 (0.17)0.120.10 to 0.1441 Providing information on taking medication properly0.86 (0.13)0.190.16 to 0.2224

9. Practice System Individual Measures Measures (# of measures)MeanSDAlpha Information Management (27)0.540.240.89 Patient activation & communication (28)0.560.230.88 Access & communication with patients (7)0.630.270.73 Safety & efficiency (11)0.720.240.82 Practice team (8)0.720.320.82 Consultation & referral (4)0.620.310.65 Practice system survey of 89 questions

10. Distribution of Physician Performance Composite Scores C+P+S C+S C+PSystem (S) Patient (P) Chart (C) -8 -4 -6 0 -2 4 2 6

11. Correlations among Composites ChartPatientSystemC+PC+SC+P+S Chart (C)1.000.220.190.760.720.66 Patient (P)1.000.150.770.220.63 System (S)1.000.200.780.66 C+P1.000.590.84 C+S1.000.87 C+P+S1.00

12. Percent who change rankings by more than one quartile* Baseline: Chart * One quartile counts for 164 rank positions Ranks better than chartRanks worse than chart

13. Percent who change rankings by more than two quartiles* * Two quartile counts for 329 rank positions Baseline: Chart Ranks better than chartRanks worse than chart

14. Examples of Extreme Discordance of Performance – more than three quartiles Ranks and (z scores) PhysicianChart (C)Patient (P)System (S)C+PC+SC+P+S 1245928335615187 (1.3)(-1.2)(1.1)(0.08)(2.4)(1.2) 2 6522874.5644586538 (-3.6)(0.23)(1.7)(-3.4)(-1.9)(-1.7) Rank 1 = Best; Rank 659 = Worst

15. Conclusions  Measuring multiple dimensions in the quality of patient care is complex –Very moderate correlations among three data streams –Rankings change considerably depending on combinations  A profile that incorporates more than one aspect of patient care tells a different story than any one of them alone

16. Limitations and Future Research  Self-report data for chart and system data  Participants are volunteers  Need more robust risk adjusters  Investigate other analytic approaches for combining individual measures into composites  Investigate stability of pass/fail decisions