1. Ongoing Evaluation of Physician Performance: Developing a Performance Portfolio Cary Sennett, MD, PhD MedBiquitous Annual Conference May 15, 2008

2. Overview  “Performance Portfolio”—what are we talking about?  Design Criteria—what would a portfolio look like?  One—I hope promising—effort (in a bit more detail)  Summary…

3. Defining terms  Portfolio: A (comprehensive) information set that a physician can use To establish his/her qualifications To improve care  A “performance portfolio” to improve care Must begin with information about practice performance Must contain elements that speak to the capability of those factors that determine practice performance

4. Elements that “speak to capability”  How my practice is performing depends on My competence (my knowledge and skill) How effectively that competence is deployed  “Deployment”—that is, the conversion of capability to results—depends upon environmental factors “Micro-environment” “Macro-environment” And (of course) the patient…  For purposes of our discussion, it may be less important to consider the “macro-environment”

5.  Practice performance Are we achieving the results that we could? What limits our ability to do better?  His or her own competence Do I have the knowledge and skill necessary to deliver excellent care? And how can I close critical gaps?  (Micro)-system Is the (micro-) system in which I function optimally configured to support my efforts to deploy that competence? Am I using it effectively? And, if not, how can I improve it?  My patients Have I maximized their ability to achieve desired health results? And, if not, how can I do so more effectively? So a portfolio should help a physician evaluate

6. Designing a portfolio (to support practice improvement)  Practice results  Physician knowledge and skill  Systems infrastructure  Patient self-management  Need for broad set of inputs

7. What “Practice Results?”  Must address the range of results that are relevant to patients (and other customers…)  It may be helpful to think of that range as the IOM does: care that is Safe Timely Effective Efficient (delivered at appropriate cost) Equitable Patient-Centered  But, in any event, it is essential to recognize that performance is multidimensional

8. What “Knowledge and Skill?” ACGME (ABMS) competencies  Professionalism  Patient Care  Medical Knowledge  Communication and Interpersonal Skill  Systems-based Practice  Practice-based Learning and Improvement

9. What “System Infrastructure?”  Information management  Patient activation  Access and communication with patients  Safety and efficiency  Consultation and referral  Team function  Improvement process

10. How do we get there?  A lot of activity right now—but widely distributed  ABMS Certification/Maintenance of Certification may be a helpful “seed” around which this work can organize  Step on the path—but not the end of the road

11. Board Certification  Professional effort to evaluate competency (capability) of individual physician  Structured so as to offer a window into other key portfolio elements Practice results Practice systems infrastructure Patient self-management  Designed—and objective is—to support improvement  But essential function is assessment (both formative and summative)

12. The Structure of Board Certification  Specific reference to ACGME competencies  Two elements Initial certification Periodic recertification/Maintenance of Certification  Structure common across 24 ABMS Boards Assessment of actions against license Self-assessment of medical knowledge High stakes secure examination of cognitive knowledge and judgment Self-assessment and improvement of practice performance  Boards vary with respect to implementation— strategies and timelines

13. Self-Assessment of Practice Performance  Designed to force “a new way of thinking” about quality and QI Quality is about what the system produces Quality improvement is not about working harder (or knowing more)—but about “diagnosing and treating” system problems  Designed to promote adult (experiential) learning  Physicians can receive up to 20 units category I CME credit, as well as credit toward renewing their certificate

14. The ABIM Practice Improvement Module (PIM™) Performance Report Improvement Patient survey Impact Plan Do Study Act Practice survey Chart review Apply quality measures to practice Compare performance to guidelines Test a process change aimed at improving care Examine practice infrastructure and process Report what was learned

15. The PIM as Portfolio  Provides window on practice results  Can link to other information about physician competency (knowledge and skill)  Provides window on systems infrastructure  Potential window on “patient competence”

16. Increasing robustness  Need broader window on practice performance— more, and more diverse—data  Need tighter and more intentional link to information about individual physician competency  Need mechanism to track (all elements over time)  Need expanded window on “patient competence”

17. Getting from here to there: “PIM Future”  Links to extant datasets—power practice and self- assessment off of available data  Performance (and personal) “dashboards”: turning data into information that speaks (meaningfully and reliably) to practice performance and to core drivers  Links to—truly integration with—support needed to go from “I see an opportunity to improve” to “I’m able to capitalize on that opportunity to improve”

19. What will it take?  More (more standardized and more available) data from many, many sources  More (much more) research  Collaboration/partnerships that will link assessment capability to improvement capability  Collaboration/partnerships to organize and focus energy and resources

20. Summary  It is possible to conceive of a portfolio that will support practice improvement  Such a portfolio will need to address a broad ranges of issues  Board certification (Maintenance of Certification) may be an important part of early efforts to build a portfolio  To get where we need to go will require collaboration—which will require much, much more standardization