1. Building and Assessing Competence
David C. Leach, M.D.
Executive Director
ACGME
September 12, 2002

2. Objectives
To clarify what you have known all of your professional life about competence
To explore how residents learn to make good clinical judgments
To define specific steps that can be taken to respond to the ACGME Outcome Initiative

3. Yet can also be immensely satisfying.
Reasons this is hard
Yet can also be immensely satisfying.

4. Reason Number One Competence is a Habit

5. Competence “…the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served.” Epstein and Hundert JAMA, Jan. 9, 2001

6. Why worry about it? Public concerns with safety
Variability in patterns of care that are not based on science
Poor customer service

7. ACGME Outcome Project The Project A long term initiative
The Vision to enhance residency education
The Process through educational outcome assessment

8. Reason Number Two ACGME/RRCs judge competence of Programs
Boards judge individuals

9. A slight problem … We accredit programs Programs don’t exist
The only things that are real are the humans and the relationships between humans in so-called programs
These relationships can either inhibit or facilitate learning

10. So what do we accredit? Humans? Sets of relationships?
Educational outcomes?

11. Reason Number Three Humans: the important things are hard to measure

12. Dee Hock’s Criteria for Hiring People
Integrity
Motivation
Capacity
Understanding
Knowledge
Experience

13. Reason Number Four Knowing the rules is not enough
Residents need to prepare for the unknown

14. Stacey, 1996 -
Chaos
Zone Of
Complexity
Agreement
Control + +
Certainty -

15. Marvin Dunn, M. D. Paul Batalden, M.D.
“Needed are a few organizing principles to have conversations about our work.”
Marvin Dunn, M. D.
Paul Batalden, M.D.

16. Organizing Principles
General competencies
Continuum
Measurements
Improvement models

17. The General Competencies
Patient care
Medical Knowledge
Practice-based Learning and Improvement
Interpersonal and Communication Skills
Professionalism
Systems-based Practice

18. “You must call your mother every Sunday.” Prescription or invitation?
Accreditation Aside
“You must call your mother every Sunday.”
Prescription or invitation?

19. The Continuum
Life after competent

20. Dreyfus Model of Skill Acquisition
Novice
Advanced Beginner
Competent
Proficient
Expert
Master

21. Dreyfus Model Novice Rules Advanced Beginner Rules + Situation
Competent
Rules + Selected Contexts + Accountable
Proficient
Accountable + Intuitive
Immediately sees what
Expert
Immediately sees how
Master
Develops style
Loves surprise

22. “To become competent you must feel bad”
Hubert Dreyfus

23. Between Advanced Beginner and Competent
The number of potentially relevant details becomes overwhelming
Exhausting to manage with rules
Choose a perspective
Result depends on the perspective adopted by the learner/risk taking
Fright replaces exhaustion

24. Two Paths Go back to rules Become fully involved
Cycle between advanced beginner and competent
Burn out
Become fully involved
Feel bad when wrong and good when right

25. Next
Proficient intuition replaces reasoned responses. - immediately sees the problem - recognizes patterns
Expert immediately sees how to solve problem
Master – styles, continuous learning

26. Conceptual Model Patient Care Med Know Practice Based Learning
Inter & Comm Skills
Profess-ionalism
System-based Practice
Novice
Advance Beginner
Comp
Proficient
Expert
Master

27. Residents seek practical wisdom
Reason Number Five
Residents seek practical wisdom

28. Aristotle Episteme Techne Phronesis Cognitive knowledge, science
Craft/Art of medicine
Phronesis
Practical wisdom

29. Accreditation Aside Minimal threshold
Do your graduates know the rules?
Can they apply them in complex contexts without supervision?

30. Accreditation Aside Improvement Model
Do your graduates have the habit of accountability?
Have they acquired practical wisdom?

31. Reason Number Six
The quality of the program is dependent on the quality of the relationships.

32. Medicine, education and management are cooperative arts rather than productive arts.
Therefore the quality of the activity is dependent on the quality of the relationships.

33. Cooperative Arts Medicine
cooperates with the body’s natural tendency to heal
Teaching
cooperates with the mind’s natural tendency to ascend to the truth
Management
cooperates with people’s natural tendency to form communities

34. Microsystems: another unit of learning

35. Real learning (intelligent adaptation) occurs in microsystems
And sometimes in macrosystems.

36. Health care systems consist of macrosystems and microsystems
Paul Batalden, M.D.

37. Substance is enduring; form is ephemeral
Substance is enduring; form is ephemeral. Preserve substance; modify form; know the difference.
Dee Hock

38. Substance attracts resources; form attracts expenses.
Dee Hock

39. Microsystems have a high substance to form ratio; macrosystems have the reverse.

40. Characteristics of High Performing Microsystems
Integration of information
Measurement
Interdependence
Supportiveness of the larger organization
Constancy of purpose
Investment in improvement
Alignment of roles and training
Connection with community Julie Mohr, Ph.D.

41. This is a big opportunity for improvement.
Reason Number Seven
Introducing learners can enhance or inhibit the function of a microsystem.
This is a big opportunity for improvement.

42. Rehearsals are good for relationships and outcomes.
A restaurant in Chicago
Simulation offers a huge opportunity for improvement.

43. Whatever we measure we tend to improve.

44. Useful Concepts about Measurement
Life is not condensable
We use models to understand life
All models are limited, some are useful
Measurements are applied to models
Both measurements and models must be constantly reassessed
We need structured dialogue about measurement

45. Useful Concepts Rules and context
Science is universal; art is always unique
Objective and subjective

46. Characteristics of good assessment
Measures actual performance
Identifies areas for improvement
Satisfies reasonable request for accountability
Is practical
Is done over time to discern growth

48. www.acgme.org/Outcome/ Assessment toolbox References
Table of “best methods”
Key considerations in selecting and implementing assessment approaches
Assessment approaches

49. RRC Think Tank Chair Gail McGuiness, M.D.
Clarify expectations for programs
Clarify operational issues for RRCs
Identify PIF questions relevant to assessment for relational database

50. What to Do Right Away

51. Forming the Initial Response
Show evidence of initial plans
Institutions build in competencies and their assessment into internal reviews
Change the verbs
Start with Global Assessments
Organize Focused Assessments
Patient and/or Professional Associate Assessment

52. What if you were crazy enough to go beyond the minimal?
What would an improvement model look like?

53. Accreditation Aside Minimal threshold – hide flaws
Improvement – expose problems and show how they were addressed
Requires much greater trust

54. Excellent programs will use rehearsals.
Simulations are not the same as simulators

55. Excellent programs will develop virtuous cycles between the microsystems and education.

56. Excellent programs and institutions get to pick their own indicators.
Remember the Northern New England Cardiovascular Surgery experience.

57. Abba Felix Desert Father
“To teach is to create a space in which obedience to truth is practiced.”
Abba Felix
Desert Father

58. What we attend to and how we attend to it defines who we are.

59. A Community of Practice
General Competencies
Open data systems
Celebrate benchmarks across disciplines
Build knowledge about medical education
Build knowledge about improving patient care
Enhance public accountability

60. To Teach/Learn is to create a Space/Community in which obedience to truth is practiced.