Teaching and Assessing Critical Reasoning in the Era of Competency-based Medical Education, Milestones and Entrustment Preventing Diagnostic Error

“ We’re pretty sure it’s the West Nile virus.”

Clinical Reasoning: A Primer Patient/situation characteristics Prior knowledge Problem Representation Information Gathering Context EvaluationAction Gruppen and Frohna, International Handbook on Research, 2002

Clinical Reasoning  Internal process  Trainees and faculty need ways to externalize and teach this process  Programs need assessment methods that document growth and competency in this skill

Critical Thinking  Good Thinking –Sensitivity Interest in gaining more information Seeking alternatives –Inclination Willing to invest energy in thinking the matter through –Ability Possess the cognitive ability  “Bad Thinking” – “cognitive misers” –Chose to take mental shortcuts, engage in heuristic thinking, –without interest in “good thinking” Krupat 2011

Adverse Events and Clinical Reasoning  Graber¹ adverse event study: – Most errors combination of individual and systems factors –Average 5.9 system +/- cognitive error per case  “ Cognitive factors” –320 cognitive factors in 74 cases –45 due to faulty data gathering –264 due to faulty synthesis (problem representation – clinical reasoning) ¹ Arch Intern Med. 2005; 165: 1493.

The Process of Clinical Reasoning

How can clinical reasoning be taught and assessed in a competency-based system to reduce diagnostic error?

CBME - a new paradigm know You must truly know the trainee has demonstrated competence and is ready to progress to the next stage of their training or career: Requires clear definition of expected outcomes (“good thinking” with decrease in diagnostic errors) Requires assessment and evaluation systems capable of demonstrating that these things are done consistently and within the contextual needs of the clinical environment

Requirements –The content of “good thinking” Define the K/S/A of “good thinking” and frame as an entrustment –An assessment and evaluation system –Faculty Development – creating a shared “mental model” or understanding of “good thinking” and how it is assessed and evaluated

The Content - Twelve Tips “to prevent diagnostic error”  Understand heuristics  Use “diagnostic timeouts”  Think “worst-case scenario medicine”  Systematic approach to common problems  Ask why  Teach/emphasize physical exam  Teach Bayesian theory  Acknowledge your emotions  Identify what doesn’t fit  Embrace zebras  “Slow down”  Admit mistakes Trowbridge Medical Teacher 2008

The “Twelve Tips” and the Internal Medicine Curricular Milestones  142 discrete milestones published in 2009  Describe developmentally the discrete K/S/A needed for competency in the six ACGME General Competencies  Cross walking the 12 tips against the milestones identifies at least 28 milestones that capture the knowledge, skills or attitudes that could be used to teach and assess critical reasoning.

Patient Care ACGME Competency Developmental Milestones Informing ACGME Competencies Approximate Time Frame Trainee to Achieve Stage Assessment Methods/Tools Clinical skills and reasoning  Manages patients using clinical skills of interviewing and physical examination Historical Data Gathering 1.Acquire accurate and relevant history from the patient in an efficiently customized, prioritized, and hypothesis driven fashion 2. Seek and obtain appropriate, verified, and prioritized data from secondary sources (e.g. family, records, pharmacy) 3. Obtain relevant historical subtleties that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient 6 months 9 months 18 months Standardized patient Direct Observation Simulation Sub- competency

“Entrustment in Medical Education”  Focused assessments around what faculty and training programs “entrust” trainees to do? Think critically to minimize error  Reflects the most important outcome of training: a trainee’s readiness to bear professional responsibility”

How do I develop an assessment? Step 1 – Describe the activity. What tasks constitute the entrustment. Step 2 – Identify the Curricular Milestones (142) that will help you assess a resident performing this activity. Step 3 – Identify specific assessment methods / tools to which you can apply the chosen Curricular Milestones.

Clinical Reasoning Step 1 – Describe the activity. What tasks are required for you to entrust this activity to a resident? The “Good Thinker” as described by the twelve tips cross walked to the Internal Medicine Curricular Milestones.

Twelve Tips “to prevent diagnostic error” – The “Good Thinker” Entrustment  Understand heuristics  Use “diagnostic timeouts”  Think “worst-case scenario medicine”  Systematic approach to common problems  Ask why  Teach/emphasize physical exam  Teach Bayesian theory  Acknowledge your emotions  Identify what doesn’t fit  Embrace zebras  “Slow down”  Admit mistakes Trowbridge Medical Teacher 2008

Clinical Reasoning Step 2 – Identify the Curricular Milestones (142) that will help you assess a resident performing this activity Key Considerations: What Curricular Milestones are best assessed in this setting?… in this context? You don’t have to choose all milestones, only those that will help you to “see” competence in the trainee. Crosswalk the twelve tips and the 142 curricular milestones.

Clinical Reasoning Step 3 – Identify specific assessment methods / tools to which you can apply the chosen Curricular Milestones. Key Considerations: What Curricular Milestones are best assessed in this setting?… in this context? You don’t have to choose all milestones, only those that will help you to “see” competence in the trainee

Methods  Portfolio – with required defense by learner –Case log –Focused narrative writing –Admit mistakes  Chart stimulated recall –Structured questioning regarding the twelve tips milestones  Bedside rounds –One minute preceptor –Time out

Please check ONLY ONE box per statementYNNA Clear chief complaint Delineation of sick vs non-sick Appropriate history Appropriate physical Appropriate analysis of lab data Appropriate differential diagnosis Appropriate thought process for differential diagnosis Treatment appropriate for diagnosis Appropriate thought process for treatment plan Overall Note: Clarity____________________________________ Organization__________________________________ Internal consistency____________________________________ Documentation____________________________________

The One Minute Preceptor – A Strategy For Busy Clinicians  Clinical teaching strategy  5 microskills Get a commitment Probe for supporting evidence Teach a general rule Reinforce what was done right Correct mistakes “Create time for reflection” Neher, Gordon, Meyer, Stevens. J Am Board Fam Pract 1992; 5:

“The System”  The Donobedian Framework –Schematic representation of a system  Every system has a structure and a process that processes that produce an outcome S + P = O S(the when/where) + P (the teaching and assessing) = O (“Good thinkers”)