1. Metacognition: Applying Intuition and Intellect to Clinical Reasoning
Clinton Pong, MD
Tufts/Cambridge Health Alliance
Family Medicine, Master Teacher Fellow, PGY-5
FMEC October 2014

2. Aims and Objectives Teaching Aims: Learning Objectives:
Give clinicians simple tools they can use to routinely pause and reflect on their clinical reasoning (CR) process
Engage in discussion on teaching/assessment of CR
Learning Objectives: 
by the end of the session, participants will be able to:
Recognize common heuristics and biases
Define the dual-process cognitive model
Apply a systematic, routine method for catching clinical errors before they occur
Reflect on current milestone level for critical thinking
Raise of hands: how many of the audience are:
Medical students? Residents? Faculty?
Taylor time of sections to presentation

3. Clinical reasoning as Detective work
Clinical Method
Detective Fiction
Started by Sir William Osler in 1890s
Moved medicine out of the classroom and invented “bedside rounds”
“Listen to your patient, he is telling you the diagnosis”
Started by Sir Arthur Conan Doyle in 1890s
Faith in Logic, keen observations and “deductive” reasoning
“You See, but you do not Observe. The distinction is clear.”
Analogies between detective fiction and clinical method: Born in the same historical period, Second half of the 19th century, Shared faith in logic, Similar working patterns

4. Thinking about thinking
Metacognition, Heuristics, and Seeing Patterns
Primer
Thinking about thinking

5. Managing One’s Own Thinking
Metacognition 
the act of “thinking about thinking (and feeling)”
Of one’s own and another’s 
Heuristic
Greek: "Εὑρίσκω", "find" or "discover"
strategies using readily accessible, though loosely applicable, information to control problem solving
Rules of Thumb
e.g. “Better safe than sorry”
Heuristic: mental short cuts to ease the cognitive load of making a decision. Examples of this method include using a rule of thumb, an educated guess, an intuitive judgment, or common sense.

6. Medical Heuristic Aphorisms
When you hear hoof beats, think of horses, not zebras
Geography is Destiny.
If it looks like a duck, walks like a duck, quacks like a duck, it’s a duck.
When all you have is a hammer, everything looks like a nail
Here are some common medical quotes that we use as rules of thumb: do you know which heuristics are being activated with these aphorisms?

7. Unpacking principle & Search satisficing
Failure to elicit key history/physical findings or neglecting to pursue pertinent negatives
Ignoring your toolkit
(Satisfy + Suffice)
Calling off the search for other diagnoses when something is found
(but other things are missed)
We have a toolkit that we typically use and sometimes we need to unpack the wrench and screwdriver instead of the hammer.
SS is a Portmanteau of “Satisfy and Suffice.” We need to be bloodhounds in hunting things down and keep sniffing for clues.

8. Anchoring, Adjustment & Context Errors
First Impressions: initially locking-on to early features without later revision
can be corrected by extending the chain, link by link
but we are subconsciously “primed” by our starting point
Work by Daniel Kahneman and Amos Tversky has shown that there is a “mental priming” that subconsciously contaminates our initial starting points.
Context errors are most noticeable when they occur at the beginning of a case presentation with novice learners.
Critical signals are distorted by background noise
Hanging weight on the wrong evidence
“Red herrings”
“getting thrown off the trail”

9. Available recall & Base-Rate Neglect
Judgment of what is likely based on:
previous experience,
recent events or
recalling what easily comes to mind
Exaggerating or minimizing the prevalence of a disease
Sometimes with flashback to prior situations.
B-R neglect typically comes to my mind when I think about specialists and tertiary care centers. They have a selection bias from referrals.

10. Premature closure
“Locking-in” a diagnosis too early; closing a mental door
“When the diagnosis is made, the thinking stops.”
This is the most common heuristic people think about with misdiagnosis and it typically occurs “post-mortem” during an M&M.

11. Representativeness restraint
Typified pattern recognition without considering atypical presentations
“Forcing a fit”
Disregard for pattern recognition by thinking its “good enough”
Dr. Centor, of strep throat fame, says it best in his blog:
Example of Representative Restraint:
Many diagnostic errors occur because we try
to fit the data to our hypothesis rather than
fitting the hypothesis to our data.
-Dr Robert Centor

12. On dispositions, biases and error
When our thin slices lead us astray
On Problems
On dispositions, biases and error

13. When Heuristics Fail Heuristics are Cognitive Dispositions to Respond
Cognitive biases
Predictable patterns of deviation in judgment
Sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or irrationality
Cognitive Errors

14. Heuristic Distribution Dispositions > Biases > Error
Most of the time, heuristics work in our favor. That’s why a great clinic day seems to “just click” and flows. This is a graph I just made up to illustrate a point: our dispositions are right most of the time, become ruts some of the time and that sometimes contributes to errors. Researchers fall prey to B-R Neglect and blame ALL Heuristics for a MINORITY of Errors

15. Biases / Error M&M discussion
Anchoring
Available recall
Base-rate neglect
Premature closure
Representativeness restraint
Unpacking principle & Search satisficing
Context errors
First Impressions
Previous experience
“Making mountains out of molehills” or vice versa
Lock-it in. Prejudice
Typical vs atypical patterns
Calling off the search too early
Red herrings
Now that you’re all primed, we’ll pause here and take some time reflecting on a recent M&M, misdiagnosis or patient error that you’ve experienced recently.  Take a few minutes in a group of two and look at the Heuristics and Biases checklist and have someone share a brief example with a partner. We’ll spend five minutes doing this.

16. Positive Dispositions: the “Anti M&M”
Observe and deduce
Recognize inconsistencies
Exhibit Tenacity
Avoid protocols and procedures when their hunch takes them in another direction
Stop and think when necessary
Adjustment, Experiential Availability
Representativeness Alert
Avoid Unpacking Principle and Search satisficing
Representativeness restraint
Switch to System II
Let’s contrast the Biases/Error point of view with a positive checklist, generated from fictional detectives. Now switch and have your partner think about how these steps SAVED you from an M&M, misdiagnosis or error. How many of you have the opposite of an M&M? What would you call THAT?
Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December ;24-31.

17. Mental Model for clinical reasoning
Type I vs Type II: Work by Croskerry, Graber, Klein, Kahneman & Tversky
On Theory
Mental Model for clinical reasoning

18. Two Approaches to Clinical Reasoning
Cognitive processing
What goes on in our brains when we are making a decision and how do we arrive at our final diagnosis/plan?
We use two types of thinking:
Type I Heuristic
Go / Fast / “Hot” / Intuition
Type II Analytic
Stop / Slow / “Cold” / Intellect

19. General properties of Type I and Type II
Property
System 1
System 2
Reasoning style
Intuitive
Analytical (“two/too” intellectual)
Heuristic
Normative
Associative
Deductive
Concrete
Abstract
Cost/Effort
Low/Minimal
High/Considerable
Awareness/Automaticity
Low/Automatic
High/Deliberate
Speed
Fast
Slow
Channels
Multiple, parallel
Single, linear
Propensities
Causal
Statistical
Action
Reflexive, skilled
Deliberate, rule-based
Prototypical
Yes
No, based on sets
Errors
Common
Few
Reliability
Low, variable
High, consistent
Vulnerability to bias
Less so
Affective valence
Often
Rarely
Context importance
High
Low
Predictive power
Scientific rigour
Type I process: the automatic hard-wired, specialized parts of the brain that deal with specific needs
Type 2 processes: deliberate training used to solve reasoning and decision problems in a systematic, analytical way. They follow rules of logic and science and generally deliver error-free solutions providing that everything works appropriately, but this isn’t always the case. Consider the computer. When patients use the internet to find a diagnosis, that’s error-prone. When doctors use their EMR, “suggestions” often hinder our care.

20. Croskerry’s Dual-Process Model for Diagnostic Thinking (modified)
Type I
intuition
Diagnosis/ Management
Decision
Pattern processing
When we consider a patient case, our brain processes it and asks “does this match a pattern?” If so, we quickly use Type I intuition or “Kirk” thinking. If not, we slowly ponder using our Type II Intellect or “Spock” thinking. They duke it out, either consciously or unconsciously, and we arrive at a diagnosis. REFLECTION is a key point to make sure that the right commander takes the helm.
Type II
intellect
Modified from Croskerry P. A universal model of diagnostic reasoning. Acad Med Aug;84(8):

21. Croskerry’s Dual-Process Model for Diagnostic Thinking (modified)
Type I
intuition
Rational Override
Pattern recognition
Diagnosis/ Management
Decision
Pattern processing
Practice
Dysrationalia Override
The goal of residency with practice and reflection is to build more and more of the heuristics from more complex decisional algorithms, shifting from Type II slow to type I fast thinking. That’s a good thing! Well, as long as we balance it out with an internal “Spock vs Kirk” debate.
Type II
intellect
Modified from Croskerry P. A universal model of diagnostic reasoning. Acad Med Aug;84(8):

22. How the Two Systems Interact
Rational Override
A Type 1 response is inappropriately triggered
The pattern isn’t matching the data bank
The decision maker stops and resets with a Type 2 analytical approach
We monitor Type I in most situations
Rational Override

23. Dysrationalia override aka residency, being on-call, etc.
“I should have known better!”
Sleep, Hunger
Fear, Irritability
Inattentiveness, Distractions
Prior experience
Dysrationalia Override

24. Balancing snap Judgments and deliberative rumination
Using Internalities, Checks and Balances
On Solutions
Balancing snap Judgments and deliberative rumination

25. Clustering of approaches on an intuitive-analytical continuum
Hypothetico-deductive reasoning
Inductive reasoning
Thin Slices
Exhaustion strategy
Blink = thin slicing and heuristics/biases methods are what medical educators/researchers focus on
Inductive vs Deductive reasoning for Detectives
Medical students and interns under stress, may revert to an exhaustion strategy
Disposition / Bias / Error
Type I
Type II
Croskerry, P. (2009) Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Advances in Health Sciences Education. 11 Aug :27–35

26. Reliability and Cost in Decision Making
We are training ourselves and our medical students and residents to become expert thinkers.
Graber M, et al. Reducing diagnostic errors in medicine: what's the goal? Acad Med Oct;77(10):

27. Checklists Checks and balances Croskerry’s General Diagnostic Time-Out
Heuristics & Biases
Differential Diagnosis
Be a Detective
I have a few simple solutions.

28. Croskerry's General Obtain your own complete medical History
Perform a focused and purposeful Physical Exam
Generate initial hypotheses and differentiate these with additional history, physical exam and diagnostic tests
Take a “Diagnostic Timeout”
Am I being comprehensive?
Am I considering the inherent flaws of heuristic thinking?
Was my judgment affected by any other bias?
Do I need to make the diagnosis now, or can I wait?
What is the worst-case scenario?
Embark on a plan, but acknowledge uncertainty and ensure a pathway for follow-up
“Crystal Ball Analysis”
If things goes awry, why?
What can you change?
For attendings and senior residents, this checklist is about proper oversight. Step back and think of yourself as a General on a battlefield rather than the frontline soldier.
Pausing will help with (Dysrationalia Overrides & allow for Calibration)

29. “Pre-mortem” M&M Halting Dysrationalia: Effortful deliberation
“Crystal Ball Analysis”
Assume your plan goes awry.
What happened?
Why?
What can you modify to avoid adverse events?
Klein, G. (2003) The Power of Intuition, 2003, p. 98–101

30. Croskerry’s Cognitive Biases and Failed Heuristics Checklist
Anchoring
Availability
Base-rate neglect
Premature closure
Representativeness Restraint
Unpacking Principle
Search Satisficing
Context Errors
Consider diagnoses other than the initially favored one
Consider diagnoses other than those that come readily to mind
Consider the relative prevalence of diseases in primary care that fit the patient’s complaint
Re-open the diagnostic process and consider alternative diagnoses before discharging the patient
Consider causes for symptoms other than the ones that readily fit the pattern
Ask questions that might confirm or rule out alternative diagnoses
Consider additional causes of the complaint after something is found
Re-think assumptions and maintain objectivity

32. Be a Detective Steps Heuristics Activated 1. Observe and deduce
Adjustment, Experiential Availability
2. Recognize inconsistencies
Representativeness Alert
3. Exhibit Tenacity
Avoid Unpacking Principle and Search Satisficing
4. Avoid protocols and procedures when a hunch leads in another direction
Ponder Representativeness restraint
5. Stop and think when necessary
Switch to System II
Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December ;24-31.

33. Mapping to ACGME Milestones for Critical Thinking
On Assessment
Formative feedback & Summative assessments of clinical reasoning in Medical School and residency

34. Milestones of Critical Thinking (Papp)
Stage 1: Unreflective Thinker
Stage 2: Beginning Critical Thinker
Stage 3: Practicing Critical Thinker
Stage 4: Advanced Critical Thinker
Stage 5: Accomplished Critical Thinker
Devolved state: the Challenged Thinker
Meta- cognition
Lacking knowledge about cognition
Unaware of different approaches to thinking
Requires external motivation to sustain reflection
Familiar with metacognitive theories
Applies conscious effort in his own critical thinking
Has a solid repertoire of approaches to thinking
Identifies different cognitive approaches
Consciously displays critical thinking
Uses theories of metacognition
Habitually monitors, revises, and rethinks approaches for continual improvement
Resists others’ perspectives, flouting prior knowledge of metacognition
Fails to recognize personal cognitive biases
Skills
Has a single approach to gathering and processing information based on crude scripts (e.g., rote memory)
Sporadically uses different approaches
Recognizes decision making principles but, disconnecting theory from practice, does not apply them in action
Articulates multiple approaches to problem solving
Uses established principles to make sense of observations and guide decisions
Uses intuitive and analytical strategies interchangeably
Avoids cognitive biases
Fundamental principles and concepts explicitly applied/demonstrated throughout approach to problem solving
Role-models critical thinking for others
Able to “toggle” adeptly between approaches
Elaborates on connections between principles to explain observed phenomena
Has the ability to create new knowledge or understanding by reasoning inductively
Entrenched in a singular approach
Does not adjust when appropriate or when aspects do not exactly fit the clinical situation
Adapted from Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720.

35. Milestones of Critical Thinking (Papp)
1: Unreflective Thinker
2: Beginning Critical Thinker
3: Practicing Critical Thinker
4: Advanced Critical Thinker
5: Accomplished Critical Thinker
Devolved state: the Challenged Thinker
Meta- cognition
Lacking knowledge about cognition
Unaware of different approaches to thinking
Requires external motivation to sustain reflection
Familiar with metacognitive theories
Applies conscious effort in his own critical thinking
Has a solid repertoire of approaches to thinking
Identifies different cognitive approaches
Consciously displays critical thinking
Uses theories of metacognition
Habitually monitors, revises, and rethinks approaches for continual improvement
Resists others’ perspectives, flouting prior knowledge of metacognition
Fails to recognize personal cognitive biases
Skills
Has a single approach to gathering and processing information based on crude scripts (e.g., rote memory)
Sporadically uses different approaches
Recognizes decision making principles but, disconnecting theory from practice, does not apply them in action
Articulates multiple approaches to problem solving
Uses established principles to make sense of observations and guide decisions
Uses intuitive and analytical strategies interchangeably
Avoids cognitive biases
Fundamental principles and concepts explicitly applied/demonstrated throughout approach to problem solving
Role-models critical thinking for others
Able to “toggle” adeptly between approaches
Elaborates on connections between principles to explain observed phenomena
Has the ability to create new knowledge or understanding by reasoning inductively
Entrenched in a singular approach
Does not adjust when appropriate or when aspects do not exactly fit the clinical situation
Maps to the following ACGME FM Milestones:
MK-2 Applies critical thinking skills in patient care
PBLI-1 Locates, appraises, and assimilates evidence from scientific studies related to the patients’ health problems
PBLI-2 Demonstrates self-directed learning
Also supports
PROF-4 Pursues continual personal and professional growth
Which Milestone have I reached?
Knowledge on metacognition, skills and attitudes (but I left this section which includes more PROF-4 items
Adapted from Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720.

36. Take Home Points Tricks: Traps: Checklists For education:
Heuristics are “Type I” shortcuts that “thin-slice” intuitive first impressions
Heuristics are excellent tools when checks and balances are in place
Traps:
Heuristics are prone to cognitive biases and therefore, subject to error
Use checklists to use avoid these pitfalls
Checklists
Rapezzis’ Call to “Be a Detective”
Croskerry’s General
Diagnostic Time-Out
Heuristics and Biases
Differential Diagnosis Checklist
For education:
Consider the positive dispositions case-discussion (“anti-M&M”)
For Assessment: Papp’s Milestones

37. References
Croskerry, P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv in Health Sci Educ (2009) 14:27–35
Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med Aug;78(8):
Croskerry P. A universal model of diagnostic reasoning. Acad Med Aug;84(8):
Ely JW, et al. Checklists to reduce diagnostic errors. Acad Med Mar;86(3):
Graber M, et al. Reducing diagnostic errors in medicine: what's the goal? Acad Med Oct;77(10):
Kahneman, D. Slovic P, Tversky A. (1982) Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.
Klein, G. (2003) The Power of Intuition, 2003, p. 98–101
Papp, K. et al. (2014) Milestones of Critical Thinking: A Developmental Model for Medicine and Nursing. Acad Med, 89(5), 715–720.
Rapezzi, C. et al. (2005) White coats and fingerprints: diagnostic reasoning in medicine and investigative methods of fictional detectives. BMJ December ;24-31.