1. Bayesian Reasoning & Base-Rate Neglect
Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/19/2017: Lecture 04-2
Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Outline
Bayesian reasoning - what is it? Bayes Theorem - what does it say?
Base-rate neglect:
What is base-rate neglect?
Base-rate neglect is a fundamental violation of Bayesian reasoning.
Evidence that people tend to ignore base-rate information.
Similarity Heuristic:
Base-rate neglect occurs because people often substitute a judgment of similarity for a judgment of probability.
(This is an example of attribute substitution).
Psych 466, Miyamoto, Aut '17
What is Bayesian Reasoning?

3. What is Bayesian Reasoning?
Bayesian Hypothesis: People reason as if uncertainty is measured by subjective probability.
For most scholars, this is a normative claim.
For some scholars, this is a descriptive claim.
Bayesian theory accomplishes two things:
Tells us how our existing beliefs should adjust to new information. Tells us how our probabilities should change as we acquire new information.
Tells us how to adopt optimal gambling strategies. Tells us how to make optimal decisions relative to our values and subjective probabilities.
Bayes Rule
Bayes Rule - What Is It? Why Is It Important?
Psych 466, Miyamoto, Aut '17

4. Bayes Rule – What Is It? Why Is It Important?
Reverend Thomas Bayes, 1702 – 1761 British clergyman & mathematician
Bayes Rule is fundamentally important to:
Bayesian statistics
Bayesian decision theory
Bayesian models in psychology
Explanation of Bayes Rule
Psych 466, Miyamoto, Aut '17

5. Bayes Rule – Explanation (Look at Handout)
Posterior Probability of the Hypothesis
Likelihood of the Data
Prior Probability of the Hypothesis
a.k.a. base rate of the hypothesis
Normalizing Constant
Derivation of the Odds Form of Bayes Rule
Psych 466, Miyamoto, Aut '17

6. Bayes Rule – Odds Form Bayes Rule for H given D
Bayes Rule for not-H given D
Odds Form of Bayes Rule
Explanation of Odds form of Bayes Rule
Psych 466, Miyamoto, Aut '17

7. Likelihood Ratio (diagnosticity)
Bayes Rule (Odds Form)
Posterior Odds
Likelihood Ratio (diagnosticity)
Prior Odds
H = a hypothesis,
e.g., H = hypothesis that the patient has cancer
~H = the negation of the hypothesis,
e.g., ~H = hypothesis that the patient does not have cancer
D = the data,
e.g., D = + test result for a cancer test
Psych 466, Miyamoto, Aut '17
Memorable Form of the Bayes Rule (Odds Version)

8. Posterior Odds = (Likelihood Ratio) x (Prior Odds)
Bayes Rule (Odds Form)
Posterior Odds = (Likelihood Ratio) x (Prior Odds)
H = a hypothesis,
e.g., H = hypothesis that the patient has cancer
~H = the negation of the hypothesis,
e.g., ~H = hypothesis that the patient does not have cancer
D = the data,
e.g., D = + test result for a cancer test
Psych 466, Miyamoto, Aut '17
Violations of Bayes Rule Are Important Fact of Human Decision Making

9. Violations of Bayes Rule in Intuitive Thinking
Intuitive violations of Bayes Rule show that rational agent model is not descriptively correct.
Intuitive violations of Bayes Rule can cause serious errors in real-world decision making.
Psych 466, Miyamoto, Aut '17
Three Types of Violations of Bayes Rule

10. Three Types of Empirical Violations of Bayes Rule
Next .
1. Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule.
Example: Physicians judgments of P( +Cancer | +Test Result) exhibit base-rate neglect
2. Judging probability in the context of widely-known base rates.
Preview of finding: Judgments ignore well-known base rates.
3. Judging probability when base rates are experimentally manipulated between subjects.
Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high.
Medical Diagnosis Problem
Psych 466, Miyamoto, Aut '17

11. Bayesian Analysis of a Medical Test Result (Look at Handout)
QUESTION: A physician knows from past experience in his practice that 1% of his patients have cancer (of a specific type) and 99% of his patients do not have the cancer, i.e.,
P(Cancer) = (prior probability of cancer)
P(No Cancer) = 1 - P(Cancer) = (prior probability of no cancer)
He also knows that:
P(+ test | Cancer) = and P(+ test | no cancer) = .096
Suppose Mr. X has a positive test result. What is the probability that Mr. X has cancer?
Have class write down their answers
Analysis of Medical Test Result Using Bayes Rule
Psych 466, Miyamoto, Aut '17

12. Bayesian Analysis of the Medical Test Result
Mr. X has a + test result. What is the probability that Mr. X has cancer?
P(+ test | Cancer) = and P(+ test | no cancer) = .096
P(Cancer) = Prior probability of cancer = .01
P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) = .99
P(Cancer | + test) = 1 / (12 + 1) =
Bayes Rule
Psych 466, Miyamoto, Aut '17
David Eddy: Fallacious Reasoning by Physicians re Interpretation of Test Result

13. Fallacious Reasoning by Physicians
Correct answer (based on Eq. (4)):
P(Cancer | + Result) = (.792)(.01)/(.103) = .077
Notice: The test is very diagnostic but still P(cancer | + result) is low because the base rate is low.
David Eddy found that about 95% of physicians thought that P(cancer | +result) is about 75% in this case
(75% is very close to the 79% likelihood of a + result given cancer).
Conclusion: Physicians sometimes overlook base rates!
Bad Advice from a Medical Textbook
Psych 466, Miyamoto, Aut '17

14. Fallacious Probabilistic Reasoning in Medicine
DeGowin & DeGowin. Bedside diagnostic examination. "When a patient consults his physician with an undiagnosed disease, neither he nor the doctor knows whether it is rare until the diagnosis is finally made. Statistical methods can only be applied to a population of thousands. The individual either has a rare disease or doesn't have it; the relative incidence of two diseases is completely irrelevant to the problem of making his diagnosis." [underlining by JM]
Eddy documents many examples in medical literature of confusion between P(cancer | +result) and P(+ result | cancer).
DeGowin & DeGowin say: Ignore the prior odds!
Review Bayes Rule (Odds Form) - Relationship to Medical Test Problem
Psych 466, Miyamoto, Aut '17

15. Bayes Rule (Odds Form) Posterior Odds of Cancer
Likelihood Ratio of +Test Result
Prior Odds of Cancer
Psych 466, Miyamoto, Aut '17
Same Slide with Comments Regarding Medical Test Problem

16. Bayes Rule (Odds Form) Applied to Medical Test Problem
Posterior Odds of Cancer
Likelihood Ratio of +Test Result
Prior Odds of Cancer
Base-Rate Neglect Posterior odds are strongly influenced by likelihood ratio. Prior odds have little or no influence.
Diagnosticity is based on similarity. Similarity has a strong inffluence on posterior odds.
Sometimes prior odds have very lilttle influence on posterior odds.
Psych 466, Miyamoto, Aut '17
Return to Slide with Three Types of Violations of Bayes Rule

17. Three Types of Empirical Violations of Bayes Rule
Previous Topic .
1. Given statistical information about prior probabilities and likelihoods, judgments of posterior probability seriously violate Bayes Rule.
Example: Physicians judgments of P( +Cancer | +Test Result) are much too high
2. Judging probability in the context of widely-known base rates.
Preview of finding: Judgments ignore well-known base rates.
3. Judging probability when base rates are experimentally manipulated between subjects.
Preview of finding: Judgments are approximately the same in conditions that differ only in base-rate.
Next Topic .
Tom W Problem
Psych 466, Miyamoto, Aut '17

18. Problem #4: Tom W (K&T, 1973) PERSONALITY DESCRIPTION OF TOM W:
Tom W. is currently a graduate student at the University of Washington. Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense.
Response Sheet
Psych 466, Miyamoto, Aut '17

19. Analysis of Problem #4: Tom W (cont.)
Between-subjects design. Condition 1: judge base rate. Condition 2: judge similarity, Condition 3: judge probability.
Estimated Similarity Probability Base Rate Rank Rank Grad specialization
_____ _____ _____ Bus AD
_____ _____ _____ Computer Science
_____ _____ _____ Engineering
_____ _____ _____ Humanities
_____ _____ _____ Law
_____ _____ _____ Library Science
_____ _____ _____ Medicine
_____ _____ _____ Physical Science
_____ _____ _____ Social Science
Condition 1
Condition 2
Condition 3
Same Slide without Emphasis Rectangles
Psych 466, Miyamoto, Aut '17

20. Analysis of Problem #4: Tom W (cont.)
Between-subjects design. Condition 1: judge base rate. Condition 2: judge similarity, Condition 3: judge probability.
Estimated Similarity Probability Base Rate Rank Rank Grad specialization
_____ _____ _____ Bus AD
_____ _____ _____ Computer Science
_____ _____ _____ Engineering
_____ _____ _____ Humanities
_____ _____ _____ Law
_____ _____ _____ Library Science
_____ _____ _____ Medicine
_____ _____ _____ Physical Science
_____ _____ _____ Social Science
Tom W and Bayes Rule
Psych 466, Miyamoto, Aut '17

21. Tom W and Bayes Rule D = Data = the personality description of Tom W
H = Hypothesis that Tom W is a ____ student, e.g., an engineering student
Probability of Grad Specialization H Given Tom W's Description
Similarity of H to D i.e., similarity of stereotype H to Tom W's Description
Relative Prevalence of Different Grad Specializations
People fail to take this into account.
Results for Tom W
Psych 466, Miyamoto, Aut '17

22. Tom W Problem - Results
"Likelihood" is just another word for "probability" and here it refers to the posterior probability of each of the grad specializations.
See ‘e:\p466\nts\tomw results table.docm’ for computation of correlation. Similarity & likelihood are reverse coded.
Correlation between judged base rate and likelihood rank = Correlation between similarity rank and likelihood rank = +.97.
Repeat: Tom W & Bayes Rule
Psych 466, Miyamoto, Aut '17

23. People fail to take this into account!
Tom W & Bayes Rule
D = Data = the personality description of Tom W
H = Hypothesis that Tom W is a ____ student, e.g., an engineering student
Probability of Grad Specialization H Given Tom W's Description
Similarity of H to D i.e., similarity of stereotype H to Tom W's Description
Relative Prevalence of Different Grad Specializations
People fail to take this into account!
Conclusions re Tom W
Psych 466, Miyamoto, Aut '17

24. Similarity Heuristic Predicts Tom W Results
Similarity heuristic is one part of the representativeness heuristic.
The judgment of probability ought to be based in part on a judgment of similarity. (Similarity determines the diagnosticity of the evidence.)
Focusing on similarity should not cause one to overlook the relevance of base rate.
People substitute a judgment of similarity, ....
How similar is Tom to the stereotype of a computer science student?
... for a judgment of probability
What is the probability that Tom is a computer science student?
Repeat: Bayes Rule Odds Form with Annotation for Components
Psych 466, Miyamoto, Aut '17

25. People fail to take this into account!
Tom W & Bayes Rule
D = Data = the personality description of Tom W
H = Hypothesis that Tom W is a ____ student, e.g., an engineering student
Probability of Grad Specialization H Given Tom W's Description
Similarity of H to D i.e., similarity of stereotype H to Tom W's Description
Relative Prevalence of Different Grad Specializations
People fail to take this into account!
Return to ThreeTypes of Empirical Violations of Bayes Rule
Psych 466, Miyamoto, Aut '17

26. Three Types of Empirical Violations of Bayes Rule
1. Given statistical information about prior probabilities and likelihoods, judgments of posterior probability seriously violate Bayes Rule.
Example: Physicians judgments of P( +Cancer | +Test Result) are much too high
2. Judging probability in the context of widely-known base rates.
Preview of finding: Judgments ignore well-known base rates.
3. Judging probability when base rates are experimentally manipulated between subjects.
Preview of finding: Judgments are approximately the same in conditions that differ only in base-rate.
Previous Topic
Next Topic .
Lawyer/Engineer Problem
Psych 466, Miyamoto, Aut '17

27. Lawyer/Engineer Problem (K&T, 1973)
DESCRIPTION OF JACK: Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles (Some subjects saw other versions of this description.)
Question with High Base Rate for Engineer (High Prior Odds): If Jack's description were drawn at random from a set of 30 descriptions of lawyers and 70 descriptions of engineers, what would be the probability that Jack is one of the engineers?
Question with Low Base Rate for Engineer (Low Prior Odds): If Jack's description were drawn at random from a set of 70 descriptions of lawyers and 30 descriptions of engineers, what would be the probability that Jack is one of the engineers?
Bayes Rule Analysis of Lawyer/Engineer Problem
Psych 466, Miyamoto, Aut '17

28. Bayesian Analysis of Lawyer/Engineer Problem
High Base Rate Condition
70 Engineers, 30 Lawyers Ratio is 7 : 3
Prior odds are large
Low Base Rate Condition
30 Engineers, 70 Lawyers Ratio is 3 : 7
Prior odds are small
Psych 466, Miyamoto, Aut '17
Likelihood Ratio Should be the Same for the Low and High Base-Rate Conditions

29. Bayesian Analysis of Lawyer/Engineer Problem
Likelihood Ratio (Diagnosticity of Evidence)
This is the same for the low and high base rate conditions.
Posterior Odds Should Be Affected Only by Prior Odds
Psych 466, Miyamoto, Aut '17

30. Bayesian Analysis of Lawyer/Engineer Problem
Posterior Odds
Should be lower in the low base-rate condition than in the high base-rate condition.
Likelihood Ratio (Diagnosticity of Evidence)
This is the same for the low and high base rate conditions.
Prior Odds
Is lower in the low base rate condition than in the high base-rate condition.
Psych 466, Miyamoto, Aut '17
Results for the Lawyer/Engineer Problem

31. Lawyer/Engineer Problem - Results
X-axis: Judged P(eng | desc) in Low Base Rate Condition
Y-axis: Judged P(eng | desc) in High Base Rate Condition
Each dot is the result for a different description – some are strongly suggestive of engineer; others are strongly suggestive of lawyer.
#Section: Hidden: bk c
#First: --- Hidden: bk G |
# plot
par( mar = c(6, 5, 4, 2) )
pLo = seq(0, 1, by = .005)
pHigh = (5.44 * pLo) / ( * pLo)
pLo = 100*pLo
pHigh = 100*pHigh
plot.jm( pLo, pHigh, xlim = c(0, 100), ylim = c(0, 100))
lines( pLo, pHigh, lwd = 2, lty = 2 )
lines( pLo, pLo, lwd = 2 )
mtext("Probability of Engineer (Low Prior)", side = 1, cex = 2, line = 3 )
mtext("Probability of Engineer (High Prior)", side = 2, cex = 2, line = 2.75 )
kt.data = matrix( c(
5, 5,
30.5, 40.2,
86, 90,
98.8, 94.8
), ncol = 2, byrow = T )
ceX = 3
points( kt.data, pch = 16, cex = ceX )
points( 50, 50, pch = 21, cex = ceX, col = "blue", lwd = 4 )
points( 30.2, 70, pch = 22, cex = ceX, col = "red", lwd = 4 )
mtext("Lawyer/Engineer Problem", side = 3, cex = 2, line = 1 )
#Last: End Hidden: bk G1 ----
#EndSection: Hidden: bk c1 ----
Dotted Curve is the Predicted Response for a Bayesian Reasoner
Psych 466, Miyamoto, Aut '17

32. Lawyer/Engineer Problem - Results
Curve is the correct response for Bayes Rule.
Diagonal is the predicted response if base rate is ignored.
Discuss 2 odd points on this graph
Psych 466, Miyamoto, Aut '17

33. Lawyer/Engineer Problem - Results
Subjects were given no description about Jack. They were only told that sampling of names at random from a 30:70 or 70:30 pile had occurred.
Psychologically: 'No Information'  'Worthless, Irrelevant Information'
Subjects were given an uninformative description about Jack, e.g., Jack is 53 years old and is married.
Conclusions re Lawyer/Engineer Problem
Psych 466, Miyamoto, Aut '17

34. Conclusions re Lawyer/Engineer Problem
Subjects were insensitive to base-rate information (30:70 versus 70:30).
Subjects showed awareness of base-rate when no description of Jack was given. (So base-rate is not treated as always irrelevant.)
Subjects ignored base-rate when irrelevant information about Jack was given.
Psychologically: 'No Information'  'Worthless, Irrelevant Information'
Summary of Evidence for the Similarity Thesis
Psych 466, Miyamoto, Aut '17

35. Representativeness Heuristic & the Similarity Hypothesis
Similarity Hypothesis: When asked to judge a probability, people often substitute a judgment of similarity for the judgment of probability.
Lawyer/Engineer Problem: People judge P(eng | desc) based on similarity of Jack’s description to a typical engineer Subjects fail to use base rate information because it is not relevant to the similarity between the personality description and the stereotype of a lawyer or engineer.
Continuation of this slide
Psych 466, Miyamoto, Aut '17

36. Objections to the Claim – People Neglect Base Rates
Distinguish between two claims:
People often/always neglect base rates in real world decisions.
People base probability judgments on similarity and this causes them to neglect base rates in particular situations.
Subjects are unmotivated – nothing is at stake in their judgments
Replications with large monetary payoffs for being accurate
Base rate neglect in market investment decisions
Claim: Correct use of base rates is found if uncertainty is expressed as frequencies rather than probabilities (Gigerenzer & Hoffrage, 1995; Sloman, Over, Slovak, & Stibel, 2003)
Evidence supports the view that expressing uncertainty as frequencies increases people’s tendency to use base rates, but usually only to a small degree.
JM’s Bottom Line: Base-rate neglect occurs in some situations (not always); it can have practical importance; it demonstrates the central role of similarity in probability judgment.
Definition of the Inside View and Outside View of Decision Problem
Psych 466, Miyamoto, Aut '17

37. The Inside View and the Outside View
"Inside View" and "Outside View": Two different ways to look at a probability judgment problem.
Inside View: Look closely at the particular case. What do you see that would guide you towards a prediction?
Outside View: Place the particular case in the context of many other similar cases. What usually happens in this population of similar cases?
Psych 466, Miyamoto, Aut '17
Inside View vs Outside View in Planning Judgments

38. Inside View vs Outside View in Planning Judgments
Think about your plans. What do you need to get this job done?
How long will each component take? Extrapolate to the entire project.
Outside View:
Think about your past experience with projects that are similar in complexity. Do people complete these projects on time?
What has caused delays in the past?
Is the current project likely to be different from past experiences with similar projects?
Psych 466, Miyamoto, Aut '17
Inside View vs Outside View in the Lawyer/Engineer Problem

39. Thursday, 19 October, 2017: The Lecture Ended Here
Psych 466, Miyamoto, Aut '17