1. Representativeness, Similarity, & Base Rate Neglect Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/27/2015: Lecture 05-1 Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Outline Base-rate neglect:  What is base-rate neglect?  How is base-rate neglect related to Bayesian reasoning Three types of evidence that people tend to ignore base-rate information. Similarity Heuristic: Base-rate neglect occurs because people often substitute a judgment of similarity for a judgment of probability (attribute substitution). Psych 466, Miyamoto, Aut '15 2 Representatitiveness Heuristic is an Example of Attribute Substitution

3. Representativeness Heuristic as Attribute Substitution Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge. ♦ E.g., when trying to judge a probability, people may substitute a judgment of availability or similarity. ♦ This is a common mental habit. People are often unaware that they are making this substitution. Psych 466, Miyamoto, Aut '15 3 Same Slide with Addition of Another Bulllet Point

4. Representativeness Heuristic as Attribute Substitution Attribute Substitution: When trying to evaluate a hard-to-judge attribute, people often substitute a judgment of a related attribute that is easy-to-judge. Similarity Thesis: People substitute a judgment of similarity for a judgment of probability. ♦ The conjunction fallacy – a consequence of similarity-based reasoning ♦ Insensitivity to sample size ♦ Insensitivity to regression effects ♦ Base rate neglect ♦ Misperceptions of randomness Psych 466, Miyamoto, Aut '15 4 Discussed Last Thursday. Discuss Today. Background to Study of Base Rate Neglect

5. What is "base-rate neglect"? ♦ "Base-rate": Another name for the prior probability of an individual event. ♦ "Base-rate neglect" has occurred if a person's judgments of probability are insufficiently influenced by variations in base-rate across different problems. Remainder of this Lecture: Bayesian Reasoning – what is it? What is a base-rate? Medical example of Bayesian reasoning Experimental evidence that people do not generally follow Bayes Rule when reasoning about probabilities Psych 466, Miyamoto, Aut '15 5 What is Bayesian Reasoning?

6. Psych 466, Miyamoto, Aut '15 6 What is Bayesian Reasoning? Bayesian Hypothesis: People reason as if uncertainty is measured by subjective probability. Possibly this is a normative claim. Possibly this is a descriptive claim. Bayesian theory accomplishes two things: 1.Tells us how our existing beliefs should adjust to new information. Tells us how our probabilities should change as we acquire new information. 2.Tells us how to adopt internally consistent gambling strategies. Tells us how to make optimal decisions relative to our values (utilities) and our subjective probabilities. Bayes Rule - What Is It? Why Is It Important? Bayes Rule

7. Bayes Rule – What Is It? Why Is It Important? Psych 466, Miyamoto, Aut '15 7 Reverend Thomas Bayes, 1702 – 1761 British clergyman & mathematician Bayes Rule is fundamentally important to: o Bayesian statistics o Bayesian decision theory o Bayesian models in psychology Explanation of Bayes Rule

8. Bayes Rule – Explanation (Look at Handout) Psych 466, Miyamoto, Aut '15 8 Derivation of the Odds Form of Bayes Rule Normalizing Constant Likelihood of the Data Posterior Probability of the Hypothesis Prior Probability of the Hypothesis a.k.a. base rate of the hypothesis

9. Psych 466, Miyamoto, Aut '15 9 Bayes Rule – Odds Form Bayes Rule for H given D Bayes Rule for not-H given D Odds Form of Bayes Rule Explanation of Odds form of Bayes Rule

10. H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 10 Memorable Form of the Bayes Rule (Odds Version) Posterior Odds Likelihood Ratio (diagnosticity) Prior Odds

11. H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 11 Interpretation of Medical Test Result Posterior Odds = (Likelihood Ratio) x (Prior Odds)

12. Psych 466, Miyamoto, Aut '15 12 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Preview of finding: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Medical Diagnosis Problem Next.

13. Psych 466, Miyamoto, Aut '15 13 Bayesian Analysis of a Medical Test Result (Look at Handout) QUESTION: A physician knows from past experience in his practice that 1% of his patients have cancer (of a specific type) and 99% of his patients do not have the cancer. He also knows the probabilities of a positive test result (+ result) given cancer and given no cancer. These probabilities are: P(+ test | Cancer) =.792andP(+ test | no cancer) =.096 Suppose Mr. X has a positive test result. What is the probability that Mr. X has cancer? P(Cancer) = Prior probability of cancer =.01 P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) =.99 Frequency-Based Solution to this problem Have class write down their answers

14. Psych 466, Miyamoto, Aut '15 14 Frequency-Based Analysis of the Medical Test Result Imagine that we have data for 1,000 patients. A 1% cancer rate means: 990 do not have cancer, and 10 have cancer (on the average) 8 of 10 patients with cancer will have a + test result (8  0.792*10). 95 of 990 patients without cancer will have a + test result (95  0.092*990). 103 = 8 + 95 : Out of 103 patients with + test results, 8 have cancer (8/103 = 0.077) and 95 do not have cancer (95/103 = 0.92). So if Mr. X has a + test result, he is much more likely to not have cancer than to have cancer. Math Analysis of the Medical Test Problem

15. Psych 466, Miyamoto, Aut '15 15 Mathematical Analysis of the Medical Test Result Mr. X has a + test result. What is the probability that Mr. X has cancer? P(+ test | Cancer) =.792andP(+ test | no cancer) =.096 P(Cancer) = Prior probability of cancer =.01 P(No Cancer) = Prior probability of no cancer = 1 - P(Cancer) =.99 P(Cancer | + test) = 1 / (12 + 1) = 0.077 David Eddy: Fallacious Reasoning by Physicians re Interpretation of Test Result Bayes Rule

16. Psych 466, Miyamoto, Aut '15 16 Fallacious Reasoning by Physicians Correct answer given the data (based on Eq. (4)) : P(Cancer | + Result) = (.792)(.01)/(.103) =.077 Notice: The test is very diagnostic but still P(cancer | + result) is low because the base rate is low. David Eddy found that about 95% of physicians think that P(cancer | +result) is about 75% in this case (very close to the 79% likelihood of a + result given cancer). Physicians sometimes overlook base rates! Bad Advice from a Medical Textbook

17. Psych 466, Miyamoto, Aut '15 17 Fallacious Probabilistic Reasoning in Medicine DeGowin & DeGowin. Bedside diagnostic examination. "When a patient consults his physician with an undiagnosed disease, neither he nor the doctor knows whether it is rare until the diagnosis is finally made. Statistical methods can only be applied to a population of thousands. The individual either has a rare disease or doesn't have it; the relative incidence of two diseases is completely irrelevant to the problem of making his diagnosis." Eddy documents many examples in medical literature of confusion between P(cancer | +result) and P(+ result | cancer). General Characteristics of Bayesian Inference DeGowin & DeGowin say: Ignore the prior odds!

18. Psych 466, Miyamoto, Aut '15 18 General Characteristics of Bayesian Inference The decision maker (DM) is willing to specify the prior probability of the hypotheses of interest. DM can specify the likelihood of the data given each hypothesis. Using Bayes Rule, we infer the probability of the hypotheses given the data. Main Point of Cognitive Critique: People tend to ignore base rates. People do not use Bayes Rule to update their probabilities. Three Types of Empirical Violations of Bayes Rule Given Information

19. Psych 466, Miyamoto, Aut '15 19 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Preview of finding: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Medical Diagnosis Problem Previous Topic. Next Topic.

20. Psych 466, Miyamoto, Aut '15 20 Problem #4: Tom W (K&T, 1973) PERSONALITY DESCRIPTION OF TOM W: Tom W. is currently a graduate student at the University of Washington. Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense. Response Sheet

21. Psych 466, Miyamoto, Aut '15 21 Analysis of Problem #4: Tom W (cont.) Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks. EstimatedSimilarityProbability Base RateRankRankGrad specialization _______________Bus AD _______________Computer Science _______________Engineering _______________Humanities _______________Law _______________Library Science _______________Medicine _______________Physical Science _______________Social Science Same Slide without Emphasis Rectangles Condition 3 Condition 2 Condition 1

22. Psych 466, Miyamoto, Aut '15 22 Analysis of Problem #4: Tom W (cont.) Between-subjects design. Separate groups of subjects judged (i) base rate, (ii) similarity ranks, (iii) probability ranks. EstimatedSimilarityProbability Base RateRankRankGrad specialization _______________Bus AD _______________Computer Science _______________Engineering _______________Humanities _______________Law _______________Library Science _______________Medicine _______________Physical Science _______________Social Science Use Bayes Rule to Analyze Tom W Problem

23. Relative Prevalence of Different Grad Specializations People fail to take this into account. Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 23 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Results for Tom W Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

24. Psych 466, Miyamoto, Aut '13 24 Tom W Problem - Results Correlation between judged base rate and likelihood rank = .63. Correlation between similarity rank and likelihood rank = +.97. "Likelihood" is just another word for "probability" and here it refers to the posterior probability of each of the grad specializations. Repeat: Tom W & Bayes Rule

25. Relative Prevalence of Different Grad Specializations People fail to take this into account! Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 25 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Conclusions re Tom W Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

26. Similarity heuristic is one part of the representativeness heuristic. The judgment of probability ought to be base in part on a judgment of similarity. ( Similarity determines the diagnosticity of the evidence. ) Focusing on similarity should not cause one to overlook the relevance of base rate. Psych 466, Miyamoto, Aut '15 26 Similarity Heuristic Predicts Tom W Results Repeat: Bayes Rule Odds Form with Annotation for Components People substitute a judgment of similarity,.... How much is Tom like the stereotype of a computer science student?... for a judgment of probability What is the probability that Tom is a computer science student?

27. Relative Prevalence of Different Grad Specializations People fail to take this into account! Tom W & Bayes Rule Psych 466, Miyamoto, Aut '15 27 D = Data = the personality description of Tom W H = Hypothesis that Tom W is a ____ student, e.g., an engineering student Return to ThreeTypes of Empirical Violations of Bayes Rule Probability of Grad Specialization H Given Tom W's Description Similarity of H to D i.e., similarity of stereotype H to Tom W's Description

28. Psych 466, Miyamoto, Aut '15 28 Tuesday, October 27, 2015 : The Lecture Ended Here

29. Psych 466, Miyamoto, Aut '15 29 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Tom W: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Lawyer/Engineer Problem Previous Topic. Next Topic.