1. Base Rate Neglect (continued) Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/29/2015: Lecture 05-2 Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Outline Experimental manipulation of base rate information. Another demonstration of base rate neglect. The inside view and the outside view. Attribute substitution in probability judgment. Psych 466, Miyamoto, Aut '15 2

3. Bayes Rule – What Is It? Why Is It Important? Psych 466, Miyamoto, Aut '15 3 Reverend Thomas Bayes, 1702 – 1761 British clergyman & mathematician Bayes Rule is fundamentally important to: o Bayesian statistics o Bayesian decision theory o Bayesian models in psychology Explanation of Bayes Rule

4. H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 4 Memorable Form of the Bayes Rule (Odds Version) Posterior Odds Likelihood Ratio (diagnosticity) Prior Odds

5. H = a hypothesis, e.g., H = hypothesis that the patient has cancer = the negation of the hypothesis, e.g., ~H = hypothesis that the patient does not have cancer D = the data, e.g., D = + test result for a cancer test Bayes Rule (Odds Form) Psych 466, Miyamoto, Aut '15 5 Interpretation of Medical Test Result Posterior Odds = (Likelihood Ratio) x (Prior Odds)

6. Psych 466, Miyamoto, Aut '15 6 Three Types of Empirical Violations of Bayes Rule Given statistical information about prior probabilities and likelihoods, people's judgments of posterior probability seriously violate Bayes Rule. ♦ Example: Physicians judgments of P( +Cancer | +Test Result) are much too high Judging probability in the context of widely-known base rates. ♦ Tom W: Judgments ignore well-known base rates. Judging probability when base rates are experimentally manipulated between subjects. ♦ Preview of finding: Judgments are approximately the same in conditions where the base rate is low or where it is high. Lawyer/Engineer Problem Previous Topic. Next Topic. Previous Topic..

7. Psych 466, Miyamoto, Aut '15 7 Lawyer/Engineer Problem (K&T, 1973) DESCRIPTION OF JACK: Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles. (Some subjects saw other versions of this description.) ♦ Question with High Base Rate for Engineer (High Prior Odds): If Jack's description were drawn at random from a set of 30 descriptions of lawyers and 70 descriptions of engineers, what would be the probability that Jack is one of the engineers? ♦ Question with Low Base Rate for Engineer (Low Prior Odds): If Jack's description were drawn at random from a set of 70 descriptions of lawyers and 30 descriptions of engineers, what would be the probability that Jack is one of the engineers? Bayes Rule Analysis of Lawyer/Engineer Problem

8. Bayesian Analysis of Lawyer/Engineer Problem Psych 466, Miyamoto, Aut '15 8 Results for Lawyer/Engineer Problem High Base Rate Condition 30 Lawyers, 70 Engineers Prior Odds of Engineer = 70/30 This is large Low Base Rate Condition 70 Lawyers, 30 Engineers Prior Odds of Engineer = 30/70 This is small

9. Psych 466, Miyamoto, Aut '15 9 Bayesian Analysis of Lawyer/Engineer Problem Notice that is the same in high base rate condition and low base rate condition. Therefore the difference in the posterior probabilities in the two conditions should be influenced only by the prior odds: Results for Lawyer/Engineer Problem

10. Psych 466, Miyamoto, Aut '15 10 Lawyer/Engineer Problem - Results X-axis: Judged P(eng | desc) in Low Base Rate Condition Y-axis: Judged P(eng | desc) in High Base Rate Condition Each dot is the result for a different description – some are strongly suggestive of engineer; others are strongly suggestive of lawyer. Dotted Curve is the Predicted Response for a Bayesian Reasoner

11. Psych 466, Miyamoto, Aut '15 11 Lawyer/Engineer Problem - Results Curve is the correct response for Bayes Rule. Diagonal is the predicted response if base rate is ignored. Data look more like the base rate neglect hypothesis than the correct (normative) response. Discuss 2 odd points on this graph

12. Psych 466, Miyamoto, Aut '15 12 Lawyer/Engineer Problem - Results Subjects were given no description about Jack. They were only told that random sampling of names from a 30:70 or 70:30 pile had occurred. Subjects were given an uninformative description about Jack, e.g., Jack is 53 years old and is married. Conclusions re Lawyer/Engineer Problem Psychologically: 'No Information'  'Worthless, Irrelevant Information'

13. Psych 466, Miyamoto, Aut '15 13 Conclusions re Lawyer/Engineer Problem Subjects were insensitive to base-rate information (30:70 versus 70:30). Subjects showed awareness of base-rate when no description of Jack was given. (So base-rate is not treated as always irrelevant.) Subjects ignored base-rate when irrelevant information about Jack was given. ♦ Psychologically: 'No Information'  'Worthless, Irrelevant Information' Summary of Evidence for the Similarity Thesis

14. Psych 466, Miyamoto, Aut '15 14 Representativeness Heuristic & the Similarity Hypothesis Similarity Hypothesis: When asked to judge a probability, people often substitute a judgment of similarity for the judgment of probability. Lawyer/Engineer Problem: People judge P(eng | desc) based on similarity of Jack’s description to a typical engineer. Subjects fail to use base rate information because it is not relevant to similarity. Bayes Rule Says that We Should Combine Similarity Information with Distributional Information

15. Psych 466, Miyamoto, Aut '15 15 Why Do People Fail to Use Base Rate Information? Bayes Rule says that we should combine base-rates with case- specific, descriptive information (diagnosticity of the data). Psychologically, we don't have a reasoning strategy for combining these two types of information, so we use one or the other. Objections to K&T claims – End of Base Rate Neglect Topic

16. Psych 466, Miyamoto, Aut '15 16 Objections to the Claim – People Neglect Base Rates Distinguish between two claims: ♦ People often/always neglect base rates in real world decisions. ♦ People base probability judgments on similarity and this causes them to neglect base rates in particular situations. Subjects are unmotivated – nothing is at stake in their judgments ♦ Replications with large monetary payoffs for being accurate ♦ Base rate neglect in market investment decisions Claim: Correct use of base rates is found if uncertainty is expressed as frequencies rather than probabilities. (Gigerenzer & Hoffrage, 1995; Sloman, Over, Slovak, & Stibel, 2003) ♦ Evidence supports the view that expressing uncertainty as frequencies increases people’s tendency to use base rates, but usually only to a small degree. Inside View and Outside View

17. The Inside View and the Outside View Inside View: Look closely at the particular case. What do you see that would guide you towards a prediction? Outside View: Place the particular case in the context of many other similar cases. What usually happens in the population of similar cases? Psych 466, Miyamoto, Aut '15 17 Frequency Format for the Lawyer/Engineer Problem

18. Example: Outside View of the Lawyer/Engineer Problem Psych 466, Miyamoto, Aut '15 18 High Base Rate Condition Low Base Rate Condition 30 Lawyers 80 Engineers Sounds like Jack 70 Lawyers 30 Engineers Sounds like Jack Outside View (Frequency Format) of the Lawyer/Engineer Problem

19. Psych 466, Miyamoto, Aut '15 19 Lawyer/Engineer Problem – Frequency Format (Outside View) DESCRIPTION OF JACK (SAME AS BEFORE): ♦ Question with High Base Rate for Engineer (High Prior Odds): Imagine a mixed pile of 1,000 personality descriptions; 700 are descriptions of engineers and 300 are descriptions of lawyers. How many engineers sound like Jack? How many lawyers sound like Jack? What is the probability that Jack is one of the engineers. ♦ Question with Low Base Rate for Engineer (Low Prior Odds): Imagine a mixed pile of 1,000 personality descriptions; 300 are descriptions of engineers and 700 are descriptions of lawyers. How many engineers sound like Jack? How many lawyers sound like Jack? What is the probability that Jack is one of the engineers. The Outside View focuses on the entire distribution of possibilities. How often will a particular outcome (he’s an engineer) occur in this distribution? Inside View of the Lawyer/Engineer Problem

20. Psych 466, Miyamoto, Aut '15 20 Lawyer/Engineer Problem (K&T, 1973) DESCRIPTION OF JACK: Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies which include home carpentry, sailing, and mathematical puzzles. (Some subjects saw other versions of this description.) ♦ Question with High Base Rate for Engineer (High Prior Odds): If Jack's description were drawn at random from a set of 30 descriptions of lawyers and 70 descriptions of engineers, what would be the probability that Jack is one of the engineers? ♦ Question with Low Base Rate for Engineer (Low Prior Odds): If Jack's description were drawn at random from a set of 70 descriptions of lawyers and 30 descriptions of engineers, what would be the probability that Jack is one of the engineers? Inside View vs Outside View in Planning Judgments

21. Inside View: ♦ Think about your plans. What do you need to get this job done? ♦ How long will each component take? Extrapolate to the entire project. Outside View: ♦ Think about your past experience with projects that are similar in complexity. Do people complete these projects on time? ♦ What has caused delays in the past? ♦ Is the current project likely to be different from past experiences with similar projects? Psych 466, Miyamoto, Aut '15 21 Attribute Substitution

22. Psych 466, Miyamoto, Aut '15 22 Attribute Substitution in Probability Judgment Target attribute: Probability of an event, ♦ E.g., Will the U.S. have a large military presence in Afghanistan in 2020? Heuristic attribute: Easy-to-judge attribute that is related to the probability of the event. ♦ E.g., similarity to other political situations or availability of analogous political situations. ♦ A.k.a. the “proxy” attribute. Hypothesis that motivated Kahneman & Tversky’s research: People substitute similarity or availability for probability. ♦ Similarity & availability – System 1 ♦ Probability theory – System 2 When Does Attribute Substitution Occur?

23. Psych 466, Miyamoto, Aut '15 23 When is Attribute Substitution Likely to Occur? (K&F) K&F: Attribute substitution is more likely when: 1.the target attribute is relatively inaccessible, i.e., hard to evaluate or unfamiliar; 2.a semantically and associatively related attribute is highly accessible (heuristic attribute); 3.the substitution of the heuristic attribute in the judgment is not rejected by critical operations of System 2. Repeat Slide with Examples of Attribute Substitution

24. Psych 466, Miyamoto, Aut '15 24 When is Attribute Substitution Likely to Occur? (K&F) K&F: Attribute substitution is more likely when: 1.the target attribute is relatively inaccessible, i.e., hard to evaluate or unfamiliar; 2.a semantically and associatively related attribute is highly accessible (heuristic attribute); 3.the substitution of the heuristic attribute in the judgment is not rejected by critical operations of System 2. Examples of common attribute substitutions: ♦ Habitual substitution of similarity for probability (representativeness) ♦ Habitual substitution of availability for probability (availability) ♦ Habitual substitution of fluency for recollective memory processes ♦ Affective response for more cognitive evaluations like worth or virtue. Recipe for Demonstrating Attribute Substitution: Method I

25. Psych 466, Miyamoto, Aut '15 25 How to Demonstrate Occurrence of Attribute Substitution Method I Collect judgment data for the heuristic attribute and the target attribute. Show that judgments of the heuristic attribute are highly correlated with judgments of the target attribute. The demonstration is especially strong when there are other objective considerations that should reduce the correlation between the heuristic attribute and the target attribute. Tom W problems exemplifies this strategy

26. Psych 466, Miyamoto, Aut '15 26 Tom W Problem - Results Correlation between judged base rate and probability rank = -.65. Correlation between similarity rank and probability rank = +.97. Tom W and Bayes Rule

27. Psych 466, Miyamoto, Aut '15 27 Graph of Tom W Results Heuristic attribute (similarity) and target attribute (probability) are highly correlated (+.97). Rule-based System 2 requires attention to base rate. Subjects appear to disregard base rate (correlation = -.65). How to Demonstrate Occurrence of Attribute Substitution: Method II Similarity Rank Rank of Judged Base Rate Rank of Judged Probability Similarity versus Judged Probability Base Rate versus Judged Probability

28. Psych 466, Miyamoto, Aut '15 28 How to Demonstrate Occurrence of Attribute Substitution Method II Experimentally manipulate the strength of the heuristic attribute. Show that judgments of the target attribute are influenced in the predicted direction by variation of the heuristic attribute. Again, the demonstration is especially strong when there are other objective considerations that should reduce the correlation between the heuristic attribute and the target attribute. Ease of Recall & Availability Exemplifies this Method

29. Example: Ease of Recall Affects Judgment Making it easy to remember examples where subject has been assertive makes subject think she is more assertive. ♦ Easier to recall 6 examples than 12 examples. ♦ Easier to recall examples when smiling than when frowning. Target attribute: Evaluation of how assertive you are. Heuristic attribute: Evaluation of how easy it is to recall the examples of being assertive. In this example, the experimenter manipulates the heuristic attribute (makes recall easier or harder for different subjects), and shows that the attribution (self-rating of assertiveness) increases in the predicted direction. Psych 466, Miyamoto, Aut '15 29 Summary of Attribute Substitution

30. Psych 466, Miyamoto, Aut '15 30 Summary A common form of heuristic reasoning is attribute substitution: I.e., judging a target attribute in terms of a more available heuristic attribute. There can be more or less direct conflicts between heuristic reasoning and rule-governed or theory-governed reasoning. Heuristic reasoning is often context sensitive ♦ Different contexts may suggest different heuristics. ♦ Different contexts may produce different emphases on heuristic versus rule- governed reasoning. Heuristic reasoning is sometimes good and sometimes bad (for the person doing the reasoning). Class discussion of attribute substitution

31. Psych 466, Miyamoto, Aut '15 31 Class Discussion Can we generate examples of attribute substitution in everyday life? Example: ♦ Target Attribute: Will my relationship with XXX last? ♦ Heuristic Attribute: How well do I get along with XXX? Example: ♦ Target Attribute: Will a career doing YYY be rewarding for me over the long run? ♦ Heuristic Attribute: How much fun do people have when doing YYY? Exercise for the Class - END

32. Psych 466, Miyamoto, Aut '15 32

33. Psych 466, Miyamoto, Aut '15 33 Outline Representativeness heuristic had two aspects: Similarity Thesis Irregularity Thesis Irregularity Thesis  Misconceptions of Chance Examplies of misconceptions of chance Independent, Stationary Model for Random Events

34. Psych 466, Miyamoto, Aut '15 34 Independent, Stationary Model for Random Events Independent stationary model: ♦ Independent – what happens in the past has no influence over what happens next. ♦ Stationary – the probability of the event doesn't change over time. Examples: Chance of rolling a "2" with a die is independent and stationary. Chance of “heads” when flipping a coin is independent and stationary. Chance of rain on successive days is not independent (rain or dry on day 1 is correlated with rain or dry on day 2) and not stationary (chance of rain varies depending on the season). NOTE: There really are “dry streaks” and “wet streaks”. Subjective Perception of Event Clusters

35. Psych 466, Miyamoto, Aut '15 35 Subjective Perception of Event Clusters People sometimes think that airplane crashes happen in clusters (several in close temporal proximity). Vaught & Dawes tested whether airplane crashes happen in clusters. Best fitting model is independent and stationary. V&D's result does not deny the existence of clusters – they deny that clusters are caused by karma, fate or other unseen forces. Consider example of people attempting to flip a coin mentally – mental coin flips are too patternless. Gambler's fallacy ( http://news.bbc.co.uk/2/hi/europe/4256595.stm,; http://www.guardian.co.uk/italy/story/0,12576,1410701,00.html ). Fallacy: A random pattern should break up a sequential pattern. http://news.bbc.co.uk/2/hi/europe/4256595.stm Gambler's Fallacy in an Italian Lottery

36. Psych 466, Miyamoto, Aut '15 36 Gambler's Fallacy in an Italian Lottery http://news.bbc.co.uk/2/hi/europe/4256595.stm, http://www.guardian.co.uk/italy/story/0,12576,1410701,00.htmlhttp://news.bbc.co.uk/2/hi/europe/4256595.stm http://www.guardian.co.uk/italy/story/0,12576,1410701,00.html Italian National Lotto (weekly): ♦ You can bet on any digit from 1 to 100. ♦ 5 numbers are drawn at random in 10 different Italian cities. Between May 2003 to February 2005, the number 53 was NOT drawn in the Venice lottery (152 consecutive lotteries). Pr(no "53" in 100 draws) =.0059 (about 1 in 170). Pr(no "53" in 150 draws) =.00046 (about 5 in 10,000) Frenzy of betting on "53" in the Venice lotto. ♦ Woman drowned herself; she left a note stating that she had lost her family's saving on "53.“ A man in Signa shot his wife and son before killing himself. At least 4 deaths, and other physical violence attributed to "53" madness. Also, personal bankrupties. Gambler's Fallacy in an Italian Lottery - Consequences

37. Psych 466, Miyamoto, Aut '15 37 Gambler's Fallacy in an Italian Lottery (cont.) Total of 3.5 billion Euros (about 4.2 billion dollars) was bet on "53". About $27.5 million was spent per month before the February appearance of "53." $806 million was spent in the month of January before the February appearance of "53." (Almost 30 times the average monthly betting on "53.") ♦ Does this indicate a rise in "53" betting towards the end? The eventual winner who bet on "53" got about $768 million. Hot Hand in Basketball

38. Psych 466, Miyamoto, Aut '15 38 Hot Hand in Basketball (Gilovich, Vallone & Tversky) If players do get a "hot hand," then a player has a higher chance of making a basket when he/she is hot than when he/she is cold. ♦ The "hot hand" hypothesis implies that shooting baskets is not independent and stationary Independent & stationary (I&S) model for a player shooting baskets: Player has a constant p-chance of making a shot. This chance is the same for every shot in a sequence of shots. The “hot hand” hypothesis and the I&S model contradict each other. 91% of basketball fans (Stanford & Cornell games) believed that a player has a better chance of making a shot after making 2 or 3 than after missing 2 or 3. Conditional Probability Results for Philadelphia 76ers

39. Psych 466, Miyamoto, Aut '15 39 Conditional Probabilities of Making a Shot 1980-81 Philadelphia 76ers. H = Hit = Made ShotM = Miss P(H | 3 M)P(H | 2 M)P(H | 1 M)P(H)P(H | 1 H)P(H | 2 H)P(H | 3 H) Clint Richardson.50.47.56.50.49.50.48 Julius Erving.52.51.52.53.52.48 Lionel Hollins.50.49.46.32 Andrew Toney.52.53.51.46.43.40.34 Runs Test for Streaks

40. Psych 466, Miyamoto, Aut '15 40 Analysis of Runs A run is a succession of H or a succession of misses. ♦ H H M H M M M H. 5 runs The Wald-Wolfowitz runs test is a test for stationarity (low p- value on the test means that stationarity is violated). Rejection of null hypothesis (p <.01) was found for 1 of 9 players (Daryl Dawkins). Analysis of shots grouped into four successive shots. Stationarity predicts the number of groups with 0 shots made, 1 shot made,..., 4 shots made. Stationarity predictions are supported for all players. Also found no evidence for game by game variation in the the chance of making a shot. ♦ Remarkable: All teams have equally good defense? Conclusions re Hot Hand

41. Psych 466, Miyamoto, Aut '15 41 Conclusion re the Hot Hand in Basketball Human intuition assumes that randomness looks even more random than real randomness. Therefore real randomness looks like there are non-random patterns in the data. Therefore we are prone to infer causal forces where none exists. In the case of the "hot hand", the causal force is physical or psychological process that changes the probability of success on the next shot. Website devoted to research into sports streaks (Alan Reifman): http://thehothand.blogspot.com/ http://thehothand.blogspot.com/ Illusion of Control

42. Psych 466, Miyamoto, Aut '15 42 Illusion of Control Ellen Langer demonstrated that people perceive themselves to have control over outcomes even when they have no control. Experiment varied contingency between subjects' responses and the probability of "winning." ♦ High contengency – subjects' strategy had a big effect on chance of "winning" ♦ Low contingency – subjects' strategy had a small effect on chance of "winning" ♦ No contingency - subjects' strategy had NO effect on chance of "winning" ♦ Subjects in ALL conditions reported that their strategies affected their chances of reward, i.e., they could do something to give themselves a better chance of reward. Illusion of control – subjects in NO contingency condition still thought they had some efficacy (some way to improve their chances). Illusion of Control in a Simulated Computer Investments

43. Psych 466, Miyamoto, Aut '15 43 Illusion of Control in Simulated Investments Fenton-O'Creevy, M., Nicholson, N., Sloane, E., & Willman, P. (2003). Trading on illusions: Unrealistic perceptions of control and trading performance. Journal of Occupational and Organizational Psychology, 76, 53-68. Traders from four British investment banks played a computer game. Traders make fictional investments which may or may not influence the value of an investment index (measure of a stock's value). Computer program was rigged so that the value of the investment index was independent from the traders' actions. The traders reported that they were able to influence the investment index to varying degrees. (Reality was that they had no influence.) Traders with the greater illusion of control earned less on the average than other traders in the investment game. Their managers rated as lower on rsk management and analytical ability. Conclusions re Misperception of Randomness

44. Psych 466, Miyamoto, Aut '15 44 Conclusions re the Perception of Randomness "Hot hand" is an illusion. ♦ Website for "hot hand" research (Alan Reifman): http://thehothand.blogspot.com/ http://thehothand.blogspot.com/ Human intuition assumes that randomness looks even more random than real randomness. Therefore real randomness looks like there are non-random patterns in the data. Therefore we are prone to infer causal forces where none exists. Outline of Attribute Substitution Topic

45. Psych 466, Miyamoto, Aut '15 45 Set Up for Instructor Classroom Support Services (CSS), 35 Kane Hall, 206-543-9900 CSS: Try setting your resolution to 1024 by 768 Run Powerpoint. For most reliable start up: ♦ Start laptop & projector before connecting them together ♦ If necessary, reboot the laptop