1. Round Table and Panel Discussion The psychology of Mistakes American Academy of Appellate Lawyers Massimo Piattelli-Palmarini Cognitive Science, University of Arizona March 24, 2006 Micro-irrationality

2. UofA March 24 2006Round Table on Errors The coverage of this field n Articles relevant to this domain of research have been published in more than 400 scientific journals n There are, by now, over 20 extensive anthologies and several popularization books. n Including my own: n Massimo Piattelli-Palmarini (1994) Inevitable Illusions: How Mistakes of Reason Rule our Mind (John Wiley) n Courses in decision research are commonly taught to students in psychology, economics, philosophy, business, management, medicine, law, and in the military academies.

3. UofA March 24 2006Round Table on Errors The coverage of this field n In the last several years, also cognitive neuroscientists have grown an interest in human decision making n Several papers have been published already on the brain counterparts of typical heuristics and biases. n Specific brain lesions have been discovered that selectively impair the decision-making ability of those patients (Antonio Damasio and collaborators at the University of Iowa) n The choices made by subjects in a particular game of cards (win/lose) is used as standard clinical diagnostic test to detect pathologically risk-prone subjects (compulsive gamblers etc.)

4. UofA March 24 2006Round Table on Errors An important specification: n In this domain, we deal with problems and situations such that n It makes sense, and it is rather obvious n That there is such a thing as n The right decision, the correct choice, the correct answer, the correct estimate n We do not deal with fleeting propensities, mere tastes, personal inclinations, wanton impulses etc. n In this domain there are normative (rational) theories that do give the right answer n Our experimental subjects want to make the right choice, give the right answer

5. UofA March 24 2006Round Table on Errors Names for this field n Behavioral Decision-Making n Psychology of reasoning n Psychology of choices n Decision research n Heuristics and Biases n Judgment and Decision-Making (JDM) n The latter is the most consolidated label n A Nobel Prize in economics changed it all

6. UofA March 24 2006Round Table on Errors Heuristics n Form the Greek heurein = to find (whence the exclamation Eureka!) n Thumb-rules and intuitive strategies that we apply to the search for a solution in a certain class of problems. n Something we do n Consciously or unconsciously, or semi- consciously. n Heuristics have no secure warrant n That is, no rational warrant, n That is, they do not derive from first principles and logical proofs n Like full, guaranteed methods (and methodologies) do.

7. UofA March 24 2006Round Table on Errors Biases n Tunnels of the mind n Of which we (usually) have no awareness n Something that happens to us n For instance (as we will see): n Anchoring n Partitions in probability estimates n Mental reference points and baselines n Ease of representation (availability) n And many more

8. UofA March 24 2006Round Table on Errors Heuristics and biases nneither rational, nor capricious n A very synthetic characterization by Daniel Kahneman and Amos Tversky

9. UofA March 24 2006Round Table on Errors Two giants of this field: n Amos Tversky (deceased in 1996, Stanford University) n Daniel Kahneman (Princeton University, Nobel Prize for Economics 2002) Daniel Kahneman

10. UofA March 24 2006Round Table on Errors Daniel Kahneman receiving the Nobel Prize for Economics 2002

11. UofA March 24 2006Round Table on Errors Daniel Kahneman I recommend his Nobel Lecture

12. UofA March 24 2006Round Table on Errors Amos Tversky He would have shared the Nobel Prize with Kahneman, had he still been alive. (1937-1996)

13. UofA March 24 2006Round Table on Errors Rapid multiplication n Take a group of subjects, and subdivide it at random into two subgroups n Group A n You ask them to make a quick, approximate mental calculation of the following multiplication n 8 7 6 5 4 3 2 n Group B n You ask them to make a quick, approximate mental calculation of the following multiplication n 2 3 4 5 6 7 8 n What do you expect the results will be?

14. UofA March 24 2006Round Table on Errors Rapid multiplication: The data n Group A: Average response = 2,250 for n 8 7 6 5 4 3 2 n Group B: Average response = 512 for n 2 3 4 5 6 7 8 n Exact result: 40,320 n No one guesses anywhere near the exact result. n Moreover n The two approximations are grossly different. n If you ask them, after they have given the estimate, n they all know the commutative property of multiplication!

15. UofA March 24 2006Round Table on Errors The cognitive explanation: n Anchoring: You start multiplying left-to-right n Then extrapolate and round up n And you are trapped by what initially comes up, left to right n This is called an anchoring effect. n Many many examples in everyday life. n Notice: You have to work with two separate groups n Otherwise its impossible to reveal this effect with this rapid multiplication task.

16. Some ultra-simple examples:

17. UofA March 24 2006Round Table on Errors The square turning into a rectangle n What is your intuition?

18. UofA March 24 2006Round Table on Errors The normative solution n Since the perimeter is kept constant n The area cannot also be constant n In fact, it decreases monotonically to zero. L L Area = L 2

19. UofA March 24 2006Round Table on Errors The normative solution n Since the perimeter is kept constant n The area cannot also be constant n In fact, it decreases monotonically to zero. L-x L + x Area = (L-x) (L+x) = L 2 - x 2 x

20. UofA March 24 2006Round Table on Errors What happens in our mind: n A strong conservation principle is evoked: n Some portion of surface is added laterally n Some portion of surface is subtracted vertically n One side grows shorter n The other side grows longer n We (wrongly) infer that these variations compensate one another n And conclude that the surface is constant.

21. UofA March 24 2006Round Table on Errors What happens next: n The surface goes to zero n Our intuition of conservation vacillates n Two reasoning strategies: n (1) Conservation prevails: n We introduce a sudden singularity n The surface is the same, until it vanishes at the limit (only at the limit) n (2) Continuity prevails n We abandon conservation, and accept that the surface must have been decreasing all the way n Notice: We do not accept contradictions and inconsistencies. We try to remedy, somehow.

22. UofA March 24 2006Round Table on Errors Another very simple cognitive illusion The two coins n Two coins are tossed into the air. I can see the result, but you cannot. I tell you truthfully that one of them has come up heads. What is the probability that the other also has come up heads? What do you say? n The vast majority answers: one half! n Why? n Two independent events (this is right) n therefore p=1/2 n (but this inference is wrong) n My report is not about two independent events, but about a cumulative event. n The interesting cognitive fact is that we do not see it.

23. UofA March 24 2006Round Table on Errors The problem of partitions n What can happen? n (a ) H H n (b ) H T n (c ) T H n (d ) T T n (d) is ruled out by my statement, therefore n p = 1/3 n A different situation: reporting about one specific coin, and asking about the other. We are blind to this difference: a cognitive oversight (neglect).

24. UofA March 24 2006Round Table on Errors A real-life judicial episode n The O.J. Simpson Trial n The probability that a husband who batters his wife will end up murdering her is 1 in 2,500 (4 in ten thousand) (US Police records for 1992) n Imagine that you are a member of the jury. n Do you find this argument u Very convincing u Somewhat convincing u Only moderately relevant u Totally irrelevant

25. UofA March 24 2006Round Table on Errors Confusing conditional probabilities n What is relevant is not the probability of murder, given beating n BUT n The probability that a wife who has been murdered, has been murdered by a partner who was known to beat her. n This is close to 90% n Also based on the same Police records n As remarked by I.J. Good in 1996, nobody pays any attention to this huge difference in conditional probabilities. n Many, many examples in all walks of life

26. UofA March 24 2006Round Table on Errors The maternity ward n A certain town is served by two hospitals. In the larger hospital about 45 babies are born each day. In the smaller hospital about 15 babies are born each day. n As you know, about 50 per cent of all babies are girls. However, the exact percentage varies from day to day. n Sometimes it may be higher than 50 per cent, sometimes lower.

27. UofA March 24 2006Round Table on Errors The maternity ward n For a period of one year, each hospital recorded the days on which 60 per cent or more of the babies born were girls. n Which hospital do you think recorded more such days? n Please indicate your choice: The larger hospital The smaller hospital About the same (within 5 per cent of each other)

28. UofA March 24 2006Round Table on Errors The data u The larger hospital 22% u The smaller hospital 22% u About the same (within 5 per cent of each other) 56% n Rationale: Sex does not depend on the size of the hospital n Of course, but a fluctuation does n The correct answer is, in fact: The smaller hospital.

29. UofA March 24 2006Round Table on Errors An interesting variant n Same story. But now the subjects are asked to estimate which hospital recorded more days in which all the babies were girls. n Now over 90% of subjects choose the smaller hospital. n Many re-consider their previous intuition n But some dont. n Notice: Those who reconsidered have not been instructed. Only questioned. n The de-biasing is a self-de-biasing.

30. UofA March 24 2006Round Table on Errors If that be madness…….. n Indeed, these heuristics and biases are neither rational, nor capricious (Amos Tversky and Daniel Kahneman) n These effects are: u Systematic u Resistant to de-biasing u Stimuli for improvised rationalizations u Independent of the level of education u (Presumably) culture-independent

31. UofA March 24 2006Round Table on Errors An important lesson: n We knew all along that people are irrational n Sure! n Because of passions, selfishness, greed, ambition, racism, prejudice, sheer stupidity, superstition, etc. n None of the above factors is involved in the problems we treat in this domain. n This is a different kind of irrationality. n I have called it micro-irrationality n Only systematic, predictable errors of intuition and reasoning. n This is what I have also called the cognitive unconscious (see my 1994 book Inevitable Illusions, Wiley).

32. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox * 1

33. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox * 2

34. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox * 3

35. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox * 4

36. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox * 5 Now here is your problem. Are you better off sticking to your original choice or switching? A lot of people say it makes no difference. There are two boxes and one contains a ten-dollar bill. Therefore, your chances of winning are 50/50. However, the laws of probability say that you should switch.

37. 6/13/2014Round Table on ErrorsTradeoff Studies ver 4 Monty Hall Paradox 6 The box you originally chose has, and always will have, a one-third probability of containing the ten- dollar bill. The other two, combined, have a two-thirds probability of containing the ten-dollar bill. But at the moment when I open the empty box, then the other one alone will have a two-thirds probability of containing the ten-dollar bill. Therefore, your best strategy is to always switch!

38. UofA March 24 2006Round Table on Errors How much would you offer for an ice-cream cup like the above, of your favorite flavor? Christopher K. Hsee (University of Chicago) (1998) Group 1

39. UofA March 24 2006Round Table on Errors How much for this cup (again, of your favorite flavor)? Group 2

40. UofA March 24 2006Round Table on Errors Vendor L is more attractive for a majority of subjects tested separately. Group 3

41. UofA March 24 2006Round Table on Errors Hardly anyone prefers vendor L when both drawings are shown (As they are here) (Group 3)

42. UofA March 24 2006Round Table on Errors Explanation n The reference point is very important n An overflowing amount in a smaller cup is a lot of ice cream n A partially un-filled amount in a larger cup is not a lot n S. Dehaene and L. Cohen (1991) have discoverd two distinct cerebral areas n One for gross approximations n One for precise calculations n Selective brain lesions actually disrupt one, but not the other

43. UofA March 24 2006Round Table on Errors Preferences for a safety apparatus in airports (Paul Slovic, 2003) n How intensely would you endorse a new safety device, knowing that: n Group A: It can save the lives of 150 people n Group B: It can save 98% of human lives, over a total of 150 people. n The rating ranges from 0 (no endorsement) to 20 (very enthousiastic endorsement) n A (as an average) 10.4 n B (as an average) 13.6

44. UofA March 24 2006Round Table on Errors Same question (Paul Slovic, 2003) n Group C: Can save 95% of human lives, over a total of 150 people n Group D: 90% n Group E: 85% n C (average) 12.9 n D (average) 11.7 n E (average) 10.9 n Reminder: A was 10.4 (for all the 150 people)

45. UofA March 24 2006Round Table on Errors Percentage of sanguine endorsement (Paul Slovic, 2003) n Percentage of responses higher than 13 (sanguine endorsement), per group : n A (150 people) 37% n B (98% of 150) 75% n C (95% of 150) 69% n D (90% of 150)35% n E (85% of 150) 31% n Saving fewer lives, with respect to A, receives greater endorsement and more enthousiastic endorsement. n Why?

46. UofA March 24 2006Round Table on Errors Explanation: n Just like for the ice cream, n 150 people is an « open » datum (Is it a lot? Is it too few?). n Hard to tell. n But 98% of 150 people is a lot! n With respect to the explicit « roof » of 100% n You like this! n And you endorse the proposal more intensely.

47. UofA March 24 2006Round Table on Errors A field called probabilistic judgment n Recipe: n Take one of the axioms of normative probability calculus. n You have reasons to suspect that people often violate it n Design a cute experiment n Publish a paper showing that a majority of subjects violate that axiom n Choose another axiom n Repeat the above procedure n Construct your favorite cognitive theory of spontaneous probability judgments.

48. UofA March 24 2006Round Table on Errors Standard example n Axiom: p( e) = 1-p(e) n Violation: Zeckhauser and the forcible Russian Roulette: n How much would you be willing to pay to remove one bullet from the drum? n From 1 bullet to zero n From 4 bullets to 3 n From 6 bullets to 5 n A steeply decreasing function from both extremes. n 1/6 of increase in the probability of survival is worth a lot if it is from 5/6 to one, and from zero to 1/6, not much if it is from 2/6 to 3/6 n Sensitivity to differences increases sharply near the endpoints.

49. UofA March 24 2006Round Table on Errors A typical subjective value function: Notice the 2.5 asymmetry between gains and losses x 2.5x

50. UofA March 24 2006Round Table on Errors A standard test: Choices between lotteries n What would you prefer: $100 for sure or $1,000 with probability 10% n A long series of well-calibrated choices between pairs of lotteries n One with low gain and high probability n One with high gain and low probability n We measure a function of objective probabilities n The subjective probability weights n How much a probability p weighs on that persons decisions w(p) n The shape of the curve is universal, and its quite interesting

51. UofA March 24 2006Round Table on Errors Data from Tversky and Kahneman 1992

52. UofA March 24 2006Round Table on Errors The behavior of W(p) for very low probabilities Drazen Prelec (2000) in a thousand zone in ten thousands zone in a million zone in 100 thousands zone

53. UofA March 24 2006Round Table on Errors The behavior of W(p) for very low probabilities Notice that the weighting function becomes flatter and flatter. One chance in a million is the same as 2 or 3 chances in a million in a thousand zone in ten thousands zone in a million zone in 100 thousands zone

54. UofA March 24 2006Round Table on Errors This shape reflects the all-or-none character of perceived risk The weighting function becomes flatter and flatter. BUT the transition from exactly zero to even very small probabilities is HUGE

55. UofA March 24 2006Round Table on Errors As we know all too well n The total cancellation of a small risk is hugely over-rated n Even for a very, very, very small risk (one in a million) n WHILE n The mere reduction of that same risk n (from, say, ten in a million to one in a million) n is vastly under-rated. n Risk, in the popular mind, is something that is there or is not there (a quantum perception) n The slope of W(p) at zero is, in fact, infinite

56. UofA March 24 2006Round Table on Errors Typical values n W(0.5) ~ 0.4 n W(0.25) ~ 0.3 n 1.33 times W(0.5)/2 n W(0.05) ~ 0.15 n 2.7 times W(0.5)/10 n As probabilities decrease, the gap widens in a spectacular way n W(0.005) ~ 0.075 n 15 times W(0.5)/10

57. UofA March 24 2006Round Table on Errors Some consequences n Most of us are risk-averse for gains and risk- seekers for losses n We tend to evaluate gains separately from losses n We over-weigh small probabilities n And under-weigh large probabilities u The dividing point is about 37% u We are well calibrated around that point n Different ways of truthfully presenting the same options can have dramatic effects on decision-making and preferences n This effects are called framing effects

58. UofA March 24 2006Round Table on Errors General culture questions n In each of the following pairs, which city has more inhabitants? n (a) Las Vegas (b) Miami n (a) Sydney (b) Melbourne n (a) Hyderabad (b) Islamabad n (a) Bonn (b) Heidelberg

59. UofA March 24 2006Round Table on Errors How do we answer? n Recollection of facts we remember, and of scraps of information from various sources n Contingent pieces of information: n This city name is quite familiar (unfamiliar) to me n My cousin visited this city n Its a capital (or is not a capital) n This one has a famous University (soccer team, arts festival, etc.), but not the other. n Fame and familiarity usually go with the size of the city.

60. UofA March 24 2006Round Table on Errors The brute facts: n These are the real data (rounded up): n (a) Las Vegas (b) Miami n 600,000 370,000 n (a) Sydney (b) Melbourne n 4 millions 1 million n (a) Hyderabad (b) Islamabad n 5.2 millions 900,000 n (a) Bonn (b) Heidelberg n 297,000 140,000

61. UofA March 24 2006Round Table on Errors Over-confidence n After each answer subjects are also asked: n How confident are you that your answer is correct? n 50% 60% 70% 80% 90% 100% n It is typical to find that for the cases in which subjects say they are 100% confident, only about 80% of their answers are correct; n for cases in which they say that they are 90% confident, only about 70% of their answers are correct; n and for cases in which they say that they are 80% confident, only about 60% of their answers are correct.

62. UofA March 24 2006Round Table on Errors Over-confidence n This tendency toward overconfidence seems to be very robust. n Warning subjects that people are often overconfident has no significant effect, nor does offering them money (or other material incentives) as a reward for accuracy. n Moreover, the phenomenon has been demonstrated in a wide variety of subject populations including undergraduates, graduate students, physicians, physicists and even CIA analysts. (Lichtenstein, Fischhoff & Phillips, 1982.)

63. UofA March 24 2006Round Table on Errors Experienced horse handicappers making predictions on specific horses for specific races (Paul Slovic, 1973) Overconfidence

64. UofA March 24 2006Round Table on Errors On overconfidence n In their domain of expertise, subjects are more accurate n But the increase in their over-confidence exceeds by far the increase in accuracy. n They are over-confident where they can do most damage

65. UofA March 24 2006Round Table on Errors A knock-down objection, and a conclusion: n Bertrand Russell, one of the greatest logicians and philosophers of all times n And a great rationalist, wrote: n It is a dangerous and self-destructive folly to think that, since reason is not sufficient, then it is not necessary. n In other words: n We do not always have to decide rationally, in every matter n But n When we intend to make a rational decision n Then, it better be rational.