Decision-making I choosing between gambles neural basis of decision-making

Do we always make the best possible decisions? Normative (or prescriptive) theories: tell us how we should make rational decisions E.g. optimize financial gain Descriptive theories: tell us how we actually make decisions, not on how we should make them. Satisficing Heuristics Behavior can deviate from normative account in systematic ways

What are rational decisions? Decisions that are internally consistent E.g., if A>B, then B<A if A>B, B>C, then A>C (transitivity) Decisions that optimize some criterion E.g. financial gain (expected utility theory)

Example What is the best choice? A) .50 chance of winning $20 B) .25 chance of winning $48

Classic Decision Theory: Expected Utility Model The utility of an outcome is a numerical score to measure how attractive this outcome is to the decision-maker. The expected utility is the utility of a particular outcome, weighted by the probability of that outcome’s occurring. A rational decision-maker should always choose the alternative that has the maximum expected utility. probability utility

Example (1) Gamble: if you roll a 6 with a die, you get $4. Otherwise, you give me $1. Take the gamble? Expected utility = p(win)*u(win) + p(lose)*u(lose) =(1/6)*(4)+ (5/6)*(-1) =-1/6 So...do not take bet

Example (2) Which job would you accept: Job A: 50% chance of a 20% salary increase in the first year Job B: 90% chance of a 10% salary increase in the first year The expected utility model predicts Job A to be better (0.5 x 0.2 > 0.9 x 0.1)

Subjective Utility For most people, the utility of an amount of money is not equivalent to the monetary value, it is based on the subjective utility Example: What is the best choice? (A) .10 chance of winning $10 million dollars (B) .99 chance of winning $1 million dollars Each additional dollar added to wealth brings less utility (“diminishing marginal utility effect”) The book is a bit confusing because they refer sometimes to the expected utility model as if the utilities are the same as the monetary values, and at other times they refer to the expected utility model as if the utilities are subjective evaluation of the value. It is better to distinguish between the expected utility model where probabilities and utilities are based on objective outcomes and subjective utility model based on subjective probabilities and utilities.

A hypothetical utility curve Diminishing marginal utility: additional gains (or losses) are not valued as much as early gains (or losses)

A hypothetical utility curve Loss-aversion the negative effect of a loss is larger than the positive effect of a gain

Risk Aversion for Gains Example Gamble 1: win $20 with 50% chance or $60 with 50% chance Gamble 2: win $40 with 100% chance

Risk Aversion for Gains Utility Example Gamble 1: win $20 with 50% chance or $60 with 50% chance Gamble 2: win $40 with 100% chance What would a person choose with the subjective utility function shown on left? 100 80 60 40 20 0 20 40 60 80 100 Monetary Value ($) Gamble 1: 74x0.5+100x0.5=87 < Gamble 2: 92x1=92

A hypothetical utility curve concave utility function for gains: Risk-aversion for gains convex utility function for losses: Risk seeking for losses

Individual Differences Utility Decision Maker I (risk avoider) 100 80 60 Decision Maker II (risk taker) 40 A risk avoider will have a concave utility function when utility is measured on the vertical axis and monetary value is measured on the horizontal axis. Individuals purchasing insurance exhibit risk avoidance behavior. A risk taker, such as a gambler, pays a premium to obtain risk. His/her utility function is convex. This reflects the decision maker’s increasing marginal value of money. 20 0 20 40 60 80 100 Monetary Value

Subjective Probability The probability of an event might not be based on objective statistical calculations but might be based on a subjective estimate Overweighting of small probabilities and underweighting of likely outcomes

Limitations of the Expected Utility Model We can make “bad decisions”—that is, decisions that are irrational according to the expected utility model Framing effects Violations of transitivity

Framing effect example: mental accounting People think of money as belonging to certain categories, but it is really all the same money Problem A. Imagine that you have decided to see a play and paid the admission price of $10 per ticket. As you enter the theater, you discover that you have lost the ticket. The seat was not marked and the ticket cannot be recovered. Would you pay $10 for another ticket? Problem B. Imagine that you have decided to see a play and paid the admission price of $10 per ticket. As you enter the theater, you discover that you have lost a $10 bill. Would you pay $10 for a ticket? Yes 46% No 54% People are more likely to pay in situation B but the decision is the same: is the play worth $10. This framing effect is also known as “psychic budgeting” (Thaler, 1980). We mentally categorize money that we have spent and the same amount in money spent can be categorized differently leading to different judgments. Yes 88% No 12% (Tversky & Kahneman, 1981)

Another example of a framing effect Problem 1: Select one of two prizes (36%) An elegant Cross pen (64%) $6 Problem 2: Select one of three prizes (46%) An elegant Cross pen (52%) $6 (2%) An inferior pen Decisions can change when other options are added – people make different choices depending on how the problem is described (Shafir & Tversky 1995)

Example: Cheeseburgers 50%

Example: Cheeseburgers 50% 10% 30% 60% 50%

Another example of a framing effect Problem 1 Suppose I give you $300, but you also have to select one of these two options: 1.0 chance of gaining $100 .50 chance of gaining $200 and a .50 chance of gaining nothing Problem 2 Suppose I give you $500, but you also have to select one of these two options: 1.0 chance of losing $100 .50 chance of losing $200 and a .50 chance of losing nothing (72%) (28%) Note that the subjective expected utility model cannot explain the preference reversal between problem 1 and 2. For example, for option A in *both* problem 1 and 2, the subjective utility is the same. (36%) (64%) (Tversky & Kahneman, 1986)

Violations of Transitivity Transitivity: If you prefer A to B and B to C then you should prefer A to C. Experiment included the following gambles (expected values were not shown): Result: subjects preferred: A>B, B>C, C>D, D>E, but also E > A If the red ball is larger than the yellow ball, and the yellow ball is larger than the green ball, then the red ball is larger than the green ball. This is the principle of transitivity, which states that if a “greater-than” relation holds between a first element and a second one and between that second element and a third one, then it must also hold between the first element and the third one. Transitivity of preferences as expressed in choices is perhaps the most fundamental principle of rational choice: if I prefer X over Y, and Y over Z, then I should prefer X over Z. Violations of transitivity can be explained by changes in the focus of attention. Sometimes people focus more on the differences in payoff in any adjacent gambles in the table (when the differences in probability are small). However, when the differences in probability become more extreme, people start paying attention to probability (as well as expected value). (Tversky, 1969)

Rationality up to a point People have limitations in memory and time Simon (1957) Bounded rationality we are rational, but within limits of human processing capabilities Satisficing We choose the first option that meets our minimum requirements people might satisfice when making decisions such as buying a car

Neural Basis of Decision-Making & Role of Emotions

Neural Bases Of Expected Utility Calculations individual neurons in the lateral intraparietal cortex were active before the reward was given Glimcher (2003)

“Surprise Reactions” in neurons The monkey is passively sitting. A stimulus in the form of a geometric shape flashes, some seconds later, a reward in the form of a juice squirt is delivered to the monkey. This whole time, the experimenter is recording the brain activity of the monkey. Here, the monkey is shown a cue that signals a reward will be delivered with probability 1. Measurement in ventral midbrain region – dopaminergic neurons Reward will be delivered with probability one Fiorillo, Tobler, and Schultz. Science. (2003)

“Surprise Reactions” in neurons Reward is delivered despite having never been delivered (for this stimulus) in the past Fiorillo, Tobler, and Schultz. Science. (2003)

Involvement of Emotional Areas in Decisions fMRI study INTRODUCTION The investigators were interested in testing the hypothesis that the differences between choices made with immediate rewards versus delayed rewards is due to the operation of two independent valuation systems in the brain: a visceral, emotional evaluation system versus a deliberative, rational system. This hypothesis was grounded in more than 50 years of behavioral research. The researchers sought to find different areas of brain activation when immediate versus delayed rewards were evaluated and, in particular, to find correlations between the amount of “emotional brain” activation and the reference for immediate outcomes. The hypothesis implied that areas of the brain known to be associated with emotional responses would be activated specifically when immediate rewards were considered. METHOD Princeton University students made choices between pairs of money rewards to be received immediately or at delayed time intervals (up to six weeks in the future), while lying in an fMRI scanner. Some of the rewards would actually be paid out at the point in time when they had been promised (as Amazon.com gift certificates), so participants were motivated to make careful choices that reflected their true valuations of the outcomes. Participants were presented with pairs of money payoff options (e.g., $25 in one week versus$34 in one month); on the left was a smaller earlier reward, on the right a larger delayed reward. They pressed a button to indicate their choice and, after a brief rest, another pair was presented. fMRI methods were used to record brain activity during the 4 seconds preceding the choice response and 10 seconds following it. The time series graphs, summarizing activity in key brain areas are shown in Figure 9–9 along with brain images indicating which areas were active. RESULTS The neuroimaging results were clear. This slide shows the results for regions of the brain that were specifically active when an immediate outcome was presented as one option in a choice pair. The ventral striatum, medial orbitofrontal cortex, medial prefrontal cortex, posterior cingulate cortex, and the left posterior hippocampus were all significantly more active when immediate options were compared to options with a two-week or one-month delay. These areas are all good candidates to support the hypothesized emotional processes. There were six additional brain regions that exhibited significant elevation in activity when choices were being made, regardless of whether they involved immediate or delayed outcomes. Some of these areas involve perceptual and motor areas and are not likely to be associated with choice processes per se. But others, involving the lateral orbitofrontal cortex, the ventrolateral prefrontal cortex, and the dorsolateral prefrontal cortex, are probably involved in the deliberative, rational processes (see Chapter 7)—these are the “rational” areas in contrast to the “emotional” ones mentioned earlier. DISCUSSION This study is an elegant illustration of how neuroimaging measures can complement and enhance the implicational power of behavioral analyses. Not all neural-behavioral collaborations are this successful, but this one was supported by several key ingredients: (1) the availability of a thorough behavioral analysis of temporal discounting and (2) the prior specification of potential neural components—specifically, the association of the emotional process with visceral-emotional systems and the rational process with a more deliberative-cognitive system. McClure et al. (2004)

The Iowa Gambling Task A B C D Four decks: On each trial, the participant has to choose a card from one of the decks. Each card carries a reward, and, sometimes, a loss…

The Iowa Gambling Task Four decks: A B C D +$100 −$350 Each deck has a different payoff structure, which is unknown to the participant. In order to maximize overall gain, the participant has to discover which decks are advantageous and which are not.

The Iowa Gambling Task Bad Decks Good Decks A B C D Reward per card $100 $100 $50 $50 Av. loss per card $125 $125 $25 $25

Behavioral Results (Bechara et al., 1999)

Skin Conductance Results (Bechara et al., 1999)

Results Control participants learned how to maximize wins. Showed elevated SCR responses in anticipation of a potential large loss Patients with ventromedial PFC damage: Performed poorly on task (stuck with bad decks). Did not show elevated SCR responses before poor choices. Somatic Marker Hypothesis (Damasio et al., 1996): we need our emotional brain areas to set markers to warn us about threats or opportunities Damasio and Bechara conjectured that advantageous decisions are based on a “gut feeling”. Patients such as Phineas Gage have a disturbed emotional regulation. They fail to properly monitor the emotional responses of their body. These emotional bodily responses (somatic markers) act as warning signals, preventing selection of bad decks.