Wallis, JD Helen Wills Neuroscience Institute UC, Berkeley Neuronal Mechanisms in Prefrontal Cortex Underlying Adaptive Choice Behavior Wallis, JD Helen Wills Neuroscience Institute UC, Berkeley

Background Role of prefrontal cortex (PFC) in reward-guided choice behavior 2 Questions: Does PFC encode reward or behavioral sequelae of reward? Is encoding specific to reward outcome or reflective of abstract value signal?

Background Orbitofrontal cortex (OFC) is a key region in choice behavior Has functions in emotions and reward Thought to regulate planning behavior associated with sensitivity to reward and punishment Paul Wicks’ brain

OFC Damage to the OFC leaves cognitive abilities intact, but impairs ability to make decisions The cautionary tale of Elliot OFC neurons encode expected rewards Neurons show response to delivery of juice rewards predicted by a visual stimulus Useful for decision making

DLPFC Reward encoding neurons are also found in the dorsolateral PFC (DLPFC) Neurons showed a difference in firing rate depending on large/small expected reward However, are these neurons encoding the value of the reward or a behavioral correlate of the reward (tensed muscles, heightened accuracy)?

Experiment 1 Comparison of reward encoding in the DLPFC and the OFC 2 monkeys choose between pictures associated with small/large fruit juice rewards

Experiment 1

Methods Each picture associated with delivery of a certain amount of juice Subjects learn to maximize reward Reward-picture contingencies reversed after 27 out of 30 successes Most important neuronal activity after 2nd picture appears

Methods Activity was recorded simultaneously from multiple electrodes in DLPFC and OFC 167 DLPFC neurons 134 OFC neurons

OFC Results Spike density histograms from 2 representative OFC neurons Activity recorded during predictive cue One neuron showed higher firing rate when the monkey expected 4 drops of juice Another encoded the reward in parametric fashion Firing rate not affected by saccade

OFC Results Figure 2

DLPFC Results DLPFC neurons show responses related to both reward and saccade One neuron discriminated between different amounts of juice only during right saccade Another encoded reward in parametric fashion (positive?) with a greater increase during left saccade

DLPFC Results Figure 3

Statistics 2-way ANOVA on mean firing rate with factors of Reward and Saccade OFC: 28% significant main effect of Reward 19% significant interaction with Saccade DLPFC 13% significant main effect of Reward 43% Reward-Saccade interaction

Statistics

Statistics Some neurons in both areas had similar properties: 27% parametric increase with reward size 15% parametric decrease with reward size 59% encode specific reward

ROC Analysis Sliding receiver operating characteristic (ROC) analysis of the selectivity time-course revealed differences in encoding of reward between OFC and DLPFC Probability that an independent observer could predict reward based on firing rate Starting from 500ms prior to 2nd picture an ROC curve was generated from 10ms increments

ROC Analysis Latency at which selectivity appeared was computed as the point at which the curve exceeded 0.6 No difference between OFC (mean 426ms) and DLPFC (mean 467ms) (t-test = 1, d.f. = 110, P > 0.1)

ROC Analysis Selectivity rose more rapidly and peaked earlier in OFC 80 ms earlier on average Short latency indicates OFC is encoding reward’s value rather than correlated behavioral/cognitive processes

ROC Analysis

Summary Neurons sensitive to expected reward are found in both the OFC and DLPFC OFC neurons encoded only reward while DLPFC neurons encoded reward and saccade OFC neurons encoded reward earlier than DLPFC

Summary Therefore OFC is first prefrontal region receiving reward information From basolateral amygdala Encodes immediate reward –Winstanley, CA et al From gustatory and olfactory cortices

Baxter, MG and Murray, EA The amygdala and reward

Summary DLPFC is where reward value converges with subjects actions: reward choice

Experiment 2 OFC response indicates to the motor system which action leads to largest reward However, decision making needs to be more complex Reward value is determined by (Reward – Cost) x P of success To what extent does OFC encode these variables?

Experiment 2 OFC may integrate all variables relevant to decision making to derive an abstract value signal Neuronal currency Another study tested whether PFC neurons were capable to responding to multiple parameters

Methods Monkeys were trained to choose between pictures associated with particular rewards Recordings from OFC, MPFC, and lateral PFC taken simultaneously

Methods 3 variables were manipulated: Probability: Some pictures predict fixed amount of juice on only a certain proportion of trials Reward: Some pictures were associated with varying amounts of juice Effort: Monkey had to earn fixed amount of juice by pressing a lever multiple times

Results 1/3 of PFC neurons responded parametrically to just 1 parameter Found in all 3 areas Some neurons responded to a combination of parameters Progressive increase from LPFC to OFC to MPFC 16% -> 27% -> 48%

Summary OFC and MPFC combine multiple variables in order to make a decision by deriving abstract value signals Too difficult to make direct comparisons between all possible choices Each choice can be valued against a common reference scale (currency) Example: how many bananas is your car worth?

Summary Abstract value signals allow flexibility and novelty Simplifies the task of the motor system Allows instantaneous choice Patients with OFC and MPFC damage show unusual patterns of decision making A > B , B > C but A < C

Conclusions Damage to the OFC impairs decision making while leaving other cognitive abilities intact OFC is implicated in reward info processing Must differentiate between reward and behavioral sequelae of reward Short latency of neuronal reward-related responses indicates encoding of reward’s value in OFC

Conclusions In contrast, DLPFC encodes reward info in relation to behavioral responses PFC neurons also encode other variables related to decision making, including probability of success and effort required OFC and MPFC neurons are responsible for integrating these variables to derive an abstract value signal