Neural circuits for bias and sensitivity in decision-making Jan Lauwereyns Associate Professor, Victoria University of Wellington, New Zealand Long-term Invitation Fellow, Japanese Society for the Promotion of Science Visiting Scholar, Tamagawa University, Tokyo, Japan

The Perfect Grandpa

Biological needs The drive reduction hypothesis Think: Inclusive fitness Think: energy, reproduction Approach Avoid

Biological needs The drive reduction hypothesis Several hours have passed since last meal

Biological needs The drive reduction hypothesis Several hours have passed since last meal Increased drive (hunger) Increased exploratory activity

Biological needs Several hours have passed since last meal Increased drive (hunger) Increased exploratory activity Find food, eat it Drive is reduced (reinforcement) The drive reduction hypothesis

Biological needs Several hours have passed since last meal Find food, eat it Drive is reduced (reinforcement) The drive reduction hypothesis Increased drive (hunger) Increased exploratory activity

Dopamine reward prediction (Schultz)

Executive control Goals, beliefs, wishes, fears… Related to motivational control Some sensory information is valuable to the individual in the sense that it may be used in the strategic (“optimal”) control of behavior Executive control would seek to maximize the extraction of valuable sensory information

How can executive control affect information processing? Two general hypotheses: Sensitivity –Selective improvement of information processing (actual perception) Bias: –Selective preparation (“anticipation”) of information processing (virtual perception) For example: “Reward”

Distinguishing effects of sensitivity and bias Signal detection theory (Green & Swets) Probability of response LATER model (Carpenter) Latency of response

Signal detection theory

Signal Noise Neuronal activity Noise Signal +

A different way to think about bias and sensitivity…

Scheme of the original LATER model (RHS Carpenter)

Nose Poke Paradigm: Spatial choice, Gives us good reaction-time distributions

Target side: 4 LEDs vs. Distracter side: 0-3 LEDs

Lauwereyns & Wisnewski (2006, JEP:ABP)

Theoretical example of bias

Theoretical example of sensitivity

How does it really work?

How does the brain incorporate reward value in the control of action?

Studied in monkeys using saccadic eye movement tasks with asymmetrical reward schedule

Biased Saccade Task (BST)

Target position = unpredictable

Biased Saccade Task (BST) Reward association = known

Biased Saccade Task (BST)

No escape!

Asymmetric position-reward mapping in “ABA” design Frequent reversal of blocks

Strong effect of reward value on saccade latency Range of 50 to 200 ms, faster on reward trials

Saccade-related brain areas (macaque monkey) FEF: frontal eye field SEF: supplementary eye field LIP: area LIP of parietal cortex CD: caudate nucleus SNr: substantia nigra pars reticulata SC: superior colliculus Clbm: cerebellum SG: brainstem saccade generators

Inputs to Striatal Medium Spiny Neuron Smith & Bolam (1990)

Medium Spiny Neuron in Striatum Preston, Bishop & Kitai (1980)

Single unit recording from Caudate Nucleus

L-CD neuron: All Reward

L-CD neuron: All Reward

Population activity of CD neurons (with contra-bias, n=25) Lauwereyns et al. (2002, Nature)

Weak correlation Strong correlation

General increase

Reward leads to general increase of neural activity = bias effect; no change in d’ Lauwereyns et al. (2002, Neuron) Data from CD

General increase: Prospective, additive Bias in anticipatory activity Linearly enhances sensory activity

General increase: Prospective, additive Bias in anticipatory activity Linearly enhances sensory activity Response = Input + Reward Bias Prefrontal cortex, basal ganglia Superior colliculus

Is it all bias? Or can we find examples of sensitivity?

Improved discrimination

Reward leads to improved discrimination of neural activity = change in d’, no bias effect Kobayashi et al. (2002, J. Neurophysiol.) Data from DLPFC

Improved discrimination: Synergistic, multiplicative Sensory properties Non-linearly enhanced by reward

Improved discrimination: Synergistic, multiplicative Sensory properties Non-linearly enhanced by reward Response = Input * Reward Gain Prefrontal cortex, parietal cortex Superior colliculus

Never the twain shall meet?

Improved discrimination & General increase

Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus

Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus

Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus Response = (Input * Reward Gain) + Reward Bias On toward the oculomotor plant

Dopamine

Excitation

Dopamine Excitation Disinhibition

Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity

Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity Bias

Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity Bias Thalamus Back to LPFC, On to posterior cortices, Back to CD

… … Only prefrontal cortex? Evolution of the dopamine system: toward innervation of more and more cortex Nieoullon, 2002

Effects of methamphetamine (METH) (speed)

Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition D1 > D2 D2 > D1 Thalamus Back to LPFC, On to posterior cortices, Back to CD