1. Neural circuits for bias and sensitivity in decision-making Jan Lauwereyns Associate Professor, Victoria University of Wellington, New Zealand Long-term Invitation Fellow, Japanese Society for the Promotion of Science Visiting Scholar, Tamagawa University, Tokyo, Japan

2. The Perfect Grandpa

4. Biological needs The drive reduction hypothesis Think: Inclusive fitness Think: energy, reproduction Approach Avoid

5. Biological needs The drive reduction hypothesis Several hours have passed since last meal

6. Biological needs The drive reduction hypothesis Several hours have passed since last meal Increased drive (hunger) Increased exploratory activity

7. Biological needs Several hours have passed since last meal Increased drive (hunger) Increased exploratory activity Find food, eat it Drive is reduced (reinforcement) The drive reduction hypothesis

8. Biological needs Several hours have passed since last meal Find food, eat it Drive is reduced (reinforcement) The drive reduction hypothesis Increased drive (hunger) Increased exploratory activity

9. Dopamine reward prediction (Schultz)

10. Executive control Goals, beliefs, wishes, fears… Related to motivational control Some sensory information is valuable to the individual in the sense that it may be used in the strategic (“optimal”) control of behavior Executive control would seek to maximize the extraction of valuable sensory information

11. How can executive control affect information processing? Two general hypotheses: Sensitivity –Selective improvement of information processing (actual perception) Bias: –Selective preparation (“anticipation”) of information processing (virtual perception) For example: “Reward”

12. Distinguishing effects of sensitivity and bias Signal detection theory (Green & Swets) Probability of response LATER model (Carpenter) Latency of response

13. Signal detection theory

16. Signal Noise Neuronal activity Noise Signal +

17. A different way to think about bias and sensitivity…

23. Scheme of the original LATER model (RHS Carpenter)

24. Nose Poke Paradigm: Spatial choice, Gives us good reaction-time distributions

25. Target side: 4 LEDs vs. Distracter side: 0-3 LEDs

26. Lauwereyns & Wisnewski (2006, JEP:ABP)

28. Theoretical example of bias

31. Theoretical example of sensitivity

34. How does it really work?

35. How does the brain incorporate reward value in the control of action?

36. Studied in monkeys using saccadic eye movement tasks with asymmetrical reward schedule

37. Biased Saccade Task (BST)

40. Target position = unpredictable

41. Biased Saccade Task (BST) Reward association = known

42. Biased Saccade Task (BST)

45. No escape!

46. Asymmetric position-reward mapping in “ABA” design Frequent reversal of blocks

47. Strong effect of reward value on saccade latency Range of 50 to 200 ms, faster on reward trials

48. Saccade-related brain areas (macaque monkey) FEF: frontal eye field SEF: supplementary eye field LIP: area LIP of parietal cortex CD: caudate nucleus SNr: substantia nigra pars reticulata SC: superior colliculus Clbm: cerebellum SG: brainstem saccade generators

49. Inputs to Striatal Medium Spiny Neuron Smith & Bolam (1990)

50. Medium Spiny Neuron in Striatum Preston, Bishop & Kitai (1980)

51. Single unit recording from Caudate Nucleus

52. L-CD neuron: All Reward

53. L-CD neuron: All Reward

54. Population activity of CD neurons (with contra-bias, n=25) Lauwereyns et al. (2002, Nature)

55. Weak correlation Strong correlation

58. General increase

59. Reward leads to general increase of neural activity = bias effect; no change in d’ Lauwereyns et al. (2002, Neuron) Data from CD

60. General increase: Prospective, additive Bias in anticipatory activity Linearly enhances sensory activity

61. General increase: Prospective, additive Bias in anticipatory activity Linearly enhances sensory activity Response = Input + Reward Bias Prefrontal cortex, basal ganglia Superior colliculus

64. Is it all bias? Or can we find examples of sensitivity?

67. Improved discrimination

68. Reward leads to improved discrimination of neural activity = change in d’, no bias effect Kobayashi et al. (2002, J. Neurophysiol.) Data from DLPFC

69. Improved discrimination: Synergistic, multiplicative Sensory properties Non-linearly enhanced by reward

70. Improved discrimination: Synergistic, multiplicative Sensory properties Non-linearly enhanced by reward Response = Input * Reward Gain Prefrontal cortex, parietal cortex Superior colliculus

73. Never the twain shall meet?

76. Improved discrimination & General increase

77. Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus

79. Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus

80. Combination of both mechanisms Seen in all areas Loops between FC, BG and SC But most common in Superior Colliculus Response = (Input * Reward Gain) + Reward Bias On toward the oculomotor plant

81. Dopamine

83. Excitation

84. Dopamine Excitation Disinhibition

85. Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity

86. Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity Bias

87. Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition Sensitivity Bias Thalamus Back to LPFC, On to posterior cortices, Back to CD

88. … … Only prefrontal cortex? Evolution of the dopamine system: toward innervation of more and more cortex Nieoullon, 2002

89. Effects of methamphetamine (METH) (speed)

90. Prospective, additive Synergistic, multiplicative Dopamine Excitation Disinhibition D1 > D2 D2 > D1 Thalamus Back to LPFC, On to posterior cortices, Back to CD