1 Asymmetric Encoding of Positive and Negative Values in the Basal Ganglia Mati Joshua Avital Adler Hagai Bergman Hagai Bergman

2 Layout Introduction Introduction Experimental design Experimental design Reward vs. Aversive Reward vs. Aversive Comparing Basal Ganglia populations Comparing Basal Ganglia populations Summary Summary

3 Introduction - Anatomy Main axis Main axis –Mainly Feed forward –GABA. Glutamate –Ionotropic –MSN,GP,SNr Modulators Modulators –Lateral connections –DA, ACh, 5-HT –Metabotropic –TAN, SNc, DR Breakefield et al, Nature Review 2008

4 The Basal Ganglia as a soccer fan: Penalty Kick Orange fan Basal Ganglia Blue fan Basal Ganglia

5 Introduction – Function Fiorillo et al Science2003 Houk et al 1995 Probabilistic Classical conditioning 75% 25% Reinforcement Learning models Experimental results Dopaminergic (DA) neurons code error signal

6 See also: Fiorillo et. al. Science 2003 Satoh et. al JNS 2003 Pan et. al JNS 2005 Bayer et. al Neuron 2005 Morris et. al. Neuron 2004 Floor effect does not enable symmetric encoding of prediction errors by DA neurons RewardNo Reward Time(ms) 0400 Time(ms) 0400

7 Research objectives Positive vs. Negative values in the Basal Ganglia Positive vs. Negative values in the Basal Ganglia –Encoding of aversive events Comparing neuro-modulators and main axis Comparing neuro-modulators and main axis Reward+AversiveReward only

8 Probabilistic classical conditioning with aversive and reward

9 Monkey is expecting food and air-puff

10 Extracellular recording of Basal Ganglia activity Modulators Main axis

11 The Basal Ganglia database Inclusion criteria: Analysis database: 2 monkeys (L and S) 890 neurons ( TANs-180; SNc-106; SNr-145; GPe-310; GPi-149) Mean Isol. Score = 0.91 (TANs:0.91; SNc:0.78; SNr,GPe, GPi:0.95) Mean fraction of ISIs < 2 ms = (~2/1000) Mean recording time = 56 minutes Mean recording trials= 320 trials Joshua M; Elias; Levine and Bergman, Quantifying the isolation quality of extracellularly recorded action potentials, J. of Neuroscience Methods, 163(2):267-82, 2007 Fraction of ISIs < 2 ms < 0.02 Stable Recording time > 20 minutes Isolation score > 0.8 (SNc>0.5) Recording database : 2002 units

12 Example - GPe

13 Population response to cue Reward > Aversive Reward > Aversive Aversive = Neutral Aversive = Neutral High reward probability> low reward probability High reward probability> low reward probability SNc positive response to aversive expectation SNc positive response to aversive expectation

14 All populations But But –Outliers –Cancellation of opposite effects Single cell analysis Single cell analysis SNc

15 Response index analysis Response index = abs (reward/aversive – neutral) Encoding expectation: Reward > Aversive Example Population Reward Response index (Spike/s) Aversive Response index (spike/s)

16 Encoding expectation: Reward > Aversive

17 Encoding probability: Reward > Aversive Probability coding index analysis Example Population Probability Coding index = abs (high – low)

18 Encoding probability: Reward > Aversive

19 Mid - summary Basal Ganglia encodes positive expectations Basal Ganglia Not BG Cerebellum? Amygdala?

20 Modulator phasic response vs. main axis sustained response Main Axis Modulators

21 Homogenous vs. Diverse Main Axis Modulators Increase/Decrease analysis Response correlation Reward Aversive

22 Summary main axis vs. modulators Modulators Modulators –Fast phasic homogeneous response –Coding the difference between expectations and reality Main Axis Main Axis –Sustained and diverse response –Coding the value of the current state Critic(error)Actor(value) ShortSustained Response pattern HomogenousDiverse Response type ++ Cue probability coding Opposite cue trend - Outcome probability coding

23 From cue to outcome Reminder – DA encoding of reward omission is limited Reminder – DA encoding of reward omission is limited –Learning values with limited RL error is impaired (Mitelman et. al. IBAGS IX) Morris et. al. Neuron 2004

24 Population analysis of the Basal Ganglia modulators - SNc Same polarity to aversive and reward events – More than reward error signal. Same polarity to aversive and reward events – More than reward error signal. No difference between omissions No difference between omissions Time(s) Spike/s

25 Population analysis of the Basal Ganglia modulators -TAN n=180 TANs Time (s ) Responses to cue are similar Responses to cue are similar Large difference in the no outcome epoch Large difference in the no outcome epoch Time(s) Spike/s

26 Modulators have complementary response SNc encodes cue but not omission SNc encodes cue but not omission TAN encode omission but not cue TAN encode omission but not cue Modulators encode more than reward error Modulators encode more than reward error

27 Summary Reward >> Aversive Reward >> Aversive Main axis vs. modulators Main axis vs. modulators TAN vs. SNc TAN vs. SNc Na ï ve model Cerebellum ? Amygdala? Value segregated Value-error Multiple errors

28 Future directions I Event related correlation Reward Aversive ITI not Correlated ITI Correlated

29 Future directions II Learning new cues 66% 33% SNr cell

30 Thanks Avital Adler Avital Adler Boris Rosin Boris Rosin Hagai Bergman Hagai Bergman Inna Finkes Inna Finkes Rea Mitelman Rea Mitelman Yael Renernt Yael Renernt Yifat Prut Yifat Prut All the BG lab All the BG lab

31

32 All population analysis Main axis Modulators

33 Monkey is expecting food and air-puff

34 Response index - modulators Cue OutcomeNo Outcome

35 Summary Main Axis vs. Modulators OmissionOutcomeCue 0+++reward 0---aversive OmissionOutcomeCue-++reward +--aversive Value – SustainedError -Phasic