Presentation is loading. Please wait.

Presentation is loading. Please wait.

Alan Pickering Department of Psychology

Similar presentations

Presentation on theme: "Alan Pickering Department of Psychology"— Presentation transcript:

1 Alan Pickering Department of Psychology
Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models? Alan Pickering Department of Psychology

2 Overview Classic CL findings and questions
DA, the striatum and learning Generate simple hypothesis about CL deficits in Parkinson’s Disease Generate simple biologically-constrained neural net to test hypothesis Simulate CL data on 2 types of matched CL tasks Conclusions – why model fails

3 Classic Findings and Questions
Parkinson’s Disease (PD) patients are impaired at CL tasks. Why? -What psychological processes are impaired? -What brain regions and neuro- transmitters are involved?

4 Category Learning in Parkinson’s Disease
Weather task: Knowlton et al, 1996

5 Category Learning in Parkinson’s Disease
Main Findings: Knowlton et al, 1996

6 Key Facts PD involves prominent damage to the striatum
CL may (sometimes) involve procedural/habit learning Striatal structures are part of cortico-striato-pallido-thalamic loops possibly implicated in procedural learning The striatum is strongly innervated by ascending DA projections

7 Simple Interpretation
CL deficits in PD may arise because of damage to … loss of ascending DA signals which compromise the functioning of (parts of) … the striatum

8 Three Learning Processes Which Might Be DA-Related
Appetitive reinforcement and motivation DA cell firing increses/decreases provide a positive/negative reinforcement signal which is required for synaptic strengthening/ weakening “3-factor learning rule” (e.g., Wickens; Brown et al etc.)

9 Corticostriatal (Medium Spiny Cell) Synapse

10 DA Receptors in Striatum
After Schultz, 1998 DA receptors: Unfilled rectangles GLU receptors: Filled rectangles

11 DA-Related Processes (cont)
Reward Prediction Error Midbrain DA neurons increase firing in response to unexpected rewards and decrease firing to nonoccurrence of expected rewards Firing change= reward prediction error Schultz, Suri, Dickinson, Dayan etc.

12 DA Cell Recordings: Evidence For Reward Prediction Error

13 DA-Related Processes (cont)
Modulation of Neural Signals Floresco et al (2001): “DA receptor activity serves to strengthen salient inputs while inhibiting weaker ones” Also: Nicola & Malenka; J.D.Cohen; Ashby & Cassale; Salum et al; Nakahara; Schultz

14 Evidence For Modulation
Nicola & Malenka, 1997 Recorded effect of DA on response of striatal cells to strong and weak inputs Strong Weak

15 Linking 3-Factor Learning & Reward Prediction Error
Striatal Cell Cue Reward prediction Reward predictionerror Reward DA Cell Excitatory Inhibitory Reinforcement

16 Simple Working Hypothesis
CL is impaired in PD patients (and other DA-compromised groups) due to “reduced DA function” in striatum (tail of caudate) The loss of ascending DA input reduces the reinforcing function of the reward prediction error signal innervating the striatum

17 Modelling Biologically-constrained neural net
Data to be simulated taken from Ashby et al (2003) Data from young and old controls (YC, OC) and PD patients Study used matched CL tasks: rule-based (RB) and Information Integration (II) Ashby and colleagues believe these tasks are handled by distinct CL systems

18 Ashby et al: II Task 3 of the 4 dimensions determine categories
Not readily verbalisable Cat A Cat B

19 Ashby et al: RB Task 1 dimension (background colour) determines category Readily verbalisable rule Cat A Cat B

20 Ashby et al: Results Proportion failing to learning to criterion in 200 trials II Task RB Task

21 RB Task: Results Trials to criterion for learners II Task RB Task

22 Modelling Constrained by input and output connections of striatum (caudate) Learning rule based on known 3-factor form of synaptic plasticity in striatum Learning rule consistent with reward prediction error properties of DA neurons

23 Connections of Striatum

24 Schematic Model Reward prediction

25 Model Learning Rule wJK = kR*E*ykout*xJout wJK = kN*E*ykout*xJout
Reward prediction error, E When reward present, E>0 wJK = kR*E*ykout*xJout When reward absent, E<0 wJK = kN*E*ykout*xJout

26 Modelling of Reduced DA Function
Loss of DA input to striatum (tail of caudate) modelled 2 ways (with same results):- a) loss of modifiability of cortico- striatal weights b) proportional reduction of reward prediction error strength Mean proportion of weights modifiable:- YC OC 0.5 PD 0.2 (with s.d. = 0.15)

27 Modelling Process Found parameters which gave good fit to YC performance on II task and set DA parameters for PD to produce appropriate level of nonlearners on same task Varied OC DA values between YC and PD Looked at fit (with these parameters) to all other data cells esp. RB task

28 Modelling II Task Results
Proportion of non-learners Trials to criterion (learners) YC PD

29 Modelling II Task Results
Proportion of non-learners Trials to criterion (learners) YC OC PD

30 Modelling II Task Results*
Proportion of non-learners Trials to criterion (learners) YC OC PD

31 Model Results II Task Performance of learners in blocks of 16 trials

32 Modelling RB Task Results
Proportion of non-learners Trials to criterion (learners) YC OC PD

33 Modelling RB Task Results*
Proportion of non-learners Trials to criterion (learners) YC OC PD

34 Conclusions & Future Work 1
Simplest realistic model of cortico-striatal learning captures only limited aspects of the CL data “Bimodal” nature (learn normally vs. fail) of data simulated only under some paramter settings No intermediate DA parameter settings in old controls which can be both PD-like for II task and YC-like for RB task

35 Conclusions & Future Work 2
Model challenges hypothesis under test: PD (and OC) deficits in some CL tasks seem unlikely to be solely due to reduced DA-related reinforcement in striatum Findings are consistent with >1 CL system Future model should add rule system (c.f. Ashby’s COVIS)

36 Alan Pickering CL Refs 2001-
Pickering, A.D., & Gray, J.A. (2001). Dopamine, appetitive reinforcement, and the neuropsychology of human learning: An individual differences approach. In A. Eliasz & A. Angleitner (Eds.), Advances in individual differences research (pp ). Lengerich, Germany: PABST Science Publishers.

Download ppt "Alan Pickering Department of Psychology"

Similar presentations

Ads by Google