Presentation is loading. Please wait.

Presentation is loading. Please wait.

Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models? Alan Pickering Department of Psychology

Similar presentations


Presentation on theme: "Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models? Alan Pickering Department of Psychology"— Presentation transcript:

1 Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models? Alan Pickering Department of Psychology

2 Overview Classic CL findings and questions DA, the striatum and learning Generate simple hypothesis about CL deficits in Parkinsons Disease Generate simple biologically- constrained neural net to test hypothesis Simulate CL data on 2 types of matched CL tasks Conclusions – why model fails

3 Classic Findings and Questions Parkinsons Disease (PD) patients are impaired at CL tasks. Why? -What psychological processes are impaired? -What brain regions and neuro- transmitters are involved?

4 Category Learning in Parkinsons Disease Weather task: Knowlton et al, 1996

5 Category Learning in Parkinsons Disease Main Findings: Knowlton et al, 1996

6 Key Facts PD involves prominent damage to the striatum CL may (sometimes) involve procedural/habit learning Striatal structures are part of cortico-striato-pallido-thalamic loops possibly implicated in procedural learning The striatum is strongly innervated by ascending DA projections

7 Simple Interpretation CL deficits in PD may arise because of damage to … loss of ascending DA signals which compromise the functioning of (parts of) … the striatum

8 Three Learning Processes Which Might Be DA-Related 1.Appetitive reinforcement and motivation DA cell firing increses/decreases provide a positive/negative reinforcement signal which is required for synaptic strengthening/ weakening 3-factor learning rule (e.g., Wickens; Brown et al etc.)

9 Corticostriatal (Medium Spiny Cell) Synapse

10 DA Receptors in Striatum After Schultz, 1998 DA receptors: Unfilled rectangles GLU receptors: Filled rectangles

11 DA-Related Processes (cont) 2.Reward Prediction Error Midbrain DA neurons increase firing in response to unexpected rewards and decrease firing to nonoccurrence of expected rewards Firing change= reward prediction error Schultz, Suri, Dickinson, Dayan etc.

12 DA Cell Recordings: Evidence For Reward Prediction Error CUEREWARD

13 DA-Related Processes (cont) 3.Modulation of Neural Signals Floresco et al (2001): DA receptor activity serves to strengthen salient inputs while inhibiting weaker ones Also: Nicola & Malenka; J.D.Cohen; Ashby & Cassale; Salum et al; Nakahara; Schultz

14 Evidence For Modulation Nicola & Malenka, 1997 Recorded effect of DA on response of striatal cells to strong and weak inputs Strong Weak

15 Linking 3-Factor Learning & Reward Prediction Error Cue Reward Striatal Cell DA Cell Reward prediction Reward prediction error ExcitatoryInhibitory Reinforcement

16 Simple Working Hypothesis CL is impaired in PD patients (and other DA-compromised groups) due to reduced DA function in striatum (tail of caudate) The loss of ascending DA input reduces the reinforcing function of the reward prediction error signal innervating the striatum

17 Modelling Biologically-constrained neural net Data to be simulated taken from Ashby et al (2003) Data from young and old controls (YC, OC) and PD patients Study used matched CL tasks: rule-based (RB) and Information Integration (II) Ashby and colleagues believe these tasks are handled by distinct CL systems

18 Ashby et al: II Task 3 of the 4 dimensions determine categories Not readily verbalisable Cat A Cat B

19 Ashby et al: RB Task 1 dimension (background colour) determines category Readily verbalisable rule Cat B Cat A

20 Ashby et al: Results Proportion failing to learning to criterion in 200 trials II Task RB Task

21 RB Task: Results Trials to criterion for learners II TaskRB Task

22 Modelling Constrained by input and output connections of striatum (caudate) Learning rule based on known 3-factor form of synaptic plasticity in striatum Learning rule consistent with reward prediction error properties of DA neurons

23 Connections of Striatum

24 Schematic Model Reward prediction

25 Model Learning Rule When reward present, E>0 w JK = k R *E*y k out *x J out When reward absent, E<0 w JK = k N *E*y k out *x J out x J out y K out yKyK Reward prediction error, E w JK

26 Modelling of Reduced DA Function Loss of DA input to striatum (tail of caudate) modelled 2 ways (with same results):- a) loss of modifiability of cortico- striatal weights b) proportional reduction of reward prediction error strength Mean proportion of weights modifiable:- YC 0.8 OC 0.5PD 0.2 (with s.d. = 0.15)

27 Modelling Process Found parameters which gave good fit to YC performance on II task and set DA parameters for PD to produce appropriate level of nonlearners on same task Varied OC DA values between YC and PD Looked at fit (with these parameters) to all other data cells esp. RB task

28 Modelling II Task Results Trials to criterion (learners) Proportion of non- learners YC PD

29 Modelling II Task Results Trials to criterion (learners) Proportion of non- learners YC PDOC

30 Modelling II Task Results* Trials to criterion (learners) Proportion of non- learners YC PDOC

31 Model Results II Task Performance of learners in blocks of 16 trials

32 Modelling RB Task Results Trials to criterion (learners) Proportion of non- learners YC PDOC

33 Modelling RB Task Results* Trials to criterion (learners) Proportion of non- learners YC PDOC

34 Conclusions & Future Work 1 Simplest realistic model of cortico- striatal learning captures only limited aspects of the CL data Bimodal nature (learn normally vs. fail) of data simulated only under some paramter settings No intermediate DA parameter settings in old controls which can be both PD-like for II task and YC- like for RB task

35 Conclusions & Future Work 2 Model challenges hypothesis under test: PD (and OC) deficits in some CL tasks seem unlikely to be solely due to reduced DA- related reinforcement in striatum Findings are consistent with >1 CL system Future model should add rule system (c.f. Ashbys COVIS)

36 Alan Pickering CL Refs Pickering, A.D., & Gray, J.A. (2001). Dopamine, appetitive reinforcement, and the neuropsychology of human learning: An individual differences approach. In A. Eliasz & A. Angleitner (Eds.), Advances in individual differences research (pp ). Lengerich, Germany: PABST Science Publishers.


Download ppt "Striatal Dopamine (DA) and Learning: Do Category Learning (CL) data constrain computational models? Alan Pickering Department of Psychology"

Similar presentations


Ads by Google