Jay McClelland Stanford University Decision Dynamics and Decision States in the Leaky Competing Accumulator Model Jay McClelland Stanford University

Is the rectangle longer toward the northwest or longer toward the northeast?

Longer toward the Northeast! 2.00” 1.99”

A Classical Model of Decision Making: The Drift Diffusion Model of Choice Between Two Alternative Decisions At each time step a small sample of noisy information is obtained; each sample adds to a cumulative relative evidence variable y. Mean of the noisy samples is +m for when one alternative is correct, –m when the other, with standard deviation s. When a bound is reached, the corresponding choice is made. Alternatively, in ‘time controlled’ or ‘interrogation’ tasks, respond when signal is given Choose choice 1 if y is positive Choose choice 2 if y is negative y

A Problem with the DDM Easy Prob. Correct Hard Accuracy should gradually improve toward ceiling levels, but this is not what is observed in data. Two possible fixes: Trial-to-trial variance in the direction of drift (Ratcliff) Evidence accumulation may reach a bound and stop, even if more time is available (Shadlen and colleagues) Hard Prob. Correct Easy

Usher and McClelland (2001) Leaky Competing Accumulator Model Proposes accumulators of noisy evidence, with leakage, and mutual inhibition: dy1/dt = I1-gy1–bf(y2)+x1 dy2/dt = I2-gy2–bf(y1)+x2 f(y) = [y]+ In time controlled tasks, choose response 1 iff y1-y2 > 0 Let y = (y1-y2). While y1 and y2 are positive, the model reduces to: dy/dt = I-ly+x [I=I1-I2; l = g-b; x=x1-x2] y1 y2 I1 I2

Time course of stimulus sensitivity in the linear approximation:

Time-accuracy curves for different |k-b| or |l|

The Full Non-Linear LCAi Model y1 Although the value of the difference variable is not well-captured by the linear approximation, the sign of the difference is approximated very closely. y2

Result of fitting the full model to individual participant data (Usher & McClelland, 2001) Prob. Correct

Distinguishing Leak Dominance From Inhibition Dominance

Kiani, Hanks and Shadlen 2008 Random motion stimuli of different coherences. Stimulus duration follows an exponential distribution. ‘go’ cue can occur at stimulus offset; response must occur within 500 msec to earn reward.

The earlier the pulse, the more it matters (Kiani et al, 2008)

These results rule out leak dominance Still viable X The bounded DDM and the full non-linear LCAi are also still viable.

Plan for the Rest of the Talk Discuss several interesting features of decision states in the non-linear LCAi Describe three experiments combining experiment and simulation that address these features.

Quasi-Continuous, Quasi-Discrete, Reversible Decision States in the Non-Linear LCAi

v Distribution of winner’s activations when correct alternative wins Distribution of winner’s activations when incorrect alternative wins v

Predictions We should be able to find signs of differences in decision states associated with correct and incorrect responses. We should be able to see signs of bifurcation even if we ask for a continuous response. We should be able see evidence of rebound of suppressed alternatives if the input changes.

Predictions We should be able to find signs of differences in ‘strength’ of decision states associated with correct and incorrect responses. We should be able to see signs of bifurcation when we ask for a continuous response. We should be able to see evidence of recovery of suppressed alternatives if the input changes.

Integration of reward and stimulus information Gao, Tortell & McClelland PLoS One, 2011

Proportion of Choices toward Higher Reward

Fits based on full LCAi

Relationship between response speed and choice accuracy

An Account: High-Threshold LCAi Gao & McClelland, (in preparation)

v Distribution of winner’s activations when correct alternative wins Distribution of winner’s activations when incorrect alternative wins v

v Distribution of activations when correct alternative wins Distribution of activations when incorrect alternative wins v

Predictions We should be able to find signs of differences in ‘strength’ of decision states associated with correct and incorrect responses. We should be able to see signs of bifurcation even when we ask for a continuous response. We should be able to see evidence of recovery of suppressed alternatives if the input changes.

Toward Continuous Measures of Decision States Lachter, Corrado, Johnston & McClelland (in progress)

Mixed difficulty levels: Can participants give a continuous readout when they have as much time to respond as they would like? To test: Participant observes display as long as desired, moves joystick to desired position, then clicks to terminate trial Mixed difficulty levels: Stimuli differ by 1, 2, 4, 8, or 16 dots

Results and Descriptive Model of Data from 1 Participant

Quasi-Discrete, Quasi-Continuous Decision States Bi-modality indicates a degree of discreteness, consistent with the bifurcation expected in the model. The position of each mode should shift as the difference in the number of dots increases, according to the model.

Follow-up Log scale ranging from 1000:1 to 1:1 to 1:1000 (extends and reshapes range) Very explicit instructions about contingencies, marks on scale. Paid for points, length of session depends on participant’s pacing of trials Ten sessions per participant

Two Participants: Session 1

Session 10 for each participant (this and next 6 slides)

Predictions We should be able to find signs of differences in ‘strength’ of decision states associated with correct and incorrect responses. We should be able to see signs of bifurcation even when we ask for a continuous response. We should be able to see evidence of recovery of suppressed alternatives if the input changes.

Decision making with non-stationary stimulus information (Tsetsos Decision making with non-stationary stimulus information (Tsetsos. Usher & McClelland in press) Phase duration distribution Evidence Switching Protocol in the Correlation Condition:

Individual Data from Correlation Condition Primacy region Indifference to starting phase P(C), A/B at start

Simulations of Two Correlated Trials Top: A/B start high Bottom: C starts high

Only LCAi can explain >50% choice of C even when A/B phase comes first Dissimilar favored first Dissimilar favored second Average Top: low noise Bottom: higher noise

Individual Data from Correlation Condition and Model Coverage

Explaining Individual Differences in the LCA Balanced, Strong L&I I > L Lots of Noise

Predictions We should be able to find signs of differences in ‘strength’ of decision states associated with correct and incorrect responses. We should be able to see signs of bifurcation when we ask for a continuous response. We should be able see evidence of recovery of suppressed alternatives.

Conclusions Evidence from several studies is consistent with the idea of quasi-continuous, quasi discrete, sometimes reversible, decision states. The LCAi model provides a simple yet powerful framework in which such states arise. Alternative models considereed have difficulties addressing aspects of the data. More work is needed to understand if the LCAi will turn out to be fully adequate, and how the full set of data might be addressed with other approaches.