Dynamical Models of Decision Making Optimality, human performance, and principles of neural information processing Jay McClelland Department of Psychology and Center for Mind Brain and Computation Stanford University

Is the rectangle longer toward the northwest or longer toward the northeast?

Longer toward the Northeast! 2.00” 1.99”

How do you decide if an urn contains more black balls or white balls? An Abstract Statistical Theory of the time course of decision making with uncertain information How do you decide if an urn contains more black balls or white balls? We assume you can only draw balls one at a time and want to stop as soon as you have enough evidence to achieve a desired level of accuracy. Optimal policy: Reach in and grab a ball Keep track of difference between the # of black balls and # of white balls. Respond when the difference reaches a criterion value C. Accuracy can be improved by increasing C but this slows down the decision. Policy produces fastest decisions for specified level of accuracy at a given level of discriminability, (b-w)/(b+w)

The Drift Diffusion Model Continuous version of the SPRT At each time step a small random step is taken. Mean direction of steps is +m for one direction, –m for the other. When criterion is reached, respond. Alternatively, in ‘time controlled’ tasks, respond when signal is given.

Two Problems with the DDM Easy Accuracy should gradually improve toward ceiling levels, even for very hard discriminations, but this is not what is observed in human data. The model predicts correct and incorrect RT’s will have the same distribution, but incorrect RT’s are generally slower than correct RT’s. Prob. Correct Hard Errors Correct Responses RT Hard -> Easy

The search for emergent regularity from complex neural processes Human behavior is often characterized by simple regularities at an overt level, yet this simplicity arises from a highly complex underlying neural mechanism. Can we understand how these simple regularities could arise?

Usher and McClelland (2001) Leaky Competing Accumulator Model Addresses the process of deciding between two alternatives based on external input (r1 + r2 = 1) with leakage, self-excitation, mutual inhibition, and noise: dy1/dt = r1-l(y1)+af(y1)–bf(y2)+x1 dy2/dt = r2-l(y2)+af(y2)–bf(y1)+x2

Wong & Wang (2006) ~Usher & McClelland (2001)

Dynamics of winning and loosing populations in the brain and the LCAM

Roles of (k = l–a) and b

Testing the model Quantitative test: Qualitative test: Predicted shapes of ‘time-accuracy curves’ Qualitative test: Understanding the dynamics of the model leads to a novel prediction

Time-accuracy curves for different |k-b|

Assessing Integration Dynamics Participant sees stream of S’s and H’s Must decide which is predominant 50% of trials last ~500 msec, allow accuracy assessment 50% are ~250 msec, allow assessment of dynamics Equal # of S’s and H’s But there are clusters bunched together at the end (0, 2 or 4). Leak-dominant Inhibition-dom. Favored early Favored late

Subjects show both kinds of biases; the less the bias, the higher the accuracy, as predicted.

Extension to N alternatives Extension to n alternatives is very natural (just add units to pool). Model accounts quite well for Hick’s law (RT increases with log n alternatives), assuming that threshold is raised with n to maintain equal accuracy in all conditions. Use of non-linear activation function increases efficiency in cases where there are only a few alternatives ‘in contention’ given the stimulus.

Neural Basis of Decision Making in Monkeys (Roitman & Shadlen, 2002) RT task paradigm of R&T. Motion coherence and direction is varied from trial to trial.

Neural Basis of Decision Making in Monkeys: Results Data are averaged over many different neurons that are associated with intended eye movements to the location of target.

Wong & Wang (2006)

Behavioral Data

Integration of reward and motion information (Rorie & Newsome) Monkey’s choices reflect a beautifully regular combination of the effects of stimulus coherence and reward.

Population response of LIP Neurons in two reward conditions Choose in Choose out

Some Open Questions How localized vs. distributed are the neural populations involved in decisions? What does it mean in the brain to pass the threshold for response initiation? How are brain dynamics tuned to allow a practiced participant to optimize decisions, combining prior and current information in just the right mix to achieve (near) optimality?