Dynamic Decision Making in Complex Task Environments: Principles and Neural Mechanisms Progress and Future Directions November 17, 2009

A High-Stakes, Time-Critical Decision A diffuse form is coming toward you rapidly: What should you do? –You could shoot at it, but it may be your friend –You can hold your fire, but it might shoot you! –You could wait to decide, but that might be risky too How can we optimize our choices, and the timing of our choices, in time-critical, uncertain situations?

FY07 MURI BAA Topic 15 Building Bridges between Neuroscience, Cognition, and Human Decision Making Objective: The general goal is to form a complete and thorough understanding of basic human decision processes … by building a lattice of theoretical models with bridges that span across fields …. The main effort of this work is intended to be in the direction of new integrative theoretical developments … using mathematical and/or computation modeling … accompanied and supported by rigorous empirical model tests and empirical model comparisons ….. From BAA , Topic 15

Our MURI Grant Builds on past neurophysiological, behavioral and theoretical investigations of the dynamics of decision making in humans and non-human primates toward the development of an integrated theory. Extends the empirical effort by employing fMRI, EEG, and MEG convergently to understand the distributed brain systems involved in decision making. Bridges to investigations concerned with decision making processes in real-life situations (e.g. those faced by air-traffic controllers and pilots).

Who We Are PIs: –McClelland, Newsome (Stanford) –Holmes, Cohen (Princeton) –Urban (Carnegie Mellon) –Johnston (NASA Ames) And many other professional scientists, post- docs, and graduate students in applied mathematics, neuroscience, cognitive science, and engineering

A Classical Model of Decision Making: The Drift Diffusion Model of Choice Between Two Alternative Decisions At each time step a small sample of noisy information is obtained; each sample adds to a cumulative relative evidence variable. Mean of the noisy samples is + for one alternative, – for the other, with standard deviation . When a bound is reached, the corresponding choice is made. Alternatively, in ‘time controlled’ or ‘interrogation’ tasks, respond when signal is given, based on value of the relative evidence variable.

The DDM is an optimal model, and it is consistent with neurophysiology It achieves the fastest possible decision on average for a given level of accuracy It can be tuned to optimize performance under different kinds of task conditions –Different prior probabilities –Different costs and payoffs –Variation in the time between trials… The activity of neurons in a brain area associated with decision making seems to reflect the DD process

Neural Basis of Decision Making in Monkeys (Shadlen & Newsome; Roitman & Shadlen, 2002) RT task paradigm of R&T. Motion coherence and direction is varied from trial to trial.

Neural Basis of Decision Making in Monkeys: Results Data are averaged over many different neurons that are associated with intended eye movements to the location of target.

Two Problems with the DDM Accuracy should gradually improve toward ceiling levels as more time is allowed, even for very hard discriminations, but this is not what is observed in human data. The model predicts correct and incorrect RT’s will have the same distribution, but incorrect RT’s are generally slower than correct RT’s. Hard -> Easy RT Errors Correct Responses Prob. Correct Hard Easy

Usher and McClelland (2001) Leaky Competing Accumulator Model Inspired by known neural mechanisms Addresses the process of deciding between two alternatives based on external input ( 1 +  2 = 1) with leakage, mutual inhibition, and noise: dx 1 /dt =  1 -k(x 1 )–f(x 2 )+ 1 dx 2 /dt =  2 -k(x 2 )–f(x 1 )+ 2 f(x) = [x] +

Wong & Wang (2006) ~Usher & McClelland (2001)

One-dimensional reduced version of the LCAM (U&M, 2001; Bogacz et al, 2006) Neglect the non-linearity in the LCAM: dx 1 /dt =  1 -k(x 1 )–(x 2 )+ 1 dx 2 /dt =  2 -k(x 2 )–(x 1 )+ 2 Then subtract and let x = x 1 -x 2, I =  1 -  2 ;  1  2 To obtain: dx/dt = I -k(x)+(x)+ This allows precise mathematical analysis, while approximating the outcomes found in the full non-linear version of the LCA… And it reduces to the DDM if k- = 0 So can see a direct link between neural mechanisms and optimal decision making under uncertainty

Roles of k and  x1 – x2 represents the difference in activation of the two accumulators for the same value of  1 –  2. Time proceeds from stimulus onset. Distribution of values of x1-x2 is shown at three different time points for three combinations of  and  Bifurcation Produces A Decision- Like Outcome

Time-accuracy curves for different |k-| |k-  = 0 |k-  =.2 |k-  =.4

Can we Distinguish Alternative Explanations? LCAM fits time-accuracy data well, but there are other possible reasons for bounded accuracy –Trial-to-trial variation in the direction of drift –Bounded integration prior to the termination of the stimulus Distinguishing these alternatives as best we can is one of the goals of our project

Specific Aims Aim 1: Extend the theory of the dynamics of decision making to address integration of stimulus and payoff information in real-time decision making task situations, integrating behavioral, neuroscientific, and theoretical investigations. Aims 2 and 3: Extend this multi-pronged approach to more complex tasks and task environments –Continuous time and space –Uncertain timing of stimulus onset –Real-life situations including distraction, multi- tasking, and payoff uncertainty

Some of the Questions we Will be Addressing in Our Work To what extent can humans and other participants achieve optimality in decision making, in a range of different decision contexts? What can we learn from deviations from optimality? Within the space of dynamic decision making models, can we find evidence that distinguishes among alternatives consistent with existing data? Can we identify the brain mechanisms that underlie the decision process and determine how these mechanisms achieve optimal performance? To what extent are the dynamics of decision making fixed characteristics of the decision mechanism, and to what extent are the tunable to task demands? If so, how is this tuning achieved?

The Schedule 0930 Introduction and scientific background Jay McClelland 1000 Integrating Payoff and stimulus information Bill Newsome 1030 Modeling the dynamics of choice behavior Phil Holmes 1100 Break 1115 Integration of payoff and stimulus information in humans Jay McClelland 1145 Using fMRI to identify decision structures in humans Jon Cohen 1215 Discussion 1230 Lunch 1330 EEG and MEG studies of decision dynamics Patrick Simen 1400 Effects of stimulus perturbations and switches on decision dynamics Juan Gao 1430 Developing continuous measures of decision state Joel Lachter & James Johnston 1430 Break 1515 Multi-single unit recording studies in primates Bill Newsome 1545 Other future directions and general discussion Jay McClelland