1. 1 TEMPORAL DYNAMICS OF DECISION-MAKING DURING MOTION PERCEPTION IN THE VISUAL CORTEX (2008) Vision Research, 48, 1345-1373 Praveen K. Pilly Stephen Grossberg

2. 2 Decision-Making? Cognitive decision-making Perceptual decision-making

3. 3 Motivation

4. 4 Main Questions How does the brain make perceptual decisions? How do we decide the direction of a moving object embedded in clutter? How does the brain perform a direction discrimination task in a context-appropriate manner?

5. 5 Motion Direction Discrimination Experiments VALUABLE PARADIGM Train monkeys to discriminate the direction of a random dot motion stimulus report the judgment via a choice saccade Record behavior and area LIP neuronal responses Shadlen & Newsome, 2001 Roitman & Shadlen, 2002

6. 6 Random Dot Motion Stimulus Interleaving of 3 uncorrelated random dot sequences Coherence level: the fraction of dots moving non-randomly 60 Hz frame rate Signal dots move from frame n to frame n+3, frame n+3 to frame n+6, and so on

7. 7 3.2% MORE AMBIGUITY Two-Alternative Forced Choice Task Right or Left?

8. 8 51.2% LESS AMBIGUITY Two-Alternative Forced Choice Task Right or Left?

9. 9 Two Experimental Contexts REACTION TIME (RT) FIXED DURATION (FD) Unlimited viewing duration before saccade in the judged direction Fixed viewing duration before saccade in the judged direction Shadlen and Newsome, 2001 Roitman and Shadlen, 2002 Roitman and Shadlen, 2002

10. 10 Data from the Experiments Accuracy of decisions in both FD and RT tasks as a function of coherence Reaction time of decisions in the RT task as a function of coherence for correct and error trials Area LIP neuronal responses during correct and error trials in both FD and RT tasks for various coherences Correlation between the temporal dynamics of LIP responses and saccadic behavior (accuracy, reaction time of decisions) Differences between sensory MT/MST and decision LIP responses

11. 11 Existing Proposals / Models BAYESIAN INFERENCE IN THE BRAIN Beck et al., 2008; Gold & Shadlen, 2001, 2007; Jazayeri & Movshon, 2006; Ma et al., 2006; Pouget et al., 2003; Rao, 2004 NEURAL MODELS Ditterich, 2006a, 2006b; Mazurek et al., 2003; Wang, 2002 Abstract; Non-neural; Propose explicit Bayesian decoders in brain areas Do not clarify important computations that need to occur between the motion stimulus and saccadic response Have a number of issues that need to addressed Rev. Thomas Bayes Treatise on Man (Rene Descartes)

12. 12 MOtion DEcision (MODE) Model MOTION BCS: Grossberg et al., 2001 Berzhanskaya et al., 2007 Contextual gating of response Choice of saccadic response Winning direction chosen and fed back to MT Pool signals over multiple orientations, opposite contrast- polarities, both eyes, multiple depths, and a larger spatial range FT signals are strengthened, ambiguous signals weakened Evidence accumulation amplifies feature tracking (FT) signals Local directional signals Random dot motion input Non-directional signals

13. 13 Motion Processing from Retina to Area MST Geometric aperture problem BARBERPOLE ILLUSION Feature tracking signals Percept Ambiguous signals How do sparse feature tracking signals capture so many ambiguous signals to determine the global motion direction?

14. 14 Local Directional Signals Fried et al., 2002, 2005 Null direction inhibition model Barlow & Levick, 1965 Grossberg et al., 2001

15. 15 Short-Range Motion Signals Local directional processes can be fooled by low coherence multiple dots interleaving of uncorrelated dot sequences

16. 16 Do Random Dot Motion Stimuli pose an Aperture Problem? INFORMATIONAL APERTURE PROBLEM

17. 17 MT-MST Circuit: Motion Capture Inter-directional competition across space in area MST Directionally-asymmetric feedback inhibition from area MST to area MT across space

18. 18 MT and MST Responses during Stimulus Viewing MODEL SIMULATIONS MT MST Britten et al., 1993 MT pref null

19. 19 Informational Aperture Problem Directional short-range filters (V1) 51.2% coherence Rightward motion

20. 20 Informational Aperture Problem Resolution Area MST 51.2% coherence Rightward motion Effectiveness of the motion capture process is limited by coherence level and also viewing duration

21. 21 LIP Recurrent Competitive Field (RCF) Grossberg, 1973+ Self-normalizes total activity like computing real-time probabilities Recurrent on-center off-surround shunting network RCFs have also been used to model reach decisions in dorsal premotor cortex Cisek, 2006 Noise-saturation problem

22. 22 Stochastic LIP RCF

23. 23 RT Task Simulations Sample Correct Trials RT Task

24. 24 LIP Responses during RT Task Correct Trials SimulationsRoitman & Shadlen, 2002 More coherence in preferred direction causes: Faster cell activation More coherence in opposite direction causes: Faster cell inhibition Coherence stops playing a role in the final stages of LIP firing for preferred choices

25. 25 FD Task Simulations Sample Correct Trials FD Task The gain of the LIP response is greater in the RT version of the task when compared to the FD task Roitman & Shadlen, 2002

26. 26 LIP Responses during FD Task Correct Trials More coherence in preferred direction causes: Faster cell activation Higher maximal cell activation More coherence in opposite direction causes: Faster cell inhibition Lower minimal cell activation Simulations Roitman & Shadlen, 2002

27. 27 Accuracy of Decisions More coherence in the motion causes more accurate decisions Simulations Mazurek et al., 2003 Roitman & Shadlen, 2002 RT task accuracy is slightly better than FD task accuracy at lower coherences (< 25%) 50

28. 28 Effect of Viewing Duration on Accuracy in FD Task Gold & Shadlen, 2003 Simulations

29. 29 LIP Responses in the RT Task during Correct and Error Trials Roitman & Shadlen, 2002 Simulations LIP encodes the perceptual decision regardless of the direction and strength of the dots, unlike sensory MT/MST neurons

30. 30 LIP Response Dynamics correlate with Reaction Time Roitman & Shadlen, 2002 Simulations 6.4%

31. 31 Speed of Decisions (RT Task) Correct (-) and Error (- -) Trials More coherence in the motion causes faster reaction time RTs on error trials are greater than those on correct trials SimulationsRoitman & Shadlen, 2002

32. 32 Slower Error Trial RTs? At low coherences, the LIP cell dynamics are controlled more by cellular noise processes As time passes, the likelihood of a wrong LIP cell being chosen increases Slower RT indirectly explains slower rate of change in LIP responses on error trials Brownian motion process

33. 33 Is Motion Direction Discrimination an example of Bayesian Decision-Making? logarithm of the likelihood ratio (logLR) provides a natural currency for trading off sensory information, prior probability and expected value to form a perceptual decision Gold & Shadlen, 2001 S 1 : direction d S 2 : opposite direction D I: spatio-temporal input logLR is proposed to be equivalent to opponent motion read-out How does this explain decision-making properties in response to a variety of perceptual stimuli and task conditions?

34. 34 Bayesian Inference is a Popular Hypothesis This approach does provides an intuitive framework Does it disclose brain mechanisms underlying perception and decision- making? Probabilistic nature of decision- making in response to uncertainty Neuronal variability Bayesian inference in the brain? Gold & Shadlen, 2001, 2007 Knill & Pouget, 2004 Pouget et al., 2003; etc.

35. 35 Brain without Bayes … We question the popular wisdom that the brain operates as an information-processing device that performs probabilistic inference … Shadlen et al., 2008 … a categorical decision is readout by a Bayesian decoder... Our work suggests that explicit representation of probability densities by neurons might not be necessary … Furman & Wang, 2008 Grossberg & Pilly, 2008 … This generality is part of its [Bayes rule] broad appeal, but is also its weakness in not proving enough constraints to discover models of any particular science …

36. 36 Wang, 2002; Wong & Wang, 2006; Mazurek et al., 2003; Ditterich, 2006a, 2006b Comparison to other Neural Models Our model goes beyond alternative models: Uses the real-time perceptual stimuli used in the experiments Does not make many of the specialized assumptions of previous models Clarifies the different roles of sensory MT/MST and decision LIP cells Simulates the effect of viewing duration on the psychometric function Incorporates the difference in LIP responsiveness to the two task conditions Considers the visual contribution to LIP response due to choice target Simulates the entire time course of LIP responses during both tasks on both correct and error trials Highlights the important role of BG in contextually gating the saccadic response …

37. 37 MODE Model Predictions Gradual resolution of the informational aperture problem in area MT Pack & Born, 2001 Explanation for the lack of coherence-independent initial transient pause in LIP activity in the FD task, unlike the RT task Lower LIP activity, before motion onset, in multiple-choice tasks Volitional top-down mechanism to make forced choices Churchland et al., 2007 Stimulus manipulations such as: higher dot density more interleaved sequences briefer signal dots should: decrease accuracy increase reaction times have influences on MT, MST, and LIP responses similar to those that occur due to lowering motion coherence