1. The Computing Brain: Focus on Decision-Making
Cogs 1
March 5, 2009
Angela Yu

2. Understanding the Brain/Mind? A powerful analogy: the computing brain
Behavior
Neurobiology ?
Cognitive
Neuroscience ≈
A powerful analogy: the computing brain

3. An Example: Decision-Making?

4. An Example: Decision-Making
What are the computations involved?
THUR
74/61
2. Uncertainty ?
1. Decision
3. Costs:
Health?
Coolness?

5. Monkey Decision-Making
Properties: info/time slowed down, linear in time, in controlled exp.
Random dots coherent motion paradigm

6. Random Dot Coherent Motion Paradigm
Properties: info/time slowed down, linear in time, in controlled exp.
30% coherence
5% coherence

7. How Do Monkeys Behave? Accuracy vs. Coherence <RT> vs. Coherence
(Roitman & Shadlen, 2002)

8. What Are the Neurons Doing? MT: Sustained response
Saccade generation system
MT: Sustained response
Time
Response
99.9%
MT tuning function
Direction ()
Preferred
Anti-preferred
Time
Response
25.6%
(Britten & Newsome, 1998)
(Britten, Shadlen, Newsome, & Movshon, 1993)

9. What Are the Neurons Doing?
Saccade generation system
LIP: Ramping response
(Roitman & Shalden, 2002)
(Shadlen & Newsome, 1996)

10. A Computational Description
1. Decision: Left or right? Stop now or continue?
Properties: info/time slowed down, linear in time, in controlled exp.
(t): input(1), …, input(t)  {left, right, wait}

11. Timed Decision-Making
wait
wait L R L R L R

12. A Computational Description
2. Uncertainty: Sensory noise (coherence)
Properties: info/time slowed down, linear in time, in controlled exp.

13. Timed Decision-Making
wait
wait L R L R L R

14. A Computational Description
3. Costs: Accuracy, speed
Properties: info/time slowed down, linear in time, in controlled exp.
cost() = Pr(error) + c*(mean RT)
Optimal policy * minimizes the cost

15. Timed Decision-Making
+: more accurate
wait
wait
-: more time L R L R L R

16. Mathematicians Solved the Problem for Us
Wald & Wolfowitz (1948):
Optimal policy:
accumulate evidence over time: Pr(left) versus Pr(right)
stop if total evidence exceeds “left” or “right” boundary
“left” boundary
“right” boundary
Pr(left)

17. Model  Neurobiology Model LIP Neural Response
Saccade generation system
Model
LIP Neural Response

18. Model  Behavior Model RT vs. Coherence Saccade generation system
boundary
Coherence

19. Putting it All Together
Saccade generation system
MT neurons signal motion direction and strength
LIP neurons accumulate information over time
LIP response reflect behavioral decision (0% coherence)
Monkeys behave optimally (maximize accuracy & speed)

20. Understanding the Brain/Mind? A powerful analogy: the computing brain
Behavior
Neurobiology ?
Cognitive
Neuroscience ≈
A powerful analogy: the computing brain

22. Sequential Probability Ratio Test
Log posterior ratio
undergoes a random walk:
rt is monotonically related to qt, so we have (a’,b’), a’<0, b’>0. b’ a’ rt
In continuous-time, a drift-diffusion process w/ absorbing boundaries.

23. Generalize Loss Function
We assumed loss is linear in error and delay:
What if it’s non-linear, e.g. maximize reward rate:
(Bogacz et al, 2006)
Wald also proved a dual statement:
Amongst all decision policies satisfying the criterion
SPRT (with some thresholds) minimizes the expected sample size <>.
This implies that the SPRT is optimal for all loss functions that
increase with inaccuracy and (linearly in) delay (proof by contradiction).

24. Neural Implementation
Saccade generation system
LIP - neural SPRT integrator?
(Roitman & Shalden, 2002; Gold & Shadlen, 2004)
Time
LIP Response & Behavioral RT
LIP Response & Coherence

25. Caveat: Model Fit Imperfect
(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
Accuracy
Mean RT
RT Distributions

26. Fix 1: Variable Drift Rate
(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
(idea from Ratcliff & Rouder, 1998)
Accuracy
Mean RT
RT Distributions

27. Fix 2: Increasing Drift Rate
(Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007)
Accuracy
Mean RT
RT Distributions

28. A Principled Approach
Trials aborted w/o response or reward if monkey breaks fixation
(Data from Roitman & Shadlen, 2002)
(Analysis from Ditterich, 2007)
More “urgency” over time as risk of aborting trial increases
Increasing gain equivalent to decreasing threshold

29. Imposing a Stochastic Deadline
(Frazier & Yu, 2007)
Loss function depends on error, delay, and whether deadline exceed
Optimal Policy is a sequence of monotonically decaying thresholds
Time
Probability

30. delta, gamma, exponential, normal
Timing Uncertainty
(Frazier & Yu, 2007)
Timing uncertainty  lower thresholds
Time
Probability
Theory applies to a large class of deadline distributions:
delta, gamma, exponential, normal

31. Some Intuitions for the Proof
(Frazier & Yu, 2007)
Continuation
Region
Shrinking!

32. Summary Review of Bayesian DT: optimality depends on loss function
Decision time complicates things: infinite repeated decisions
Binary hypothesis testing: SPRT with fixed thresholds is optimal
Behavioral & neural data suggestive, but …
… imperfect fit. Variable/increasing drift rate both ad hoc
Deadline (+ timing uncertainty)  decaying thresholds
Current/future work: generalize theory, test with experiments