From T. McMillen & P. Holmes, J. Math. Psych. 50: 30-57, 2006. MURI Center for Human and Robot Decision Dynamics, Sept 13, 2007. Phil Holmes, Jonathan.

Slides:



Advertisements
Similar presentations
Neural Computation Chapter 3. Neural Computation Outline Comparison of behavioral and neural response on a discrimination task –Bayes rule –ROC curves.
Advertisements

Quasi-Continuous Decision States in the Leaky Competing Accumulator Model Jay McClelland Stanford University With Joel Lachter, Greg Corrado, and Jim Johnston.
Decision Dynamics and Decision States: the Leaky Competing Accumulator Model Psychology 209 March 4, 2013.
How well can we learn what the stimulus is by looking at the neural responses? We will discuss two approaches: devise and evaluate explicit algorithms.
For stimulus s, have estimated s est Bias: Cramer-Rao bound: Mean square error: Variance: Fisher information How good is our estimate? (ML is unbiased:
CS 8751 ML & KDDEvaluating Hypotheses1 Sample error, true error Confidence intervals for observed hypothesis error Estimators Binomial distribution, Normal.
Sequential Hypothesis Testing under Stochastic Deadlines Peter Frazier, Angela Yu Princeton University TexPoint fonts used in EMF. Read the TexPoint manual.
1 Simple Linear Regression Chapter Introduction In this chapter we examine the relationship among interval variables via a mathematical equation.
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 7: Coding and Representation 1 Computational Architectures in.
Distinguishing Evidence Accumulation from Response Bias in Categorical Decision-Making Vincent P. Ferrera 1,2, Jack Grinband 1,2, Quan Xiao 1,2, Joy Hirsch.
Does Math Matter to Gray Matter? (or, The Rewards of Calculus). Philip Holmes, Princeton University with Eric Brown (NYU), Rafal Bogacz (Bristol, UK),
Theory of Decision Time Dynamics, with Applications to Memory.
An Integrated Model of Decision Making and Visual Attention Philip L. Smith University of Melbourne Collaborators: Roger Ratcliff, Bradley Wolfgang.
1 Computational Vision CSCI 363, Fall 2012 Lecture 31 Heading Models.
FMRI Methods Lecture7 – Review: analyses & statistics.
IE241: Introduction to Hypothesis Testing. We said before that estimation of parameters was one of the two major areas of statistics. Now let’s turn to.
Xiao-Jing Wang Department of Neurobiology Yale University School of Medicine The Concept of a Decision Threshold in Sensory-Motor Processes.
Optimality, robustness, and dynamics of decision making under norepinephrine modulation: A spiking neuronal network model Joint work with Philip Eckhoff.
Biological Modeling of Neural Networks: Week 12 – Decision models: Competitive dynamics Wulfram Gerstner EPFL, Lausanne, Switzerland 12.1 Review: Population.
Decision Making Theories in Neuroscience Alexander Vostroknutov October 2008.
Collective neural dynamics and drift-diffusion models for simple decision tasks. Philip Holmes, Princeton University. Eric Brown (NYU), Rafal Bogacz (Bristol,
Dynamic Decision Making in Complex Task Environments: Principles and Neural Mechanisms Annual Workshop Introduction August, 2008.
Dynamic Decision Making in Complex Task Environments: Principles and Neural Mechanisms Progress and Future Directions November 17, 2009.
D ECIDING WHEN TO CUT YOUR LOSSES Matt Cieslak, Tobias Kluth, Maren Stiels & Daniel Wood.
BCS547 Neural Decoding. Population Code Tuning CurvesPattern of activity (r) Direction (deg) Activity
A View from the Bottom Peter Dayan Gatsby Computational Neuroscience Unit.
BCS547 Neural Decoding.
The Computing Brain: Focus on Decision-Making
Modeling interactions between visually responsive and movement related neurons in frontal eye field during saccade visual search Braden A. Purcell 1, Richard.
What’s optimal about N choices? Tyler McMillen & Phil Holmes, PACM/CSBMB/Conte Center, Princeton University. Banbury, Bunbury, May 2005 at CSH. Thanks.
Decision Dynamics and Decision States in the Leaky Competing Accumulator Model Jay McClelland Stanford University With Juan Gao, Marius Usher and others.
6. Population Codes Presented by Rhee, Je-Keun © 2008, SNU Biointelligence Lab,
Optimal Eye Movement Strategies In Visual Search.
Maximum likelihood estimators Example: Random data X i drawn from a Poisson distribution with unknown  We want to determine  For any assumed value of.
Response dynamics and phase oscillators in the brainstem
Progress in MURI 15 ( ) Mathematical modeling of decision behavior. AFOSR, Alexandria, VA, Nov 17th, 2009 Phil Holmes 1. Optimizing monkeys? Balancing.
Dynamics of Reward and Stimulus Information in Human Decision Making Juan Gao, Rebecca Tortell & James L. McClelland With inspiration from Bill Newsome.
The Physics of Decision-Making: Cognitive Control as the Optimization of Behavior Gary Aston-Jones ∞ Rafal Bogacz * † ª Eric Brown † Jonathan D. Cohen.
Statistics for Business and Economics Module 1:Probability Theory and Statistical Inference Spring 2010 Lecture 4: Estimating parameters with confidence.
Psychology and Neurobiology of Decision-Making under Uncertainty Angela Yu March 11, 2010.
Dynamics of Reward Bias Effects in Perceptual Decision Making Jay McClelland & Juan Gao Building on: Newsome and Rorie Holmes and Feng Usher and McClelland.
Optimal Decision-Making in Humans & Animals Angela Yu March 05, 2009.
Mechanisms of Simple Perceptual Decision Making Processes
Dynamics of Reward Bias Effects in Perceptual Decision Making
Jay McClelland Stanford University
Piercing of Consciousness as a Threshold-Crossing Operation
Contribution of spatial and temporal integration in heading perception
Dynamical Models of Decision Making Optimality, human performance, and principles of neural information processing Jay McClelland Department of Psychology.
A Classical Model of Decision Making: The Drift Diffusion Model of Choice Between Two Alternatives At each time step a small sample of noisy information.
Dynamical Models of Decision Making Optimality, human performance, and principles of neural information processing Jay McClelland Department of Psychology.
Using Time-Varying Motion Stimuli to Explore Decision Dynamics
A Neural Implementation of Wald’s Sequential Probability Ratio Test
Human Reward / Stimulus/ Response Signal Experiment: Data and Analysis
Marius Usher, Phil Holmes, Juan Gao, Bill Newsome and Alan Rorie
Choice Certainty Is Informed by Both Evidence and Decision Time
Braden A. Purcell, Roozbeh Kiani  Neuron 
Perceptual Decision Making in Rodents, Monkeys, and Humans
Probabilistic Population Codes for Bayesian Decision Making
Banburismus and the Brain
C. Shawn Green, Alexandre Pouget, Daphne Bavelier  Current Biology 
Posterior parietal cortex
Decision Making as a Window on Cognition
Volume 36, Issue 5, Pages (December 2002)
Liu D. Liu, Christopher C. Pack  Neuron 
Moran Furman, Xiao-Jing Wang  Neuron 
Neural Mechanisms of Speed-Accuracy Tradeoff
Redmond G. O’Connell, Michael N. Shadlen, KongFatt Wong-Lin, Simon P
Volume 75, Issue 5, Pages (September 2012)
Perceptual learning Nisheeth 15th February 2019.
Volume 75, Issue 5, Pages (September 2012)
Presentation transcript:

From T. McMillen & P. Holmes, J. Math. Psych. 50: 30-57, MURI Center for Human and Robot Decision Dynamics, Sept 13, Phil Holmes, Jonathan Cohen + many collaborators. The diffuse dynamics of choice

A really simple decision task: “ On each trial you will be shown one of two stimuli, drawn at random. You must identify the direction (L or R) in which the majority of dots are moving. ” The experimenter can vary the coherence of movement (% moving L or R) and the delay between response and next stimulus. Correct decisions are rewarded. “ Your goal is to maximize rewards over many trials in a fixed period. ” A speed-accuracy tradeoff.. 30% coherence 5% coherence Courtesy: W. Newsome Behavioral measures: reaction time distributions, error rates. More complex decisions: buy or sell? Neural economics.

An optimal decision procedure for noisy data: the Sequential Probability Ratio Test Mathematical idealization: During the trial, we draw noisy samples from one of two distributions p L (x) or p R (x) (left or right-going dots). The SPRT works like this: set up two thresholds and keep a running tally of the ratio of likelihood ratios: When first exceeds or falls below, declare victory for R or L. Theorem: (Wald, Barnard) Among all fixed sample or sequential tests, SPRT minimizes expected number of observations n for given accuracy. [For fixed n, SPRT maximises accuracy (Neyman-Pearson lemma).] p L (x)p R (x)

The continuum limit is a DD process Take logarithms: multiplication in becomes addition. Take a continuum limit: addition becomes integration. The SPRT becomes a drift-diffusion (DD) process (a cornerstone of 20th century physics): drift rate noise strength Here is the accumulated evidence (the log likelihood ratio). When reaches either threshold, declare R or L the winner. But do humans (or monkeys, or rats) drift and diffuse? Evidence comes from three sources: behavior, direct neural recordings, and mathematical models.

Behavioral evidence: RT distributions Human reaction time data in free response mode can be fitted to the first passage threshold crossing times of a DD process. Prior or bias toward one alternative can be implemented by starting point. (Ratcliff et al., Psych Rev. 1978, 1999, 2004, … ) thresh. +Z thresh. -Z drift A

Neural evidence: firing rates Spike rates of neurons in oculomotor areas rise during stimulus presentation, monkeys signal their choice after a threshold is crossed. J. Schall, V. Stuphorn, J. Brown, Neuron, Frontal eye field (FEF) recordings. J.I Gold, M.N. Shadlen, Neuron, Lateral interparietal area (LIP) recordings. thresholds

An optimal speed-accuracy tradeoff 1 Threshold too low Too high Optimal D RT $ $ DDD D D D $$ XXX $$$ X DDD The task: maximize rewards for a succession of free response trials in a fixed period. Reward Rate: response-to-stimulus interval RT DD (% correct/average time between resps.)

An optimal speed-accuracy tradeoff 2 How fast to be? How careful? The DDM delivers an explicit solution to the speed-accuracy tradeoff in terms of just 3 parameters: normalized threshold and signal-to-noise ratio and D. So, setting the threshold we can express RT in terms of ER and calculate a unique, parameter-free Optimal Performance Curve: RT/D = F(ER)

A behavioral test Do people adopt the optimal strategy? Some do; some don ’ t. Is this because they are optimizing a different function? e.g. weighting accuracy more? Or are they trying, but unable to adjust their thresholds? OPC

Biasing choices via initial conditions 1 Prior expectations can be incorporated into DD models by biasing the initial conditions. For example, in case of biased stimuli S1 with probability and S2 with probability, we set: Unbiased DD: Biased DD: Simen et al. In preparation, Biased rewards can be treated similarly, with dependence on D also. Use similar ideas to model social interactions?

Biasing choices via initial conditions 2 More frequent stimuli (S1 > S2) or biased rewards also affect optimal thresholds, and initial conditions and thresholds can collide, predicting instant responding. Biased stimuli Biased rewards Simen et al. In preparation, Thresholds Initial conds SNR RSI Always choose more probable alternative if above critical surface.