Dr. Bruce J. West Chief Scientist Mathematical & Information Science Directorate Army Research Office UNCLASSIFIED / FOR OFFICIAL USE ONLY Decision Making and Stochastic Delay at Workshop on Social Computing, Behavior Modeling and Prediction 1 April 2008

Discounted Utility Model & intertemporal choice Anomalies from discounted utility theory –irrationality –hyperbolic discounting Objective and subjective time – entropy and the direction of time – time as a stochastic variable Individuality and paternalism – some experiments – fit of theory to data Conclusions Outline of talk Decision Making and Delay

Discounted Utility Model (DUM) Discount factor δ compresses many mechanisms mortality, uncertainty, time compression,… Accepted as both normative (how things should be) and descriptive (how things are)…..but was initially arbitrary Samuelson (1937). Exponential form implies time consistency (rationality) Decision Making and Delay

Anomalies from DUM Time inconsistency – empirical discount factor is not constant over time across type of intertemporal choices Delay effect (hyperbolic discounting) Interval effect (non-stationarity) Sign effect (gains vs. loses) Magnitude effect (small vs. large) Direction effect Sequence effects (ordered set vs. single) Decision Making and Delay

Model comparison Exponential delay model –monotonic decrease in value with objective time –constant rate results in time consistency –rationality Hyperbolic delay model –decreasing rate results in time inconsistency –irrationality (preference reversal) hyperbolic exponential hyperbolic exponential Decision Making and Delay

Objective vs. subjective time Hyperbolic models – Objective time clockwork universe entropy and the direction of time – Subjective time unidirectional probability and statistics Motivate decision-making using ensemble distributions – subjective time – stochastic delays Decision Making and Delay

Delay and uncertainty Decision-making models of intertemporal choice can be extended to incorporate probabilistic choice where p is the probability of reward at time t and F is an unspecified function. discrete continuous No reward before delay time t Delay-time probability density Decision Making and Delay

Deterministic discount rate is replaced with a conditional probability per unit time The ratio of the delay time distribution function to the survival probability density, integrates to The utility function in terms of subjective time is therefore Stochastic rate Decision Making and Delay

Example rate Rate of reward production suggested by hyperbolic distribution Probability of no reward before time t is so that the utility function is inverse power law T measures response time and α measures irrationality Decision Making and Delay

Experimental data Students (20) asked to make decisions about hypothetical money to be received immediately or at a later time, concerning the subjects themselves or another person not known to them. Takahashi, Physica A (2007). self other T=31 α=0.28 T=1.85 α=0.11 Decision Making and Delay

Implications from experiments The response times could describe paternalistic policy making government officials, where irrationality is enhanced. Irrationality is nowhere more significant than in the military where choices may determine whether others live or die. Decision Making and Delay

Nonlinear dynamic equation solved on the interval (0,1). define a delay-time distribution density assume a uniform distribution of initial conditions to obtain delay-time distribution density is non-Poisson, renewal and non-ergodic

Measured discount rates Higher discount rates compared with controls* ( smaller T and α in stochastic intertemporal model) – smoking – excessive alcohol consumption – illicit drug use (cocaine, crack-cocaine and heroin) – pathological gambling – age – cognitive ability (negative correlation with intellectual achievement) Consistent with neuroeconomic hypothesis that prefrontal cortex is essential for patient (forward looking) decision making. * Chabris, Laibson & Schuldt, The New Palgrave Dictionary of Economics (2007). Decision Making and Delay

Brain Activity Sanfey, Loewenstein, McClure and Cohen, TRENDS in Cognitive Science (2006). β network”: midbrain dopamine network; reward processing (ventral striatum V.Str. and medial prefrontal cortex mPFC.) δ network: cognition; dorsolateral prefrontal cortex (diPFC) and right posterior parletal cortex (R.Par.) Decision Making and Delay

More brain activity Two discounting slopes < one year > one year Different parts of the brain light up under fMRI short-term long-term Wittmann & Paulus (2007) Decision Making and Delay

Conclusions and Speculations decision-making is not always rational irrationality in intertemporal choice models take a hyperbolic form inverse power laws or hyperbolic utility functions can be generated by stochastic delay times different parts of the brain control decisions associated with long and short delay times the complexity of the brain produces the subjective nature of biological time