Presentation is loading. Please wait.

Presentation is loading. Please wait.

Intertemporal choice, hyperbolic discounting, and mental time travel: A comparative and evolutionary discussion Stephen Lea Psychology, University of Exeter.

Similar presentations

Presentation on theme: "Intertemporal choice, hyperbolic discounting, and mental time travel: A comparative and evolutionary discussion Stephen Lea Psychology, University of Exeter."— Presentation transcript:

1 Intertemporal choice, hyperbolic discounting, and mental time travel: A comparative and evolutionary discussion Stephen Lea Psychology, University of Exeter UK

2 ICABEEP Summer School, 2010 Stephen Lea2 Aims of this paper To account for the key facts about human inter-temporal choice within a framework of evolutionary psychology To do that we must –Know what the key facts are, perhaps beyond the “stylised” level –Go beyond “armchair adaptationism” –Use modern data on both human and non- human preferences and cognition

3 ICABEEP Summer School, 2010 Stephen Lea3 A simple experiment (which might not work) €100 Now €200 Later

4 ICABEEP Summer School, 2010 Stephen Lea4 The stylised fact about inter-temporal choice Humans are terrible at making choices between outcomes that appear at different times in the future, for example: One marshmallow now vs. two marshmallows in 5 minutes’ time Extra sleep now vs. getting to work punctually in an hour’s time Having a washing machine now vs. being debt-free next year Luxury expenditure now vs. enough income in retirement A cigarette now vs. living 20 years longer Experimental data support this. It is by far the biggest deviation from rational choice in the whole of economic behaviour

5 ICABEEP Summer School, 2010 Stephen Lea5...aka Failure to delay gratification Lack of self control Weakness of will Yielding to temptation Impulsiveness Myopia Short time-horizon Inability to consider future consequences [excessive] Delay discounting Failure of self-regulation...etc

6 ICABEEP Summer School, 2010 Stephen Lea6 In what ways are we bad at intertemporal choice? 1.Impatience (Fisher, 1930): We choose the outcome that will arrive sooner even when it is much less valuable than the more delayed one, beyond any point that could be justified by a rational analysis 2.Inconsistency (Ainslie, 1974): We change our preferences between immediate and delayed outcomes as they (both) come closer in time – though this cannot happen if delay is discounted exponentially, as rationally it should be

7 ICABEEP Summer School, 2010 Stephen Lea7 In terms of discount rates: 1.We have to set a “subjective discount rate” at an absurdly high level (Friedman: 33%) to account for the (econometric) facts of human saving (and even higher for experimental data) 2.No single discount rate will account for all the inter-temporal choices that an individual makes even in one situation, let alone across situations

8 ICABEEP Summer School, 2010 Stephen Lea8 A positive alternative These problems had been known for decades – centuries even – but had had little influence What made them bite was the proposal of a positive alternative theory of intertemporal (and other) choice: –Positive in that it accounts for the anomalous data –Positive in that it forms part of a data-driven science

9 ICABEEP Summer School, 2010 Stephen Lea9 Hyperbolic discounting The non-exponential shape of the hyperbolic discount curve......generates a discount rate that is higher for outcomes further into the future But why pick that particular functional form?

10 ICABEEP Summer School, 2010 Stephen Lea10 The matching law Herrnstein’s matching law, formulated to describe the behaviour of pigeons working on concurrent variable interval schedules of reinforcement: R 1 /R 2 = Rf 1 /Rf 2 Here R i is a rate of responding (pecks/unit time) and Rf i is a rate of reinforcement (food deliveries/unit time) This is consistent with optimal foraging theory, which stresses the rate of prey acquisition per unit time In its original context (pigeons, conc VI VI schedules, food reward) this is an immensely robust relationship. Beyond that, it becomes more open to question Picture of Herrnstein

11 ICABEEP Summer School, 2010 Stephen Lea11 Operant choice in discrete trials In concurrent schedules, the two options are continuously available However, many researchers (including Ainslie, Rachlin, Killeen, Fantino, Davison, Mazur) wanted to apply the matching law to discrete trial choice In a single trial, there is exactly one reinforcement, after a delay D i ; and the natural analogue of reinforcement rate is then simply 1/D i – leading to the prediction that relative subjective value will be discounted hyperbolically Picture of Ainslie Picture of Rachlin

12 ICABEEP Summer School, 2010 Stephen Lea12 A1A1 Producing inconsistent choices experimentally: Ainslie & Rachlin’s “commitment” task for pigeons t0t0 t1t1 t2t2 A2A2 A2A2 1 st choice point 2 nd choice point commitment impulsiveness If t 0 is long enough, birds that would make the impulsive choice at the 2 nd choice point will nonetheless make the commitment choice at the 1 st choice point no choice

13 ICABEEP Summer School, 2010 Stephen Lea13 Commitment and hyperbolic discounting Ainslie and Rachlin’s commitment result follows directly from the discrete- trial version of the matching law Mazur shows that the following form is more precise: V i = A i /(1+KD i ) Where V i is “subjective value”, A i is reward size, D i is delay, and K is a fitting constant V i values can be used to predict choice Picture of Mazur

14 ICABEEP Summer School, 2010 Stephen Lea14 First steps in an evolutionary analysis Matching is a consequence of basic laws of operant conditioning / associative learning These processes are available in all vertebrates that have been tested, and at least some invertebrates, e.g. bees, molluscs This is what we would expect: these mechanisms are evolutionarily conservative because they make (nearly) optimal foraging possible, and almost all animals need to forage But they will only work over the time and quantity scales that are relevant to natural foraging Humans are no exception to the need to forage… but…

15 ICABEEP Summer School, 2010 Stephen Lea15 Note! We have pulled off a significant trick here We have used a mechanism (matching) that produces an explanation of a “near enough” and optimal outcome (in foraging) to explain a massive irrationality in intertemporal choice Yet foraging is inherently concerned with choices in time – choices between streams of prey availability The point is that foraging is concerned with rates of prey capture / reward But also the time scales are different…

16 ICABEEP Summer School, 2010 Stephen Lea16 The methodological leap Mazur’s formula comes from detailed work with rats and pigeons; we are interested in humans The delays concerned are a few seconds, the rewards a small fraction of a daily food requirement; whereas in human applications we may be interested in large sums of money over periods of years or even decades Because of the short time intervals, in the animal experiments, subjects can experience the choices trial after trial, for an hour or more each day, for days and weeks on end In most human experiments, we ask “which of these would you choose?” or at best “which of these will you have?” – once only (though there are human operant discrete choice experiments) In the big applications, e.g. savings/pensions choices, none of us can go back and try again having experienced the consequences of our choices

17 ICABEEP Summer School, 2010 Stephen Lea17 Examples: –birds gaining weight before migration –hibernators gaining weight in the autumn –hoarders (both scatter hoarders e.g. some corvids, some squirrels, and larder hoarders e.g. hamsters) spending much time and energy creating caches in the food-rich season These behaviours are subserved by specific physiological or instinctual mechanisms and/or specific cognitive capacities (e.g. seasonal change in size of stomach or hippocampus) Such mechanisms have evolved because they enable animals to exploit particular niches (in these examples, seasonal variation) 17 Long-term choices in other animals

18 ICABEEP Summer School, 2010 Stephen Lea18 Could we ever ask other animals to do the delayed gratification tasks humans do? Whether the contingencies are hypothetical or real, we ask people to imagine the future Lea (2001), Clayton et al (2003) point out that this is closely linked to recollecting the past – episodic memory (what happened, where it happened, when it happened) Suddendorf & Corballis (1997, 2007) call these capacities “Mental time travel” and claim that they constitute a uniquely human cognitive capacity Other names include “episodic future thought” (Szpunar, 2010) Roberts (2002): “Are animals stuck in time?” Picture of Suddendorf Picture of Roberts

19 ICABEEP Summer School, 2010 Stephen Lea19 What, when and where? “Rhymes with Orange” cartoon for 7 th January 2010

20 ICABEEP Summer School, 2010 Stephen Lea20 “Episodic-like” memory in non-humans Clayton, Dickinson and colleagues have claimed to demonstrate what-where-when memory in: scrub jays (scatter hoarders) Similar claims for other scatter hoarders e.g. magpies (Smulders & colleagues) & chickadees (Sherry & colleagues) Also male meadow voles, who have to keep track of the location and status of multiple females (Ferkin et al., 2008) However, also accumulating evidence for what-where-when memory in rats (facultative hoarders), but also pigeons and rhesus monkeys (no known hoarding propensities) Picture of Clayton Picture of a meadow vole

21 ICABEEP Summer School, 2010 Stephen Lea2126th January 2010Stephen Lea: Köln21 Evolutionary analysis, Step 2 In response to specific niche demands, some species have evolved specific mechanisms that enable adaptive long- term choices –Perhaps a capacity for mental time travel is one such mechanism, if it turns out most species really don’t have it However, the only niches we know of that make that demand are those with big annual variations in food availability (or mate availability as in voles) This isn’t obviously a property of the East African hunter/gatherer niche (though it could be) So why would our “adaptive toolbox” (Tooby & Cosmides, Gigerenzer & Todd) include anything for coping with it? Perhaps human mental time travel has a different origin? –If so it might not be very accurate for making long-term choices

22 ICABEEP Summer School, 2010 Stephen Lea22 Making the methodological leap from the opposite direction Even if (some) animals do have (something like) episodic memory, we don’t yet know how to ask them questions about anticipation - so we still can’t put animals into the situations we use with humans But can we put humans into the situations we use with animals? There have been many experiments testing the matching law with humans Results are mixed…

23 ICABEEP Summer School, 2010 Stephen Lea23 Testing the matching law in humans (samples from a large literature) Conger & Killeen (1974): “conversation” situation, reinforcement by agreement – quite good matching obtained Bradshaw et al. (e.g. 1979): concurrent schedule for points reward (later exchanged for cash), matching law generally upheld Horne & Lowe (1993): concurrent schedules for points reward, only a minority of participants match – and whether they do or not is closely correlated with their understanding of the situation as gleaned from post-experimental questionnaires Picture of Lowe

24 ICABEEP Summer School, 2010 Stephen Lea24 The human difference Humans have more than one mode of responding in matching experiments If conditions favour “mindless” performance, the laws of operant conditioning will hold But when people approach a task with a verbalising, problem-solving mindset, quite different results, with no parallel in animal studies, may be seen: “rule-governed” rather than “contingency-governed” behaviour”

25 ICABEEP Summer School, 2010 Stephen Lea25 Similar ideas… Opposition of reason to emotion and motivation dating back to classical times and continuing to William James Freud’s id vs. ego/superego Rules vs. associations in concept acquisition (e.g. Ashby) Automatic vs controlled processing (Schneider & Shiffrin, 1977 et seq.) Neurological theories (see Frank et al., 2009) Thaler’s planner vs. doer Habitual buying vs problem solving (Katona, reviewed 1975) …and many others Picture of Katona

26 ICABEEP Summer School, 2010 Stephen Lea26 Evolutionary analysis, Step 3 For whatever reason (there are many hypotheses), humans have evolved the capacity for reflective thought. This gives us a consciousness of the self (Tulving e.g. 2001: autonoetic consciousness)– and thereby makes mental time travel over long periods possible, in a new way However that is probably not its original or main function So when we are comparing an imagined future with a present reality, we don’t have anything in our adaptive toolbox that will ensure we make a wise choice…

27 ICABEEP Summer School, 2010 Stephen Lea27 Self-control and intra-psychic conflict Because we have two mechanisms available for making many kinds of decisions, they will sometimes lead in opposite directions When we use reflective thought to override the response that would be made on the basis of the laws of conditioning, we call that self-control or self-regulation (though it would better be called control by the self) This “mental tool” is used in many situations – including inter-temporal choice

28 ICABEEP Summer School, 2010 Stephen Lea28 Self-control as a limited resource Baumeister and others have shown that the capacity of self-regulation is a limited resource It is required for many situations where the “instinctual” response is not the optimal one, e.g. moral behaviour It is easily reduced by use, but also by fatigue, frustration, etc (“ego depletion”) The less we have available, the more likely we are to make maladaptively impulsive choices (among other choices our “better selves” would prefer not to make) Picture of Baumeister

29 ICABEEP Summer School, 2010 Stephen Lea29 The origins of hyperbolic discounting However, myopia in general and hyperbolic discounting in particular are not the result of self-control breaking down and operant conditioning taking over If they were, the degree of myopia would be even worse than it typically is But nothing in human evolution adapts us to make accurate value comparisons over delays of years: we are not (unlike squirrels or stilts) natural inter-temporal choosers

30 ICABEEP Summer School, 2010 Stephen Lea30 So how do we discount? To assess the value of far distant rewards, we have to rely on: –Calculation, which depends on theoretical knowledge and computational ability –Intuition, which is likely to be guided by the laws of operant conditioning, which will tell us how good an amount-delay combination “feels” This means we are likely to observe: –Substantial individual differences in both discount rates and discount functions –Substantial intra-individual inconsistencies –Little correspondence between actual and rational discounting, either in terms of discount functions or discount rates And there will always be a push towards the outcome predicted by pure conditioning – a stronger push the more ego-depleted we are

31 ICABEEP Summer School, 2010 Stephen Lea31 Evolutionary analysis, final step Humans are not natural inter-temporal choosers We get by, by using two tools that derive from our capacity for imagination: –Mental time travel –Self control Because neither evolved under the pressure to make good inter-temporal choices, they only do a moderately good job at it. And operant conditioning mechanisms are always pushing us towards myopia and hyperbolic discounting.

32 ICABEEP Summer School, 2010 Stephen Lea32 Picture credits Copyright photos (or those whose copyright status could not be determined) are available as links to their original sites. The following materials are used by permission of the photographers; they should not be re-used without similar acknowledgement Grey Squirrel: photographs by SEGL Hyperbolic discounting graphs by Wikimedia Commons user Moxfyre, used with permission Red knot photograph by Jan van de Kam from PLoS, used with permission Western scrub jay photo by Jarek Tuszynski from Wikimedia Commons, used with permission Polar bear photo by Ansgar Walk from Wikimedia Commons, used with permission Swiss Army knife photo by Jonas Bergstein from Wikimedia Commons, used with permission

33 ICABEEP Summer School, 2010 Stephen Lea33 References Ainslie, G. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, Babb, S. J., & Crystal, J. D. (2006). Episodic-like memory in the rat. Current Biology, 16, Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology, 74, Bradshaw, C. M., Szabadi, E., Bevan, P., & Ruddle, H. V. (1979). The effects of signaled reinforcement availability on concurrent performances in humans. Journal of the Experimental Analysis of Behavior, 32, Clayton, N. S., Bussey, T. J., & Dickinson, A. (2003). Can animals recall the past and plan for the future? Nature Reviews Neuroscience, 4, Clayton, N. S., & Dickinson, A. (1998). Episodic-like memory during cache recovery by scrub jays. Nature, 395. Conger, R., & Killeen, P. (1974). Use of concurrent operants in small group research - demonstration. Pacific Sociological Review, 17, Feeney, M. C., Roberts, W. A., & Sherry, D. F. (2009). Memory for what, where, and when in the black-capped chickadee (Poecile atricapillus). Animal Cognition, 12, Ferkin, M. H., Combs, A., del Barco-Trillo, J., Pierce, A. A., & Franklin, S. (2008). Meadow voles, Microtus pennsylvanicus, have the capacity to recall the "what", "where", and "when" of a single past event. Animal Cognition, 11, Fisher, I. (1930). The theory of interest. New York: Macmillan. Frank, M. J., Cohen, M. X., & Sanfey, A. G. (2009). Multiple systems in decision making: A neurocomputational perspective. Current Directions in Psychological Science, 18, Friedman, M. (1953). The methodology of positive economics. In M. Friedman (Eds.), Essays in Positive Economics, pp Chicago: University of Chicago Press. Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (Eds.), Simple heuristics that make us smart. New York: Oxford University Press. Herrnstein, R. J. (1961). Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, Horne, P. J., & Lowe, C. F. (1993). Determinants of human performance on concurrent schedules. Journal of the Experimental Analysis of Behavior, 59, Katona, G. (1975). Psychological economics. New York: Elsevier. Lea, S. E. G. (2001). Anticipation and memory as criteria for special welfare consideration. Animal Welfare, 10, S195-S208. Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, Piersma, T., Dietz, M. W., Dekinga, A., Nebel, S., van Gils, J. A., Battley, P. F. & Spaans, B. (1999). Reversible size-changes in stomachs of shorebirds: when, to what extent, and why? Acta Ornithologia, 34, Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of the Experimental Analysis of Behavior, 17, Roberts, W. A. (2002). Are animals stuck in time? Psychological Bulletin, 128, Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information-processing: 1: Detection, search, and attention. Psychological Review, 84, Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genetic Social and General Psychology Monographs, 123, Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, Szpunar, K. K (2010). Episodic future thought: An emerging concept. Perspectives on Psychological Science, 5, Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of Political Economy, 89, Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. Barkow, L. Cosmides & J. Tooby (Eds.) (Eds.), The adapted mind, pp New York: Oxford University Press. Tulving, E (2001). Episodic memory and common sense: how far apart? Proceedings of the Royal Society of London Series B, 356, Zentalll, T. R., Singer, R. A., & Stagner, J. R. (2008). Episodic-like memory: Pigeons can report location pecked when unexpectedly asked. Behavioural Processes, 79, Zinkivskay, A., Nazir, F., & Smulders, T. V. (2009). What-Where-When memory in magpies (Pica pica). Animal Cognition, 12,

Download ppt "Intertemporal choice, hyperbolic discounting, and mental time travel: A comparative and evolutionary discussion Stephen Lea Psychology, University of Exeter."

Similar presentations

Ads by Google