1. Trieste, October 19, 2006 Time Scales in Evolutionary Dynamics Angel Sánchez Grupo Interdisciplinar de Sistemas Complejos (GISC) Departamento de Matemáticas – Universidad Carlos III de Madrid Instituto de Biocomputación y Física de Sistemas Complejos (BIFI) Universidad de Zaragoza with Carlos P. Roca and José A. Cuesta

2. Time Scales in Evolution 2 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Cooperation: the basis of human societies Occurs between genetically unrelated individuals Anomaly in the animal world:

3. Time Scales in Evolution 3 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Cooperation: the basis of human societies Shows high division of labor Anomaly in the animal world:

4. Time Scales in Evolution 4 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Cooperation: the basis of human societies Valid for large scale organizations… Anomaly in the animal world: …as well as hunter-gatherer groups

5. Time Scales in Evolution 5 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Cooperation: the basis of human societies Some animals form complex societies… …but their individuals are genetically related

6. Time Scales in Evolution 6 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Altruism: key to cooperation Altruism: fitness-reducing act that benefits others Pure altruism is ruled out by natural selection acting on individuals á la Darwin

7. Time Scales in Evolution 7 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 He who was ready to sacrifice his life (…), rather than betray his comrades, would often leave no offspring to inherit his noble nature… Therefore, it seems scarcely possible (…) that the number of men gifted with such virtues (…) would be increased by natural selection, that is, by the survival of the fittest. Charles Darwin (Descent of Man, 1871) How did altruism arise?

8. Time Scales in Evolution 8 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Altruism is an evolutionary puzzle

9. Time Scales in Evolution 9 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 A man who was not impelled by any deep, instinctive feeling, to sacrifice his life for the good of others, yet was roused to such actions by a sense of glory, would by his example excite the same wish for glory in other men, and would strengthen by exercise the noble feeling of admiration. He might thus do far more good to his tribe than by begetting offsprings with a tendency to inherit his own high character. Charles Darwin (Descent of Man, 1871) Group selection? Cultural evolution?

10. Time Scales in Evolution 10 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Answers to the puzzle…  Kin cooperation (Hamilton, 1964) common to animals and humans alike  Reciprocal altruism in repeated interactions (Trivers, 1973; Axelrod & Hamilton, 1981) primates, specially humans  Indirect reciprocity (reputation gain) (Nowak & Sigmund, 1998) primates, specially humans None true altruism: individual benefits in the long run

11. Time Scales in Evolution 11 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 … but only partial!  Strong reciprocity (Gintis, 2000; Fehr, Fischbacher & Gächter, 2002) typically human (primates?)  altruistic rewarding: predisposition to reward others for cooperative behavior  altruistic punishment: propensity to impose sanctions on non-cooperators Strong reciprocators bear the cost of altruistic acts even if they gain no benefits Hammerstein (ed.), Genetic and cultural evolution of cooperation (Dahlem Workshop Report 90, MIT, 2003)

12. Time Scales in Evolution 12 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 One of the 25 problems for the XXI century: E. Pennisi, Science 309, 93 (2005) “Others with a more mathematical bent are applying evolutionary game theory, a modeling approach developed for economics, to quantify cooperation and predict behavioral outcomes under different circumstances.”

13. Time Scales in Evolution 13 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Evolution There are populations of reproducing individuals Reproduction includes mutation Some individuals reproduce faster than other (fitness). This results in selection Game theory Formal way to analyze interactions between agents who behave strategically (mathematics of decision making in conflict situations) Usual to assume players are “rational” Widely applied to the study of economics, warfare, politics, animal behaviour, sociology, business, ecology and evolutionary biology

14. Time Scales in Evolution 14 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Everyone starts with a random strategy Everyone in population plays game against everyone else Population is infinite Payoffs are added up Total payoff determines the number of offspring: Selection Offspring inherit approximately the strategy of their parents: Mutation John Maynard Smith 1972 (J.B.S. Haldane, R. A. Fisher, W. Hamilton, G. Price) Evolutionary Game Theory Successful strategies spread by natural selection Payoff = fitness

15. Time Scales in Evolution 15 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 replicator-mutator Price equation Quasispecies equation Lotka-Volterra equation Adaptive dynamics Game dynamical equation replicator Price equation Replicator-mutator equation Price equation Equations for evolutionary dynamics

16. Time Scales in Evolution 16 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Case study on strong reciprocity and altruistic behavior: Ultimatum Games, altruism and individual selection

17. Time Scales in Evolution 17 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 The Ultimatum Game (Güth, Schmittberger & Schwarze, 1982) experimenter proposer responder M euros M-u u OK u M-u NO 0 0

18. Time Scales in Evolution 18 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Experimental results Extraordinary amount of data Camerer, Behavioral Game Theory (Princeton University Press, 2003) Henrich et al. (eds.), Foundations of Human Sociality : Economic Experiments and Ethnographic Evidence from Fifteen Small-Scale Societies (Oxford University Press, 2004) “At this point, we should declare a moratorium on creating ultimatum game data and shift attention towards new games and new theories.”

19. Time Scales in Evolution 19 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 What would you offer?

20. Time Scales in Evolution 20 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Experimental results Proposers offer substantial amounts (50% is a typical modal offer) Responders reject offers below 25% with high probability Universal behavior throughout the world Large degree of variability of offers among societies (26 - 58%) Rational responder’s optimal strategy: accept anything Rational proposer’s optimal strategy: offer minimum

21. Time Scales in Evolution 21 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Model...... N players player i t i, o i : thresholds (minimum share player i accepts / offers) f i : fitness (accumulated capital) M monetary units (M=100) A.S. & J. A. Cuesta, J. Theor. Biol. 235, 233 (2005)

22. Time Scales in Evolution 22 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 o p t r < Game event...... N players proposer responder opfpopfp trfrtrfr o p t r ≥ +M-o p +o p

23. Time Scales in Evolution 23 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 t, o min f min t’, o’ max f max new player Reproduction event (after s games)...... N players minimum fitness maximum fitness t, o max f max mutation: t’, o’ max = t, o max ± 1 (prob.=1/3)

24. Time Scales in Evolution 24 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 N =1000, 10 9 games, s = 10 5, t i = o i =1 initial condition accept offer Slow evolution (large s)

25. Time Scales in Evolution 25 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 N =1000, 10 6 games, s =1, uniform initial condition accept offer Fast evolution (small s)

26. Time Scales in Evolution 26 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Adaptive dynamics (“mean-field”) results Results for small s (fast selection) differ qualitatively Implications in behavioral economics and evolutionary ideas on human behavior !

27. Time Scales in Evolution 27 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Selection/reproduction interplay in simpler settings: Equilibrium selection in 2x2 games

28. Time Scales in Evolution 28 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Select one, proportional to fitness Substitute a randomly chosen individual Moran Process Game event Choose s pairs of agents to play the game between reproduction events Reset fitness after reproduction 2x2 game P. A. P. Moran, The statistical processes of evolutionary theory (Clarendon, 1962) C. P. Roca, J. A. Cuesta, A. Sánchez, Phys. Rev. Lett. 97, 158701 (2006)

29. Time Scales in Evolution 29 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Fixation probability Probability to reach state N when starting from state i =1 Absorbing states 1-x 1 x1x1

30. Time Scales in Evolution 30 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Fixation probability Probability to reach state N when starting from state n

31. Time Scales in Evolution 31 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Fixation probability Probability to reach state N when starting from state n

32. Time Scales in Evolution 32 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Fixation probability Probability to reach state N when starting from state n Number of games s enters through transition probabilities

33. Time Scales in Evolution 33 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Fixation probability Probability to reach state N when starting from state n Fitness: possible game sequences times corresponding payoffs per population

34. Time Scales in Evolution 34 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 1: Harmony game Payoff matrix: Unique Nash equilibrium in pure strategies: (C,C) (C,C) is the only reasonable behavior anyway

35. Time Scales in Evolution 35 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 1: Harmony game s infinite (round-robin, “mean-field”)

36. Time Scales in Evolution 36 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 1: Harmony game s = 1 (reproduction following every game)

37. Time Scales in Evolution 37 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 1: Harmony game Consequences Round-robin: cooperators are selected One game only: defectors are selected! Result holds for any population size In general: for any s, numerical evaluation of exact expressions

38. Time Scales in Evolution 38 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 1: Harmony game Numerical evaluation of exact expressions

39. Time Scales in Evolution 39 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 2: Stag-hunt game Payoff matrix: Two Nash equilibria in pure strategies: (C,C), (D,D) Equilibrium selection depends on initial condition

40. Time Scales in Evolution 40 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 2: Stag-hunt game Numerical evaluation of exact expressions

41. Time Scales in Evolution 41 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 3: Snowdrift game Payoff matrix: One mixed equilibrium Replicator dynamics goes always to mixed equilibrium Moran dynamics does not allow for mixed equilibria

42. Time Scales in Evolution 42 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 3: Snowdrift game Numerical evaluation of exact expressions

43. Time Scales in Evolution 43 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 3: Snowdrift game Numerical evaluation of exact expressions s = 5 s = 100

44. Time Scales in Evolution 44 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example 4: Prisoner’s dilemma Payoff matrix: Paradigm of the emergence of cooperation problem Unique Nash equilibrium in pure strategies: (C,C)

45. Time Scales in Evolution 45 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Example : Prisoner’s dilemma Numerical evaluation of exact expressions

46. Time Scales in Evolution 46 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Results are robust Increasing system size does not changes basins of attrractions, only sharpens the transitions Small s is like an effective small population, because inviduals that do not play do not get fitness Introduce background of fitness: add f b to all payoffs

47. Time Scales in Evolution 47 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 Background of fitness: Stag-hunt game Numerical evaluation of exact expressions f b = 0.1 f b = 1

48. Time Scales in Evolution 48 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 In general, evolutionary game theory studies a limit situation: s infinite! (every player plays every other one before selection) Number of games per player may be finite, even Poisson distributed Fluctuations may keep players with smaller ‘mean-field’ fitness alive Changes to equilibrium selection are non trivial and crucial Conclusions New perspective on evolutionary game theory: more general dynamics, dictated by the specific application (change focus from equilibrium selection problems)

49. Time Scales in Evolution 49 http://www.gisc.es anxo@math.uc3m.es Trieste, October 19, 2006 C. P. Roca, J. A. Cuesta, A. Sánchez, arXiv:q-bio/0512045 (submitted to European Physical Journal Special Topics) A. Sánchez & J. A. Cuesta, J. Theor. Biol. 235, 233 (2005) A. Sánchez, J. A. Cuesta & C. P. Roca, in “Modeling Cooperative Behavior in the Social Sciences”, eds. P. Garrido, J. Marro & M. A. Muñoz, 142–148. AIP Proceedings Series (2005). C. P. Roca, J. A. Cuesta, A. Sánchez, Phys. Rev. Lett. 97, 158701 (2006) Time Scales in Evolutionary Dynamics