1. Heterogeneous Payoffs and Social Diversity in the Spatial Prisoner’s Dilemma game Dept Computer Science and Software Engineering Golriz Rezaei Dr. Michael Kirley SEAL08 Conference 8 Dec 2008

2. Evolution of cooperation Open ended question in many areas Evolutionary Computation (IEEE Trans, CEC, GECCO) Autonomous agents and multi agent systems (AAMAS) Distributed Artificial Intelligence (DAI) Physics (Statistical Physics) Biology (Theoretical biology, Nature) Prisoner’s Dilemma (PD game)  Different individual conditions (Heterogeneity) have impact In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game. Good abstract Game theoretic approach Mathematical model Applied in many areas (biology, economics, and sociology)

3. Today’s Agenda Brief overview of Prisoner’s Dilemma game and different variations The challenge and related works Proposed model Evaluation by experiments Conclusion Questions

4. Prisoner’s Dilemma C cooperate D Defect C cooperate R=3 T=5 S=0 D Defect S=0 T=5 P=1 2 players / agents 2 choices (C or D) Actual values  order Order change  game change (D,D)  Nash Equilibrium But i) T > R > P > S ii) 2R >= (T + S) Iteration  reciprocal interaction Spatial  local neighbourhood

5. Spatial Prisoner’s Dilemma Limited to local neighbourhood interaction only Accumulates received payoffs from games  fitness At the end of each round  selection process imitation of the most successful neighbour (MSN) Clusters of cooperators  outweigh losses against defectors 123 45 678

6. The Challenge Typically  “Universal fixed payoff matrix” Hypothesis  Introducing “social diversity” alters trajectory of the population.

7. Related work Few studies  investigated the impact of varying the magnitude of the payoff matrix values 1. Tomochi and Kono [ Physical Review E 2002 ]: Payoff matrix evolved based on the ratio of defectors (considered R and P only) - Universal payoff matrix 2. Perc and Szolnoki [ Physical Review E 2008 ]: Random noise added to the individual payoff matrix at the beginning of the game - Fixed matrix till the end 3. Fort [ Physica A 2007 ]: The payoff matrix was correlated with a spatial and temporal zones (considered only T) - The Prisoner’s Dilemma inequality was relaxed.

8. Proposed model Idea  Associated payoffs evolve based on individual experience. Each agent Dynamic payoffs  each agent has its own version of payoff matrix and it gets updated at each time step based on the level of the agent’s experience Age  increases at each time step α i (t+1) = α i (t) + 1 Life-span  expected life time (λ i ) randomly drawn from a uniform distribution αi(t) == λi  die and replaced by a new random agent

9. Proposed model Update  Where is the payoff values for agent i at time t is the default payoff matrix values T, R, P, S is the magnitude of the rescaled values is the age of agent i at time t is the expected life time of agent i is limiting factor and characterises the uncertainty related to the environment 1) 2)

10. Three scenarios 1.Standard PD  universal fixed Payoffs no Age 2.Homogeneous model  universal fixed Payoffs Age 3.Heterogeneous model  individual Payoffs Age What is the equilibrium state?

11. Experimental Setup Implemented in Netlogo4.0 [ Wilensky 1999 ] Underlying framework  Standard Spatial Iterated Prisoner’s Dilemma. Agents mapped on 2-D regular lattice (32*32 torus) Population initialized  20% cooperators Each trial  1000 iterations All configurations  30 times Statistical results are reported

12. Experiment 1  sensitivity to the base payoff values Two different base level payoff values T, R, P, S and K = 0.2 a) Big  5, 3, 1, 0 b) Small  1, 1, 0, 0

13. Experiment 2  sensitivity to the magnitude of K base level payoff values T, R, P, S  5, 3, 1, 0 K was changed systematically K represents environmental constraint on social diversity

14. Snapshots Evolving population for homogeneous and heterogeneous model K = 0.1 and initial cooperation 20% Varying size clusters of cooperators (black) Homogeneous  Heterogeneous 

15. Conclusion Results  heterogeneous social diversity, promotes cooperation. Differences to previous work  each agent is equipped with their own evolving payoff matrix. The evolving payoff matrix  agents’ age or experience level. More realistic approach  real world scenarios. Future work  extend the model to distributed multiagent systems (P2P, MANET)

16. Questions? Thank you

17. References H. Fort, On evolutionary spatial heterogeneous games, Physica A (2007). M. Perc and A. Szolnoki, Social diversity and promotion of cooperation in spatial prisoner's dilemma game, Physical Review E 77 (2008). M. Tomochi and M. Kono, Spatial prisoner's dilemma games with dynamic payoff matrices, Physical Review E 65 (2002), no. 026112. Wilensky, U.: NetLogo is a cross-platform multi-agent programmable modeling environment. In: Modeling Nature’s Emergent Patterns with Multi-agent Languages. Proceedings of EuroLogo 2002 (2002),http://ccl.northwestern.edu/netlogo/

18. Experiment 3  sensitivity to the life span (λ) base level payoff values T, R, P, S  5, 3, 1, 0 K = 0.2 λ from different range

19. Experiment 4  sensitivity to the replacement strategy base level payoff values T, R, P, S  5, 3, 1, 0 K = 0.2 Replacement with random generated agent and defector agent

20. Related work Few studies have examined the impact of varying the magnitude of the payoff matrix values in PD Tomochi and Kono: Payoff matrix was designed to evolve based on the ratio of defectors (cooperators) to the whole population. (considered R and P only) Universal payoff matrix applicable to all agents at time t. The level of cooperation within population was directly related to the payoff matrix values

21. Related work … Perc and Szolnoki: Random noise drawn from alternative statistical distributions was added to the payoff matrix at the beginning of the game. (fixed matrix till the end) They concluded that this correlated “social diversity mechanism” promoted higher-levels of cooperation in the spatial game examined. It was suggested that variable social status might play a crucial role in the evolution of cooperation.

22. Related work … Fort: The payoff matrix was correlated with a spatial and temporal zones. (considered only T) It was possible that the payoffs for an agent and their opponent were not equal – reminiscent of what happens in general in real life. The results reported suggested that the effect of asymmetries in the interactions between agents, which takes into account the effect of asymmetries in the costs and benefits on the evolution of cooperation, had a direct impact on the proportion of agents cooperating in the population. The Prisoner’s Dilemma inequality was relaxed, and when the payoff matrix values changed, the game oscillated between the Prisoner’s Dilemma game and Chicken game or the game becomes Stag Hunt game.

23. What is the idea? Ex./ You and your friend, colleague Ex./ 2 countries  punishment system for the same crime. Different individual conditions (Heterogeneity) have impact on the behaviour of two people/agents and may alter their interaction and their cooperation. In this paper we investigate this idea on a version of the Spatial Prisoner’s Dilemma (SPD) game. Why? Good abstract  many real world scenarios. Famous game theoretic approach  capture agents interaction Mathematical model  study the evolution of cooperation Applied in many areas  biology, economics, and sociology