1. Evolving Cooperation in the N-player Prisoner's Dilemma: A Social Network Model Dept Computer Science and Software Engineering Golriz Rezaei Michael Kirley Jens Pfau ACAL09 Conference – practice talk Oct 2009

2. Motivation The Dilemma: - Contribution to the social community beneficial for everybody - Autonomous self-interested individuals rational, maximize their utility “Tragedy of the Commons” [Hardin 1968 science] - Theoretical biology / Game theory they should “Defect” - Nature / Reality they “Cooperate” Important Question in many areas : How/Why does cooperation emerge? What about Artificial Multi-agent systems? Frame work: N-player Dilemma games on social groups. Distributed Artificial Intelligence (DAI) Physics (Statistical Physics) Biology (Theoretical biology, Nature) Evolutionary Computation (IEEE Trans, CEC) Multi agent systems (AAMAS)

3. Overview What is a social network Brief overview of Prisoner’s Dilemma (N-player PD game) PD on network Proposed model Evaluation by experiments Conclusion Questions

4. Complex Networks every where Social Networks Networks Topology Function Social ties Behaviour

5. Network Basics Network graph, G(N, E), N finite set of nodes (vertices) E finite set of edges (links) G represented by N×N adjacency matrix a ij = 1 there is an edge between node i and j a ij = 0 otherwise

6. Examples of Social Net Internet-Map

7. Red, blue, or green: departments Yellow: consultants Grey: external experts www.orgnet.com Structure of an organization

8. Topological properties Degree, k i, of a node Path length, L average separation between any two nodes Clustering coefficient, C i, of a node probability that two nearest neighbours of a node are also nearest neighbours of each other.

9. Prisoner’s Dilemma ( 2 players ) (D,D) Nash Equilibrium C cooperate D Defect C cooperate b-c-c D Defect b0 2 players / agents 2 choices (C or D) Payoff joint actions

10. N-Player Prisoner’s Dilemma  Natural extension  Utility [Boyd and Richerson 1988 J. Th. Biology]  Conditions defection is preferred for individuals contribution to social welfare is beneficial for the group Conventional EG (D,D, … all D)

11. PD on Spatial structure  Local neighbourhood interaction  Clusters of cooperators  Enhance cooperation

12. Related work Santos Et. Al. [2009 Nature] Heterogeneous graphs (number and size of the game) Promotes cooperation Ohtsuki Et. Al. [2006 Nature] Correlation cost and benefit & the underlying connectivity of agents Ellis & Yao [2007 IEEE CEC] Reputation mechanism reputation scores embedded in social network

13. Contribution - Hypothesis Introducing more cognitive agents (base their decision on some function of the opponents) Incorporating “social network” into N-player PD (network evolves by cooperative behaviour) Encourage high levels of cooperation Persist for longer Analyse the state of underlying network

14. Proposed model Algorithm: Social network based N-PD model Require: Population of agents P, iteration = i max, players N 2 1: for i = 0 to i max do 2: G = 0; 3: while g = NextGame(P,G, N) do 4: G = G {g} 5: PlayGame(g) 6: AdaptLinks(g) 7: end while 8: a,b = Random Sample(P) 9: CompareUtilityAndSelect(a,b) 10: end for Decision How

15. Game Execution Two scenarios (cognitive abilities) Pure strategy (always cooperate/defect) Mixed strategy (play probabilistically) Based on a function of average links weight ( ) (β generosity) (α gradient of the function) – Agents receive corresponding payoff based on outcomes (Boyd and Richerson function) Decision

16. Link adaptation Agents play cooperatively form social links (reinforced) One agent defects breaks his links with the opponents How slow positive / fast negative

17. Snapshots of the model Self-organize social ties based on their self-interest Strategy update cultural evolution (a) Iteration 5(b) Iteration 100 (c) Iteration 1000

18. Experimental Setup population size = 1000 ε = 0.9 (game formation) b = 5 and c = 3 (payoff values benefit & cost) pure strategy scenario (50% pure C – 50% pure D) mixed strategy scenario (33.3% each) α = 1.5 and β = 0.1 (decision function) average20 independent trials up to 40000 iterations What is the equilibrium state and network topology?

19. Experiment 1 Group size vs. Strategy 2 4 5 10 15 20 2 4 5 10 15 20 Pure strategyMixed strategy Ratio of Cooperation Time (iteration)

20. Experiment 2 Emergent social networks Pure strategyMixed strategy 2 4 5 10 15 20 2 4 5 10 15 20 Cluster Coefficient Time (iteration)

21. Experiment 3 Final Degree Distribution N = 2N = 10 log(k) log(P(k))

22. Conclusion – Results validate the hypothesis Incorporating “social network” into N-player PD encourage high levels of cooperation and persist for longer – Social nets important in promoting and sustaining cooperation (specially with cognitive agent) – Endogenous network formation – Analysis of the emergent social networks high average clustering broad-scale heterogeneity – Local structure hierarchical organization of cooperation

23. Questions? Thank you