1. Towards Realistic Models for Evolution of Cooperation
LIK MUI

2. … about procedure … Briefly go over the paper
Clarify major points
Describe simulations (not in paper)

3. RoadMap 
Introduction
Cooperation Models
Simulations
Conclusion

4. Evolution of Cooperation
Animals cooperate
Two questions:
How does cooperation as a strategy becomes stable evolutionarily?
How does cooperation arise in the first place?

5. Darwinian Natural Selection
“Survival of the fittest”
If evolution is all about individual survival, how can cooperation be explained?
Fittest what?

6. Fittest what ? Individual Group Gene Organization
Rational agency theory (Kreps, 1990)
Group
Group selection theory (Wilson, 1980)
Gene
Selfish gene hypothesis (Dawkins, 1979)
Organization
Classic organizational theory (Simon, 1969)

7. RoadMap Introduction Cooperation Models  Simulations Conclusion
Group Selection
Kinship Theory
Direct Reciprocity
Indirect Reciprocity
Social Learning
Simulations
Conclusion 

8. Group Selection Intuition: we ban cannibalism but not carnivorousness
Population/species: basic unit of natural selection
Problem: explain war, family feud, competition, etc.

9. Kinship Theory I Intuition: nepotism Hamilton’s Rule:
Individuals show less aggression and more cooperation towards closer kin if rule is satisfied
Basis for most work on kinship theory
Wright’s Coefficient of Related: r
Self: r=1
Siblings: r=0.5
Grandparent-grandchild: r=0.25

10. Kinship Theory II Cannot explain: Competition in viscuous population
Symbioses
Dynamics of cooperation

11. Direct Reciprocity Intuition: being nice to others who are nice
“Reciprocal Altruism”
Trivers (1971)
Tit-for-tat and PD tournament
Axelrod and Hamilton (1981)
Cannot explain:
We cooperate not only with people who cooperate with us

12. Indirect Reciprocity Intuition: respect one who is famous
Social-biological justifications
Biology: generalized altruism (Trivers, 1971, 1985)
Sociobiology: Alexandar (1986)
Sociology: Ostrom (1998)
3 types of indirect reciprocity:
Looped
Observer-based
Image-based

13. Indirect Reciprocity: Looped
Looped Indirect Reciprocity
Boyd and Richerson (1989)

14. Indirect Reciprocity: Observers
Observer-based Reciprocity
Pollock and Dugatkin (1992)

15. Indirect Reciprocity: Image
Image (reputation) based Reciprocity
Nowak and Sigmund (1998, 2000)

16. Social Learning Intuition: imitate those who are successful
Cultural transmission
Boyd and Richerson (1982)
Docility
Simon (1990, 1991)

17. Critiques of Existing Models
Many theories each explaining one or a few aspects of cooperation
Unrealism of model assumptions

18. Unrealism for Existing Models
asexual, non-overlapping generations
simultaneous play for every interaction
c.f., Abell and Reyniers, 2000
dyadic interactions
mostly predetermined behavior
c.f., May, 1987 (lack of modeling stochasticity)
discrete actions (cooperate or defect)
social structure and cooperation
c.f., Simon, 1991; Cohen, et al., 2001
extend social learning
c.f., Simon, 1990

19. RoadMap Introduction Cooperation Models Simulations  Conclusion
Nowak and Sigmund Game
Prisoner’s Dilemma Game
Simon’s Docility Hypothesis
Conclusion 

20. Nowak and Sigmund Game Payoff Matrix Image Adjustment Interact
B = 1.0
Image Adjustment
A = 1
Interact
Not interact
Donor -C
Recipient B
Interact
Not interact
Donor A -A
Recipient

21. Using Global Image: 1 Run

22. Using Global Image: 100 Runs

23. Dynamics using Global Reputation

24. Using 10 Observers/Interactions

25. Action = { cooperate, defect }
Evolutionary PD Game
Repeated Prisoners’ Dilemma Game
Agent Actions:
Action = { cooperate, defect }
Payoff Matrix: C D
3/3
0/5
5/0
1/1

26. PD Game Agent Strategies
All defecting (AllD)
Tit-for-tat (TFT)
Reputational Tit-for-tat (RTFT): using various notions of reputation

27. Base Case: PD Game

28. Simple Groups: social structures
Group structure affects members
Interactions, observations, and knowledge
Persistent structure
Groups actions
Observed indirectly through member's actions
Hierarchy
family, school, town, country
Structure
member identity,
isolation (different islands)
integration (shared habitat)
groups could evolve

29. Group Membership Member agents Group Structure is a Tree
Have public group identity
Directly associated with one environment
Group Structure is a Tree
Least common ancestral node (LCAN)
Events occur with respect to a shared environment
Events occur in shared environment
simple analogy to human society
strangers often attempt to find shared reference point in initial conversation

30. Shared Environment Example
Agents Group
A1,A2 G1
A3,A4 G2
A5,A2 G1
A1,A3 G0
A5,A3 G0
People belong to different families
People interact as members of shared community
town, religious group, school

31. PD Game with Group Reputation (varying encounters per generation EPG)

32. PD Game with Group Reputation (100 EPG; varying Inter-group interaction probability)

33. Groups/Organizations: bounded rationality explanation
Docility
Cooperation (altruism) as an explanation for the formation of groups/organizations
Why individuals “identify” with a group?
boundedly rational individuals
increase their survival fitness
(Simon, 1969, 1990, 1991)

34. PD Game with Docility (50 cooperators and 50 defectors; 100 EPG; 1
PD Game with Docility (50 cooperators and 50 defectors; 100 EPG; 1.0 IP)

35. Conclusion
Reviewed 5 major approaches to study evolution of cooperation
Provided 2 main critiques for existing models
Constructed model extensions addressing the critiques

36. Implications for Computer Science
Artificial intelligence
Benevolent agents are not good enough
(c.f., multi-agents systems)
Learning theory can be used to study evolution of cooperation
Systems
Improve system design by understanding the dynamics of agents
Accountability substrate needed for distributed systems

37. Future Plan Extend the simple group social structure
Overlapping generations
Sexual reproduction
Extend social learning using realistic/robust learning model

38. Modeling Diploid Organisms

39. Modeling Diploid Organisms

40. Modeling Diploid Organisms
Parental Chromosomes
One of 2 Child Chromosomes

41. Simulation Demo C D R/R S/T T/S P/P Recall PD payoff matrix:
PD strategies: viewed as a probability vectors
Strategy: <PI, PT, PR, PP, PS>
TFT: < 1, 1, 1, 0, 0 >
AllD: < 0, 0, 0, 0, 0 >
AllC: < 1, 1, 1, 1, 1 >
STFT: < 0, 1, 1, 0, 0 >

42. Simulation: a search problem
Search Optimal PD Strategy
Search space: I, T, R, P, S probabilities