1. The Role of Altruistic Punishment in Promoting Cooperation
Tongkui Yu, Honggang Li
Department of Systems Science,
Beijing Normal University

2. The social norm is evolving
Han Fei Tzu: Legalism
(law and punishment）
Kung-fu-tzu: Confucianism (ethics)

3. Qin Dynasty (turbulent society)

4. 汉武帝：
罢黜百家，独尊儒术(Confucianism)
Han Dynasty (stable society)

5. Outline
Motivation
Model
Analysis
Conclusions and future work

6. Cooperation is very important for a society
of not only human being
Cooperation is very important for the smooth running of a society of not only human being but also other biological system.
Outline: Motivation Model  Analysis  Conclusions and future work

7. Cooperation is very important for a society
but also birds, animals, ……
Cooperation is very important for the smooth running of a society of not only human being but also other biological system.
Outline: Motivation Model  Analysis  Conclusions and future work

8. Why people cooperate? Natural selection (Biology) Competition
Rationality (Economics) Self-interest
Some specific mechanisms for the evolution of cooperation
Kin selection
Direct reciprocity
Indirect reciprocity
Network reciprocity
Group selection
Neither the basic assumption of natural selection in Darwin’s evolution theory nor the rational agent assumption in economics theory can lead to cooperation directly. There must be some specific mechanisms for the evolution of cooperation. A paper of Nowak in 2006 discussed five mechanisms in which cooperation can flourish, such as kin selection, direct, indirect and network reciprocity, and group selection.
M. A. Nowak (2006) Science 314,
Outline: Motivation Model  Analysis  Conclusions and future work

9. Altruistic punishment as a mechanism to promote cooperation
People voluntarily incur costs to punish violations of social norms.
Recently, altruistic punishment is proposed as an alternative mechanism to promote cooperation. Altruistic punishment means that people voluntarily incur costs to punish violations of social norms. That is if you are good guy, I will cooperate; while if you are bad guy, I will punish you even with a cost of myself.
D. J. Quervain, et al (2004) Science 305,
Outline: Motivation Model  Analysis  Conclusions and future work

10. The role of altruistic punishment in promoting cooperation is ambiguous
Some researchers support
Fehr, E. & Gachter, S. (2002) Nature 415, 137–140
Henrich, J. etc. (2006) Science 312 ,
Some researchers against
Ohtsuki, H. etc. (2009) Nature 457,
Altruistic punishment can not lead to an efficient equilibrium in most situations, and the efficient strategy is to withhold help for defectors rather than punishing them.
Some researcher believe that
Some other think that
But at least, we are sure that altruistic punishment exist in many situation. Whether or not it helps to promote or maintain cooperation.
Outline: Motivation Model  Analysis  Conclusions and future work

11. Altruistic punishment as a mechanism of promoting cooperation is ambiguous
In real world, altruistic punishment does exist
Some researcher believe that
Some other think that
But at least, we are sure that altruistic punishment exist in many situation. Whether or not it helps to promote or maintain cooperation.
K. Sigmund, et al (2010)
Nature 466,
J. Whitfield (2002) Nature 416,
Outline: Motivation Model  Analysis  Conclusions and future work

12. Controversy Real world: Theory: Question Altruistic punishment exists
Ohtsuki (2009) : Altruistic punishment provides no efficiency
Question
What does altruistic punishment exist for ?
Outline: Motivation Model  Analysis  Conclusions and future work

13. Our argument
Ohtsuki’s analysis only focuses on the equilibrium (i.e. Cooperative Evolutionary Stable State, CESS)
Although in CESS, punishment is not the most efficient.
But from an initial state far away from equilibrium, altruistic punishment may play an different role in the process of approaching cooperative evolutionary stable state (CESS).
Outline: Motivation Model  Analysis  Conclusions and future work

14. Our work
Extends Ohtsuki’s model to a framework with 3-level of evolution
Reputation is updated instantaneously once an agent takes an action.
Strategy is updated by agents according to their personal payoff in a short period of time.
Social norm evolves very slowly according to the global benefit of all social members in a considerably long period of time.
Studies the dynamics of the strategy frequency under social norms with different punishment attitudes
Non-punishment social norm
Punishment-alternative social norm
Punishment-provoking social norm
Outline: Motivation Model  Analysis  Conclusions and future work

15. Our results
Altruistic punishment really works in promoting cooperation in at least two ways:
(1) enlarges the attraction basin of CESS
Non-punishment social norm
Punishment-alternative social norm
Punishment-provoking social norm
Outline: Motivation Model  Analysis  Conclusions and future work

16. Our results
Altruistic punishment really works in promoting cooperation in at least two ways:
(2) increases the rate of convergence to CESS
Blue: Punishment-alternative norm
Red: Punishment-provoking norm
Outline: Motivation Model  Analysis  Conclusions and future work

17. The Role of Altruistic Punishment in Promoting Cooperation
Motivation
Model
Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work

18. Model A society with infinitely large population.
Each individual is endowed with a binary reputation: good (G) or bad (B).
Everyone in the society agrees on the reputation of an individual.
Outline: Motivation Model  Analysis  Conclusions and future work

19. Model
At each small time interval, two players are sampled randomly. One as donor and the other as recipient.
Donor has 3 choices: cooperation (C), defection (D), and punishment (P). Recipient does nothing.
If C, donor spends a cost c (c=2) to give a benefit b (b=3) to recipient; if D, no gain no loss; If P, donor spends a cost α (α=1) to give a loss β (β=4) to recipient. D R
Recipient
Donor C
(b,-c)
(0,0) D P
(-β,-α)
Outline: Motivation Model  Analysis  Conclusions and future work

20. Strategy (or action rule)B C D P X J
Each player has a strategy
The strategy determines the donor’s action X (C, D, or P) according to the reputation J (G or B) of recipient.
Since donors have 3 choice to each of the two states of reputation, the total number of all possible strategies is 2 to 3 equals nine. In this paper, we only consider the normal 4 of 9 strategies. The abnormal ones such as DC, i.e. defect to good guy and cooperation to bad guy is not considered.
Outline: Motivation Model  Analysis  Conclusions and future work

21. Model
After each interaction, the reputation of the donor is updated according to the ‘social norm’ D R D
Outline: Motivation Model  Analysis  Conclusions and future work

22. Social norm Assign a new reputation to the donor
According to not only the action (X) of the donor, but also the reputation (J) of the recipient. G B C D P X J
To assign a reputation to each of the 3 actions to each of the 2 reputations, we have 6 entries in one social norm. Each entry has two possible assignment, we have 64 possible social norms.
Outline: Motivation Model  Analysis  Conclusions and future work

23. Cooperative Evolutionary Stable State (CESS)
Cooperative : Most agents cooperate
Evolutionary Stable : Given all other agents take some strategy, the best choice for an agent is to take that strategy. G B C D P
CD(CESS): all agents take CD strategy
Outline: Motivation Model  Analysis  Conclusions and future work

24. Cooperative Evolutionary Stable State (CESS)
Whether is there a CESS, what is the CESS
depends on social norm G B C D P
DD (all agents take DD strategy) is the only stable state.
No CESS.
Outline: Motivation Model  Analysis  Conclusions and future work

25. Ohtsuki’s work(1)
Finds social norms that can foster cooperative evolutionary stable state (CESS) from 64 candidates G B C D P * G B C D P *
Ohtsuki’s work finds social norms that can foster cooperative evolutionary stable state from 64 candidates. In these social norms where star means both G and B are OK, CD can be a CESS. That is, if all agents take the strategy CD, i.e. cooperation to good guy and defection to bad ones, the best choice for an agent is such a situation is to take the strategy CD. And in social norm like these, CP can be a CESS. They also give the conditions for all possible CESS. Notice that in all conditions, DD i.e. defect for both good and bad recipient is always a stable state, but not CESS, because there is few cooperation. In the first condition, only DD is stable. In the second condition, a social norm like this can foster CESS CD. In the third condition, these social norm can foster a CESS CP. And in the forth condition, both CD and CP are possible.
*: both G and B are OK.
Outline: Motivation Model  Analysis  Conclusions and future work

26. Ohtsuki’s work(2)
Find the most efficient social norm for different parameter settings
Indirect reciprocity provides only a narrow margin of efficiency for altruistic punishment.
Outline: Motivation Model  Analysis  Conclusions and future work

27. Extend Ohtsuki’s work Not just focus on the equilibrium
But try to model the dynamics of strategy frequency explicitly
Study the dynamics of the strategy frequency under social norms with different punishment attitudes.
Outline: Motivation Model  Analysis  Conclusions and future work

28. Social norms Non-punishment norm Punishment-alternative norm
We consider three typical social norms. The first is non-punishment social norm. That is donors can only cooperate or defect, they has no choice of punishment.
The 2nd is Punishment-alternative norm, that is punishment is a alternative choice for donors, but it is not so encouraged because both D and P to a bad reputation are assigned a good reputation, and the cost of D is zero. The 3rd is the punishment-provoking norm. In this norm, punishment is provoked because we you meet a bad guy, you can only cooperate or punish to get a good reputation. We will compare the dynamics of strategy frequency in these 3 social norm to understand the role of punishment in promoting cooperation.
Non-punishment norm
Punishment-alternative norm
Punishment-provoking norm
Outline: Motivation Model  Analysis  Conclusions and future work

29. Extend Ohtsuki’s work
To explicitly model the dynamics of strategy frequency
Key question
When and how does an agent update his strategy ?
Can not use the income of one interaction to measure the fitness of a strategy, because
Agents can be donor or recipient in one interaction.
Agents’ reputation changes instantly.
To explicitly model the dynamics of strategy frequency, the key question is When and how an agent update his strategy.
Outline: Motivation Model  Analysis  Conclusions and future work

30. Stable reputation frequency
For a fixed strategy frequency, there is a stable reputation frequency for each strategy .
For a fixed strategy frequency, there is a stable reputation frequency for each strategy. For example in the non-punishment social norm. x1 is the proportion of agents taking CC strategy, and x2 is the proportion of agents taking CD strategy, and x3 equaling 1-x1-x2 is the proportion of agents taking DD strategy. G1 is the proportions of agents with good reputation in all agents taking CC strategy. And g2 and g3 are the proportions of good agents with good reputation in all agents taking CD and DD strategy respectively.
If x1, x2 and x3 are fixed, g1, g2 and g3 converge to stable state quickly. This figure gives an illustration of such a stable frequency. From different initial g1 g2 and g3
Non-punishment norm
Outline: Motivation Model  Analysis  Conclusions and future work

31. When and how does an agent update his strategy
After they can actually perceive the payoff of different strategies, i.e. after reputation frequecy converges to the stable state.
According to the expected payoff of his strategy in the stable reputation frequecy
Anaylitical tractable  Replicator dynamics
Outline: Motivation Model  Analysis  Conclusions and future work

32. Model Strategy Frequency Dynamics Non-punishment norm
The differential equation for strategy frequency dynamics in the non-punishment norm is like this.
Non-punishment norm
Outline: Motivation Model  Analysis  Conclusions and future work

33. Punishment-alternative norm
Model
Strategy Frequency Dynamics
Punishment-alternative norm
Outline: Motivation Model  Analysis  Conclusions and future work

34. Punishment-provoking norm
Model
Strategy Frequency Dynamics
Punishment-provoking norm
Outline: Motivation Model  Analysis  Conclusions and future work

35. The Role of Altruistic Punishment in Promoting Cooperation
Motivation
Model
Result Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work

36. Phase portrait of three social norms
Attraction basin ratio of CESS in entire space
15%
Firstly, we plot the Phase portrait of three social norms with typical parameter setting. For non-punishment social norm, the blue part is the attraction basin of CESS (CD). The ratio of CESS attraction basin in entire space is 15%. For punishment-alternative social norm, the part over the surface is the attraction basin of CESS (CD). The ratio of CESS attraction basin is 60%. For punishment-provoking social norm, the part over the surface is the attraction basin of CESS (CP). The ratio of CESS attraction basin is 81%.
81%
60%
Outline: Motivation Model  Analysis  Conclusions and future work

37. Typical trajectories under ‘GGBGBG’ and ‘GGBBBG’ norm
Outline: Motivation Model  Analysis  Conclusions and future work

38. The effect of parameters to CESS attraction ratioα β
alpha and beta no influence to GGBG norm
GGBG norm attraction ratio is always the smallest.
Alpha small, punishment provoking norm has the largest attraction ratio. Alpha is the cost of punishment. Too large the cost of punishment, the attraction ratio decreases.
Beta is the punishment strength, it is large, attraction ratio increases. That of punishment provoking norm is always the largest no matter how beta changes.
B larger, attraction ratio increases. punishment provoking norm has the largest.
C larger, attraction ratio decreases. punishment provoking norm has the largest most situation. C very small. Little cost for large benefit, punishment is not necessary. c b

39. Converge rate from the same initial point in different social norm
Blue: Punishment-alternative norm
Red: Punishment-provoking norm
Outline: Motivation Model  Analysis  Conclusions and future work

40. The Role of Altruistic Punishment in Promoting Cooperation
Motivation
Model
Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work

41. Conclusion
Altruistic punishment really works in promoting cooperation in at least two ways:
(1) enlarges the attraction basin of CESS
(2) increases the rate of convergence to CESS
Outline: Motivation Model  Analysis  Conclusions and future work

42. Han Fei Tzu: Legalism
(law and punishment）
Kung-fu-tzu: Confucianism (ethics)

43. Future works Formal modeling of converge rate
Group selection for social norm evolution by simulation
The effect of abnormal strategies (such as DC strategy)
Experiment to test the converge rate and attraction basin ratio in different social norm
Outline: Motivation Model  Analysis  Conclusions and future work

44. Acknowledgement Prof. Shu-heng Chen
AI-ECON Research Center, National Chengchi University

45. Thank you!