The Role of Altruistic Punishment in Promoting Cooperation Tongkui Yu, Honggang Li Department of Systems Science, Beijing Normal University

The social norm is evolving Han Fei Tzu: Legalism (law and punishment） Kung-fu-tzu: Confucianism (ethics)

Qin Dynasty (turbulent society)

汉武帝： 罢黜百家，独尊儒术(Confucianism) Han Dynasty (stable society)

Outline Motivation Model Analysis Conclusions and future work

Cooperation is very important for a society of not only human being Cooperation is very important for the smooth running of a society of not only human being but also other biological system. Outline: Motivation Model  Analysis  Conclusions and future work

Cooperation is very important for a society but also birds, animals, …… Cooperation is very important for the smooth running of a society of not only human being but also other biological system. Outline: Motivation Model  Analysis  Conclusions and future work

Why people cooperate? Natural selection (Biology) Competition Rationality (Economics) Self-interest Some specific mechanisms for the evolution of cooperation Kin selection Direct reciprocity Indirect reciprocity Network reciprocity Group selection Neither the basic assumption of natural selection in Darwin’s evolution theory nor the rational agent assumption in economics theory can lead to cooperation directly. There must be some specific mechanisms for the evolution of cooperation. A paper of Nowak in 2006 discussed five mechanisms in which cooperation can flourish, such as kin selection, direct, indirect and network reciprocity, and group selection. M. A. Nowak (2006) Science 314, 1560 -1563 Outline: Motivation Model  Analysis  Conclusions and future work

Altruistic punishment as a mechanism to promote cooperation People voluntarily incur costs to punish violations of social norms. Recently, altruistic punishment is proposed as an alternative mechanism to promote cooperation. Altruistic punishment means that people voluntarily incur costs to punish violations of social norms. That is if you are good guy, I will cooperate; while if you are bad guy, I will punish you even with a cost of myself. D. J. Quervain, et al (2004) Science 305, 1254 -1258 Outline: Motivation Model  Analysis  Conclusions and future work

The role of altruistic punishment in promoting cooperation is ambiguous Some researchers support Fehr, E. & Gachter, S. (2002) Nature 415, 137–140 Henrich, J. etc. (2006) Science 312 ,1767-1770 Some researchers against Ohtsuki, H. etc. (2009) Nature 457, 79-82. Altruistic punishment can not lead to an efficient equilibrium in most situations, and the efficient strategy is to withhold help for defectors rather than punishing them. Some researcher believe that Some other think that But at least, we are sure that altruistic punishment exist in many situation. Whether or not it helps to promote or maintain cooperation. Outline: Motivation Model  Analysis  Conclusions and future work

Altruistic punishment as a mechanism of promoting cooperation is ambiguous In real world, altruistic punishment does exist Some researcher believe that Some other think that But at least, we are sure that altruistic punishment exist in many situation. Whether or not it helps to promote or maintain cooperation. K. Sigmund, et al (2010) Nature 466, 861 - 863 J. Whitfield (2002) Nature 416, 782-784 Outline: Motivation Model  Analysis  Conclusions and future work

Controversy Real world: Theory: Question Altruistic punishment exists Ohtsuki (2009) : Altruistic punishment provides no efficiency Question What does altruistic punishment exist for ? Outline: Motivation Model  Analysis  Conclusions and future work

Our argument Ohtsuki’s analysis only focuses on the equilibrium (i.e. Cooperative Evolutionary Stable State, CESS) Although in CESS, punishment is not the most efficient. But from an initial state far away from equilibrium, altruistic punishment may play an different role in the process of approaching cooperative evolutionary stable state (CESS). Outline: Motivation Model  Analysis  Conclusions and future work

Our work Extends Ohtsuki’s model to a framework with 3-level of evolution Reputation is updated instantaneously once an agent takes an action. Strategy is updated by agents according to their personal payoff in a short period of time. Social norm evolves very slowly according to the global benefit of all social members in a considerably long period of time. Studies the dynamics of the strategy frequency under social norms with different punishment attitudes Non-punishment social norm Punishment-alternative social norm Punishment-provoking social norm Outline: Motivation Model  Analysis  Conclusions and future work

Our results Altruistic punishment really works in promoting cooperation in at least two ways: (1) enlarges the attraction basin of CESS Non-punishment social norm Punishment-alternative social norm Punishment-provoking social norm Outline: Motivation Model  Analysis  Conclusions and future work

Our results Altruistic punishment really works in promoting cooperation in at least two ways: (2) increases the rate of convergence to CESS Blue: Punishment-alternative norm Red: Punishment-provoking norm Outline: Motivation Model  Analysis  Conclusions and future work

The Role of Altruistic Punishment in Promoting Cooperation Motivation Model Analysis Conclusions and future work Outline: Motivation Model  Analysis  Conclusions and future work

Model A society with infinitely large population. Each individual is endowed with a binary reputation: good (G) or bad (B). Everyone in the society agrees on the reputation of an individual. Outline: Motivation Model  Analysis  Conclusions and future work

Model At each small time interval, two players are sampled randomly. One as donor and the other as recipient. Donor has 3 choices: cooperation (C), defection (D), and punishment (P). Recipient does nothing. If C, donor spends a cost c (c=2) to give a benefit b (b=3) to recipient; if D, no gain no loss; If P, donor spends a cost α (α=1) to give a loss β (β=4) to recipient. D R Recipient Donor C (b,-c) (0,0) D P (-β,-α) Outline: Motivation Model  Analysis  Conclusions and future work

Strategy (or action rule) B C D P X J Each player has a strategy The strategy determines the donor’s action X (C, D, or P) according to the reputation J (G or B) of recipient. Since donors have 3 choice to each of the two states of reputation, the total number of all possible strategies is 2 to 3 equals nine. In this paper, we only consider the normal 4 of 9 strategies. The abnormal ones such as DC, i.e. defect to good guy and cooperation to bad guy is not considered. Outline: Motivation Model  Analysis  Conclusions and future work

Model After each interaction, the reputation of the donor is updated according to the ‘social norm’ D R D Outline: Motivation Model  Analysis  Conclusions and future work

Social norm Assign a new reputation to the donor According to not only the action (X) of the donor, but also the reputation (J) of the recipient. G B C D P X J To assign a reputation to each of the 3 actions to each of the 2 reputations, we have 6 entries in one social norm. Each entry has two possible assignment, we have 64 possible social norms. Outline: Motivation Model  Analysis  Conclusions and future work

Cooperative Evolutionary Stable State (CESS) Cooperative : Most agents cooperate Evolutionary Stable : Given all other agents take some strategy, the best choice for an agent is to take that strategy. G B C D P CD(CESS): all agents take CD strategy Outline: Motivation Model  Analysis  Conclusions and future work

Cooperative Evolutionary Stable State (CESS) Whether is there a CESS, what is the CESS depends on social norm G B C D P DD (all agents take DD strategy) is the only stable state. No CESS. Outline: Motivation Model  Analysis  Conclusions and future work

Ohtsuki’s work(1) Finds social norms that can foster cooperative evolutionary stable state (CESS) from 64 candidates G B C D P * G B C D P * Ohtsuki’s work finds social norms that can foster cooperative evolutionary stable state from 64 candidates. In these social norms where star means both G and B are OK, CD can be a CESS. That is, if all agents take the strategy CD, i.e. cooperation to good guy and defection to bad ones, the best choice for an agent is such a situation is to take the strategy CD. And in social norm like these, CP can be a CESS. They also give the conditions for all possible CESS. Notice that in all conditions, DD i.e. defect for both good and bad recipient is always a stable state, but not CESS, because there is few cooperation. In the first condition, only DD is stable. In the second condition, a social norm like this can foster CESS CD. In the third condition, these social norm can foster a CESS CP. And in the forth condition, both CD and CP are possible. *: both G and B are OK. Outline: Motivation Model  Analysis  Conclusions and future work

Ohtsuki’s work(2) Find the most efficient social norm for different parameter settings Indirect reciprocity provides only a narrow margin of efficiency for altruistic punishment. Outline: Motivation Model  Analysis  Conclusions and future work

Extend Ohtsuki’s work Not just focus on the equilibrium But try to model the dynamics of strategy frequency explicitly Study the dynamics of the strategy frequency under social norms with different punishment attitudes. Outline: Motivation Model  Analysis  Conclusions and future work

Social norms Non-punishment norm Punishment-alternative norm We consider three typical social norms. The first is non-punishment social norm. That is donors can only cooperate or defect, they has no choice of punishment. The 2nd is Punishment-alternative norm, that is punishment is a alternative choice for donors, but it is not so encouraged because both D and P to a bad reputation are assigned a good reputation, and the cost of D is zero. The 3rd is the punishment-provoking norm. In this norm, punishment is provoked because we you meet a bad guy, you can only cooperate or punish to get a good reputation. We will compare the dynamics of strategy frequency in these 3 social norm to understand the role of punishment in promoting cooperation. Non-punishment norm Punishment-alternative norm Punishment-provoking norm Outline: Motivation Model  Analysis  Conclusions and future work

Extend Ohtsuki’s work To explicitly model the dynamics of strategy frequency Key question When and how does an agent update his strategy ? Can not use the income of one interaction to measure the fitness of a strategy, because Agents can be donor or recipient in one interaction. Agents’ reputation changes instantly. To explicitly model the dynamics of strategy frequency, the key question is When and how an agent update his strategy. Outline: Motivation Model  Analysis  Conclusions and future work

Stable reputation frequency For a fixed strategy frequency, there is a stable reputation frequency for each strategy . For a fixed strategy frequency, there is a stable reputation frequency for each strategy. For example in the non-punishment social norm. x1 is the proportion of agents taking CC strategy, and x2 is the proportion of agents taking CD strategy, and x3 equaling 1-x1-x2 is the proportion of agents taking DD strategy. G1 is the proportions of agents with good reputation in all agents taking CC strategy. And g2 and g3 are the proportions of good agents with good reputation in all agents taking CD and DD strategy respectively. If x1, x2 and x3 are fixed, g1, g2 and g3 converge to stable state quickly. This figure gives an illustration of such a stable frequency. From different initial g1 g2 and g3 Non-punishment norm Outline: Motivation Model  Analysis  Conclusions and future work

When and how does an agent update his strategy After they can actually perceive the payoff of different strategies, i.e. after reputation frequecy converges to the stable state. According to the expected payoff of his strategy in the stable reputation frequecy Anaylitical tractable  Replicator dynamics Outline: Motivation Model  Analysis  Conclusions and future work

Model Strategy Frequency Dynamics Non-punishment norm The differential equation for strategy frequency dynamics in the non-punishment norm is like this. Non-punishment norm Outline: Motivation Model  Analysis  Conclusions and future work

Punishment-alternative norm Model Strategy Frequency Dynamics Punishment-alternative norm Outline: Motivation Model  Analysis  Conclusions and future work

Punishment-provoking norm Model Strategy Frequency Dynamics Punishment-provoking norm Outline: Motivation Model  Analysis  Conclusions and future work

The Role of Altruistic Punishment in Promoting Cooperation Motivation Model Result Analysis Conclusions and future work Outline: Motivation Model  Analysis  Conclusions and future work

Phase portrait of three social norms Attraction basin ratio of CESS in entire space 15% Firstly, we plot the Phase portrait of three social norms with typical parameter setting. For non-punishment social norm, the blue part is the attraction basin of CESS (CD). The ratio of CESS attraction basin in entire space is 15%. For punishment-alternative social norm, the part over the surface is the attraction basin of CESS (CD). The ratio of CESS attraction basin is 60%. For punishment-provoking social norm, the part over the surface is the attraction basin of CESS (CP). The ratio of CESS attraction basin is 81%. 81% 60% Outline: Motivation Model  Analysis  Conclusions and future work

Typical trajectories under ‘GGBGBG’ and ‘GGBBBG’ norm Outline: Motivation Model  Analysis  Conclusions and future work

The effect of parameters to CESS attraction ratio α β alpha and beta no influence to GGBG norm GGBG norm attraction ratio is always the smallest. Alpha small, punishment provoking norm has the largest attraction ratio. Alpha is the cost of punishment. Too large the cost of punishment, the attraction ratio decreases. Beta is the punishment strength, it is large, attraction ratio increases. That of punishment provoking norm is always the largest no matter how beta changes. B larger, attraction ratio increases. punishment provoking norm has the largest. C larger, attraction ratio decreases. punishment provoking norm has the largest most situation. C very small. Little cost for large benefit, punishment is not necessary. c b

Converge rate from the same initial point in different social norm Blue: Punishment-alternative norm Red: Punishment-provoking norm Outline: Motivation Model  Analysis  Conclusions and future work

The Role of Altruistic Punishment in Promoting Cooperation Motivation Model Analysis Conclusions and future work Outline: Motivation Model  Analysis  Conclusions and future work

Conclusion Altruistic punishment really works in promoting cooperation in at least two ways: (1) enlarges the attraction basin of CESS (2) increases the rate of convergence to CESS Outline: Motivation Model  Analysis  Conclusions and future work

Han Fei Tzu: Legalism (law and punishment） Kung-fu-tzu: Confucianism (ethics)

Future works Formal modeling of converge rate Group selection for social norm evolution by simulation The effect of abnormal strategies (such as DC strategy) Experiment to test the converge rate and attraction basin ratio in different social norm Outline: Motivation Model  Analysis  Conclusions and future work

Acknowledgement Prof. Shu-heng Chen AI-ECON Research Center, National Chengchi University

Thank you!