1. Acqui Terme, 2 September 2010 Cooperation from a Game Theory perspective Marco Dardi University of Florence marco.dardi@unifi.it

2. Acqui T. 2 September 20102 In common language cooperation means mutual assistance in order to get benefits that would be unavailable to agents acting non-cooperatively. GT can be applied to analyzing problems that arise in the effort to reap this cooperative surplus. In the language of GT the terms cooperative/non- cooperative have also a technical meaning connected with the formal specification of solution concepts. This lecture investigates the relationship between the two levels at which GT deals with cooperation: as an object problem and as a formal definition.

3. Acqui T. 2 September 20103 1. GT, conflict, cooperation In a nutshell: GT elaborates methods for describing the structure of interactive situations, focusing on the choices available to individual agents, on their knowledge and preferences (theory of representations) prescribing ways of behaving in interactive situations that comply with notions of intelligent pursuit of each agent’s own interests (theory of solutions) First, representation. The normal or strategic form of so- called non-cooperative games (on these, later on) is the most convenient representation for the purposes of this lecture. It constitutes a sort of compromise between the extensive form representation, that can be reduced to normal form at the cost of losing relevant detail, and the coalitional form representation of so-called cooperative games, that can be derived from the normal form, again with some loss of detail. The normal form representation consists of:  a set of agents or players (in number n  1)  for each agent, a set of strategies  Strategic space  a set of outcomes  for each agent, preferences on the outcomes  represented by von Neumann utility numbers

4. Acqui T. 2 September 20104 According to the normal form representation, a game form G is a mapping G:  strategic space    outcomes  A game g is a game form + a specification of the preferences of the agents involved. Formally, with a numerical representation of preferences, a game is a composite mapping g = U (n) G, with U (n) the n-vector of the players’ utility functions. Hence, g:  strategic space    n Interpretation: G describes the “physical” rules that apply in a situation (distribution of the agents’ powers over the outcomes). Each G generates a class of games, one for each specification of the preferences of the agents involved (think of a plan of organization and all the firms organized according to it; of a legal code, and all the specific situations regulated by it; etc) Warning: the main behavioral feature implicit in GT representations is not self-interest or egoism, but the instrumentality of the choices made in playing a game. Players are not interested in the actions they or the other players choose, but in the outcomes brought about by these actions. The motivations lying behind their preferences on the outcomes needn’t be made explicit. Altruism is as good a motivation as any other.

5. Acqui T. 2 September 20105 Just looking at the images of g in  n we get an idea of what conflict and cooperation look like in a GT framework. Case 1: the vector inequality  (strong Pareto preference) does not apply in the range of g  all the outcomes are Pareto optima  g describes a pure conflict situation Case 2: The range of g is completely ordered by   Pareto and individual preferences are in agreement  conflict is non-existent Case 3: The range of g is ordered by  but only partially  some (more than one) outcomes are Pareto optima, some are Pareto-dominated  conflict and common interest co-exist In 1 there are no gains from cooperation (e.g., zero-sum games). In 2, the gains can be reaped unproblematically because individual and common interests accord with each other (the only problem may be coordination). The only case of interest for GT turns out to be 3, where conflicts of interest may prevent the players from picking the potential gains from cooperation. Note: the conflict/cooperation mix does not depend on the game form but on “who” the players are

6. Acqui T. 2 September 20106 2. Cooperation problems: basic patterns To get an idea, take the simplest possible case: two agents, A and B, each with two strategies, c = follow a cooperative line of conduct, d = don’t follow such a line (whatever this may mean). No player can make his/her choice conditional upon the other player’s choice (no information leaks). Game form (outcomes in greek letters) cdcd c d A B     Two games ( = A’s preference; = B’s preference) cdcd c d A B     cdcd c d A B     “PRISONER”“HAWK-DOVE”

7. Acqui T. 2 September 20107 In both games we can observe: gains from cooperation, symmetrical incentives to defect, each player prefers the other to cooperate. PRISONER unconditionally prefers not to cooperate; H-D prefers to cooperate if the other doesn’t. Consequently, gains from cooperation are greater in H-D than PRISONER as revealed by Pareto preference cdcd c d A B     cdcd c d A B     PRISONERHAWK-DOVE In order to avoid the non-cooperative, Pareto-dominated outcome  some sort of agreement is needed. Any agreement in PRISONER is liable to defection. So is agreement  in H-D, while agreements  and , although secure against defection, may be refused on grounds of justice. The analysis suggests remedies that in all cases, barring the possibility of changing the agents’ preferences, require modifications of the game form…

8. Acqui T. 2 September 20108 In PRISONER: expand the game so as to add a post-play stage in which players have a possibility of sanctioning the agreement by means of penalties. The expansion may consist of a number of repetitions of the game, provided no repetition is known with certainty to be the last one. In H-D: introduce correlated randomization (NB: not the independent randomization known as “mixed strategies”) of the outcomes. In some cases this will require changes of the game form through the introduction of an umpire or an external information system (with suitable utility numbers the best fair agreement requires prob(  ) = 0 and  to be drawn with the same probability as  and  ). These examples provide a clue to one of the most thriving lines of research in applied GT during the 1980s and 1990s: how to design a game form such that the cooperative gains latent in a situation do not go unexploited (implementation theory (IT) or “mechanism design”). IT provides a new basis for the theory of contracts, industrial organization, imperfect markets and other microeconomic applications. The object of IT is: given the agents’ preferences, re-design the situation as a game such that sticking to the agreement to cooperate in reaping the existing Pareto gains is the only intelligent line of conduct for all the players in the game. A preliminary step is of course clarifying what an intelligent line of conduct in a game is. This leads to the particular theory of solutions on which IT depends…

9. 3. Stability In the theory of solutions a solution concept is a rule that selects a subset of strategy profiles out of the  strategic space  of the game according to some criterion of rationality,  solution    strategic space  with  solution  being defined by some common property of the strategy profiles included. The main property used in GT solution concepts is that of stability. A strategy profile is stable whenever for each player the following statement holds: “if nobody has any reason to refuse to do his/her part in this profile, I have no reason either”. A cooperative agreement has a chance of being effective only if it prescribes strategies which make up a profile that belongs to a stable  solution . Thus, cooperative agreements should be stable solutions of an appropriately designed game. Stability may mean various things depending on the way “reasons to refuse” in the above statement are specified. Here for the first time we have to deal with the technical distinction in GT between so-called non-cooperative and cooperative solutions. 9Acqui T. 2 September 2010

10. 10 From now on: cooperative/non-cooperative in the technical sense of GT solutions will be marked off with a *. A stable solution is non-cooperative* if “reasons to refuse” are referred exclusively to individual players. An individual player has reason to refuse to do his/her part in a prescribed strategy profile if, in the hypothesis that the others do theirs, he/she has the power of bringing about a preferred outcome. If no player has such reasons, then the prescribed strategy profile is a Nash non-cooperative* equilibrium (NE). Hence, a stable non-cooperative*  solution  coincides with the set of Nash non-cooperative* equilibria. Note: by basing cooperative agreements on NE, as is usually done in IT, we have a theory of cooperation in which the stability of agreements relies entirely on a non- cooperative* solution concept. Far from being a paradox, this is the essence of the so-called “Nash program” (Nash 1951) for reducing all cooperative solutions to non- cooperative* equilibrium analysis. But it is to be noted that Von Neumann and Morgenstern refused to take Nash’s cooperative*/non-cooperative* partition into consideration. The statement “ the general, typical game – in particular all significant problems of a social exchange economy – cannot be treated without these devices [of cooperation] ” remained unchanged in the third edition of their work (1953, p. 44), after they had taken cognizance of Nash’s papers on non-cooperative* games.

11. Acqui T. 2 September 201011 Stable cooperative* solutions differ from NE in that “reasons to refuse” may be referred not only to individuals but also to groups of individuals acting cooperatively (coalitions). Refusing is not necessarily an individual affair, individuals may refuse by forming a coalition in order to get a better outcome. A coalition has reason to refuse to do its part in a prescribed strategy profile if, in the hypothesis that the other players do theirs, it has the power of bringing about an outcome which is preferred by all its members. That a strategy profile belongs to NE is a necessary but not a sufficient condition for it to be stable with respect to coalitions. Generally, cooperation in refusing restricts the domain of stability. If all possible coalitions are considered to be equally feasible, the relevant cooperative* solution concept is (from Edgeworth) the CORE. A strategy profile belongs to the CORE if and only if no individual or coalition has reason to refuse to do its part in it on condition that all the others do theirs. Obviously, CORE  NE. Basing a cooperative agreement upon a cooperative* solution concept turns out to be more difficult than relying on a non-cooperative* solution such as NE. In particular cases the CORE may even be empty (for example, PRISONER has NE =  (d,d)  but no CORE).

12. Acqui T. 2 September 201012 However, not all the conceivable coalitions are generally equally feasible. More sophisticated concepts of cooperative* stability refer to the stability of the coalitions themselves. A strategy profile s, which is liable to be refused by a coalition, may be considered to be stable all the same if the agreement within the potentially refusing coalition can in turn be challenged by some other coalition. Members of the former, knowing that the latter could thwart their plans, could be dissuaded from forming it. Thus, although not in the CORE, s may remain unchallenged. These considerations open the way to a variety of sophisticated cooperative* solution concepts (BARGAINING SET, KERNEL, SHAPLEY VALUE etc.). The specific definitions depend on the way that the notion of challenging coalitions and counter-coalitions is modelled. In general, all these concepts are more permissive than the CORE. Von Neumann & Morgenstern proposed a cooperative* solution concept that, while of a sophisticated kind, lies on a completely different line from those we have considered so far. Their concept of “stable set” (SSET) was defined not on the basis of a common property of the strategy profiles that belong to it, but on the basis of a structural property of the set itself. They insisted on stability in social theories being “a property of the system as a whole and not of the single imputations [here, read “strategy profiles”] of which it is composed” (1953, p. 36).

13. Acqui T. 2 September 201013 A set of strategy profiles is a SSET if and only if (i)For each profile included in it: if a coalition has reason to refuse it, this must be in favour of a profile excluded; (ii)For each profile excluded from it: there is at least a coalition that has reason to refuse it in favour of a profile included SSET may be empty (as in PRISONER). In the same game there may be more than one SSET (as in the purely conflictual game known as “matching pennies”, with empty NE and CORE ). If more than one SSET exist, these have no intersection. And, of course, in general CORE  SSET, while there is no general relationship between SSET and NE. Von Neumann & Morgenstern interpreted SSET as a formalization of the notion of an “established order of society” or “accepted standard of behavior”. It describes a variety of modes of behavior, none of which is able to unsettle the others. Some of them may be unsettled by some non-conforming modes of behavior, but all of the latter are unsettled by one or another of the accepted ones. Lastly, the same game or social situation may express more than one such “order” or standard. However, for all its evocative power, this notion has had little application in economics and in social theory in general.

14. 4. Final remarks Cooperative* solution concepts have been little used in economic applications. Perhaps the very variety of concepts available, with the ensuing feeling of ad hoc constructions, has been an obstacle to generalized adoption. Thus, cooperation on a non-cooperative* basis, in the sense explained above, constitutes the unifying methodological framework of great part of contemporary microeconomics (Moulin, 1995: “Cooperation in the economic tradition is mutual assistance between egoists”). The “Nash program” has prevailed over Von Neumann & Morgenstern’s more “social” approach. But note that Moulin’ s reference to “egoism”, as remarked above (slide 4), is anyway wide of the mark, since acting on the basis of individual preferences has no necessarily egoistic implications. Moreover, we should not be induced to view the non- cooperative* approach to cooperation as an expression of an inevitably atomistic social philosophy. All the arguments that try to justify NE as the only solution concept consistent with individual rationality resort to some kind of “communality of thought” that, as Schelling (1960) and Lewis (1969) have pointed out, presupposes that some social convention is in force. It cannot be an exclusively individual affair. 14Acqui T. 2 September 2010

15. 15 Recall the premise of the conditional statement underlying individual stability (slide 9): “if the others have no reason to refuse to do their part in this profile, I have no reason either”. Doing my part is rational if the premise is true. But why should I believe it to be true? A moment’s reflection shows that the basis of this belief is the belief that it is shared by everybody, that it is believed to be shared by everybody, and so on ad infinitum: briefly, it must be, in a specially strong sense, a “common” belief. Common beliefs presuppose conscience that on some matters there is something like communality of thought, thought that does not need to be communicated. This would seem to be an unlikely phenomenon in a rigorously atomistic society because it implies that individuals do not think independently of each other on all matters, and therefore some “accepted standards” of thought (in Von Neumann & Morgenstern’s language) must be well- established. Individualism itself must be an expression of such a standard. Thus, non-cooperative* foundations of cooperation (as in IT) must in turn be founded on sociological premises lying at a deeper level than individual rationality. Nash program is at best half a program for GT-based social research; the other half would require relating solution concepts to types of social culture.

16. Acqui T. 2 September 201016 5. Selected references For the basics of GT, IT, coalitions: according to my taste and teaching experience, the best (although by no means elementary) introduction is Osborne & Rubinstein, A course in GT (1994) MIT Press, chapters 1-5, 10, 13-14. For the game-theoretic outlook on cooperation in economics: see Moulin, Cooperative microeconomics (1995) Prentice Hall. For stability, cooperative/non-cooperative games, and the Nash program: see von Neumann & Morgenstern, Theory of games and economic behavior (3 rd edition, 1953) Princeton UP, chapts. I.4, V, XI; Nash, “Noncooperative games”, Annals of Mathematics (1951); Myerson, “Nash equilibrium and the history of economic theory”, JEL (1999). For the foundations of non-cooperative NE: Schelling, The strategy of conflict (1960) Harvard UP; Lewis, Convention: A philosophical study (1969) Blackwell; Bacharach, “A theory of rational decision in games”, Erkenntnis (1987). For all matters related to GT: see the relevant chapters in the Handbook of GT with economic applications, Aumann & Hart editors, 3 vols., 1992-1994-2002: North-Holland.