Algorithmic Mechanism Design: an Introduction (with an eye to some basic network optimization problems) Guido Proietti Dipartimento di Ingegneria e Scienze dell'Informazione e Matematica University of L'Aquila

Roadmap 1. (Tuesday January 14, 2014, 11-13): Reviews of basics of game theory. The implementation problem. An example: second-price auction. Mechanism design. Strategy-proof mechanisms. Utilitarian problems. VCG mechanisms. Clarke payments. 2. (Thursday January 16, 2014, 16-18): Computational aspects of mechanisms. VCG-mechanism for the private-edge Shortest-Path problem. Trivial implementation and efficient O(m + n log n) time implementation. 3. (Friday January 17, 2014, 11-13): VCG-mechanism for the private- edge Minimum Spanning Tree problem. Trivial implementation and efficient O(m α(m,n)) time implementation. 4. (Tuesday January 21, 2014, 11-13): Non-utilitarian, one-parameter problems. One-parameter mechanisms. One-parameter mechanism for the private-edge Shortest-Paths Tree problem. Trivial implementation and efficient O(m + n log n) time implementation. 5. (Thursday January 23, 2014, 16-18): Approximate one-parameter mechanisms: the single-minded combinatorial auction problem.

Suggested readings Algorithmic Mechanism Design, Noam Nisan and Amir Ronen, STOC’99, ACM Press, Algorithmic Game Theory, Edited by Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani, Cambridge University Press, Blog by Noam Nisan

Algorithmic game theory It aims to answer the following fundamental question: How the computational aspects of a game (in standard theoretical computer science terminology, a set of self-interested agents of a strategic distributed system) should be addressed? Theory of Algorithms Game Theory Algorithmic Game Theory +=

Review of basics of Game Theory (informally) A game consists of: A set of N players (or agents) A specification of the information available to each player A set of rules of encounter: Who should act when, and what are the possible actions (strategies) A specification of payoffs for each possible outcome (combination of strategies) of the game Game Theory studies the interaction among the players, and attempts to predict the final outcomes (or solutions) of the game by taking into account the individual behavior of the players

Solution concept How do we establish that an outcome is a solution? Among the possible outcomes of a game, those enjoying the following property play a fundamental role: Equilibrium solution: strategy combination in which players are not willing to change their state. This is quite informal: w hen a player does not want to change her state? In the Homo Economicus model, this makes sense when she has selected a strategy that maximizes her individual payoff, knowing that other players are also doing the same.

A famous warm-up game: the Prisoner’s Dilemma Prisoner I Prisoner II Don’t Implicate Implicate Don’t Implicate 1, 16, 0 Implicate0, 65, 5 Strategy Set Strategy Set Payoffs Technically speaking: Non-cooperative, symmetric, non- zero sum, simultaneous, perfect information, one-shot, 2- player game

Prisoner I’s decision Prisoner I’s decision: If II chooses Don’t Implicate then it is best to Implicate If II chooses Implicate then it is best to Implicate It is best to Implicate for I, regardless of what II does: Dominant Strategy Prisoner I Prisoner II Don’t ImplicateImplicate Don’t Implicate1, 16, 0 Implicate0, 65, 5

Prisoner II’s decision Prisoner II’s decision: If I chooses Don’t Implicate then it is best to Implicate If I chooses Implicate then it is best to Implicate It is best to Implicate for II, regardless of what I does: Dominant Strategy Prisoner I Prisoner II Don’t ImplicateImplicate Don’t Implicate1, 16, 0 Implicate0, 65, 5

Hence… It is best for both to implicate regardless of what the other one does Implicate is a Dominant Strategy for both (Implicate, Implicate) becomes the Dominant Strategy Equilibrium Note: If they might collude, then it’s beneficial for both to Not Implicate, but it’s not an equilibrium as both have incentive to deviate Prisoner I Prisoner II Don’t Implicate Implicate Don’t Implicate 1, 16, 0 Implicate0, 65, 5

Dominant Strategy Equilibrium Dominant Strategy Equilibrium: this is a strategy combination s * = (s 1 *, s 2 *, …, s N * ), such that s i * is a dominant strategy for each i, namely, for any possible alternative strategy profile s= (s 1, s 2, …, s i, …, s N ): p i (s 1, s 2, …, s i *, …, s N ) ≥ p i (s 1, s 2, …, s i, …, s N ) Dominant Strategy is a best response to any strategy of other players: in other words, a player needs not to know payoffs of other players in order to decide her action! If a game has a DSE, then players will immediately converge to it Of course, not all games (only very few in the practice!) have a dominant strategy equilibrium

A more relaxed solution concept: Nash Equilibrium [1951] Nash Equilibrium: this is a strategy combination s * = (s 1 *, s 2 *, …, s N * ) such that for each i, s i * is a best response to (s 1 *, …,s i-1 *,s i+1 *,…, s N * ), namely, for any possible alternative strategy s i of player i p i (s 1 *, s 2 *, …, s i *, …, s N * ) ≥ p i (s 1 *, s 2 *, …, s i, …, s N * )

Nash Equilibrium In a NE no agent unilaterally deviates from her strategy given others’ strategies as fixed (i.e., each agent has to take into consideration the strategies of the other agents) If the game is played repeatedly and players converge to a solution, then it has to be a NE, but if a game has one or more NE, players need not to converge to it Dominant Strategy Equilibrium  Nash Equilibrium (but the converse is not true)

Nash Equilibrium: The Battle of the Sexes (coordination game) (Stadium, Stadium) is a NE: Best responses to each other (Cinema, Cinema) is a NE: Best responses to each other  but they are not Dominant Strategy Equilibria … are we really sure they will eventually go out together???? Man Woman StadiumCinema Stadium2, 10, 0 Cinema0, 01, 2

The implementation problem (informally) Imagine now that you are a planner who develops criteria for social welfare, and you want to design a game such that the equilibrium of the game conforms to some concept of social optimality (i.e., aggregation of players’ preferences w.r.t. to a certain outcome). However, these preferences of individuals are now private, and so you have to disclose them in order to precisely evaluate the social utility of a certain outcome. Observe that it may be in the best interest of some agent to lie about her preferences: Indeed, this may lead to a certain outcome which improves its personal benefit, regardless if this may negatively affect other agents! Thus, in this strategic setting, which techniques can be used to convince players to cooperate honestly with the system? In one word, incentive them to reveal the truth!

The implementation problem (a bit more formally) Given: An economic system comprising of self-interested, rational players, which hold some secret information A system-wide goal (social-choice function (SCF), i.e., an aggregation of players’ preferences) Question: Does there exist a mechanism that can enforce (through suitable economic incentives) the players to behave in such a way that the desired goal is implemented (i.e., the outcome of the interaction is an equilibrium in which the SCF is optimized)?

Designing a Mechanism Informally, designing a mechanism means to define a game in which a desired outcome must be reached (in equilibrium) However, games induced by mechanisms are games with incomplete information: Players hold independent private values The payoff are a function of these types, and so each player doesn’t really know about the other players’ payoffs, but only about her one!  The DSE is a suitable solution concept here, since no knowledge about other players’ payoff (i.e., preferences) is required

In poor words… Game theory aims to investigate rational decision making in conflict situations, whereas mechanism design just concerns the reverse question: given some desirable outcome, can we design a game that produces it (in equilibrium)?

An example: sealed- bid auctions t 1 =10 t 2 =12 t 3 =7 r 1 =11 r 2 =10 SCF: the winner should be the guy having in mind the highest value for the painting The mechanism tells to players: (1)How the item will be allocated (i.e., who will be the winner), depending on the received bids (2) The payment the winner has to return, as a function of the received bids t i : is the maximum amount of money player i is willing to pay for the painting If player i wins and has to pay p then her utility is u i =t i -p, otherwise it is 0 r i : is the amount of money player i bids (in a sealed envelope) for the painting r 3 =7

A simple mechanism: no payment t 1 =10 t 2 =12 t 3 =7 r 1 =+  r2=+r2=+ r3=+r3=+ …it doesn’t work… ?!? Mechanism: The highest bid wins and the price of the item is 0

Another simple mechanism: pay your bid t 1 =10 t 2 =12 t 3 =7 r 1 =9 r 2 =8 r 3 =6 Is it the right choice? Mechanism: The highest bid wins and the winner will pay her bid The winner is player 1 and she will pay 9 Player i may bid r i < t i (in this way she is guaranteed not to incur a negative utility) …and so the winner could be the wrong one… …it doesn’t work…

An elegant solution: Vickrey’s second price auction t 1 =10 t 2 =12 t 3 =7 r 1 =10 r 2 =12 r 3 =7 every player has convenience to declare the truth! (we prove it in the next slide) I know they are not lying Mechanism: The highest bid wins and the winner will pay the second highest bid The winner is player 2 and she will pay 10

Theorem In the Vickrey auction, for every player i, r i =t i is a dominant strategy proof Fix i and t i, and look at strategies for player i. Let R= max j  i {r j } Case t i ≥ R (observe that R is unknown to player i) declaring r i =t i gives utility u i = t i -R ≥ 0 (player wins if t i > R, while if t i = R then player can either win or lose, depending on the tie-breaking rule, but her utility in this case is always 0) declaring any r i > R, r i ≠t i, yields again utility u i = t i -R ≥ 0 (player wins) Case t i < R declaring r i =t i yields utility u i = 0 (player loses) declaring any r i < R, r i ≠t i, yields again utility u i = 0 (player loses) declaring any r i > R yields u i = t i -R < 0 (player wins) declaring any r i < R yields u i =0 (player loses)  In all the cases, reporting a false type produces a not better utility, and so telling the truth is a dominant strategy!

Mechanism Design Problem: ingredients N players; each player i, i=1,..,N, has some private information t i  T i (actually, the only private info) called type Vickrey’s auction: the type is the value of the painting that a player has in mind, and so T i is the set of positive real numbers A set of feasible outcomes X (i.e., the result of the interaction of the players with the mechanism) Vickrey’s auction: X is the set of players (indeed an outcome of the auction is a winner of it, i.e., a player)

Mechanism Design Problem: ingredients (2) For each vector of types t=(t 1, t 2, …, t N ), and for each feasible outcome x  X, a SCF f(t,x) that measures the quality of x as a function of t. This is the function that the mechanism aims to implement (i.e., it aims to select an outcome x* that minimizes/maximizes it, but the problem is that types are unknown!) Vickrey’s auction: f(t,x) is the type associated with a feasible winner x (i.e., any of the players), and the objective is to maximize f, i.e., to allocate the painting to the bidder with highest type Each player has a strategy space S i and performs a strategic action; we restrict ourselves to direct revelation mechanisms, in which the action is reporting a value r i from the type space (with possibly r i  t i ), i.e., S i = T i Vickrey’s auction: the action is to bid a value r i

Mechanism Design Problem: ingredients (3) For each feasible outcome x  X, each player makes a valuation v i (t i,x) (in terms of some common currency), expressing her preference about that output x Vickrey’s auction: if player i wins the auction then her valuation is equal to her type t i, otherwise it is 0 For each feasible outcome x  X, each player receives a payment p i (x) by the system in terms of the common currency (a negative payment means that the agent makes a payment to the system); payments are used by the system to incentive players to be collaborative. Vickrey’s auction: if player i wins the auction then she makes a payment equal to -r j, where r j is the second highest bid, otherwise it is 0 Then, for each feasible outcome x  X, the utility of player i (in terms of the common currency) coming from outcome x will be: u i (t i,x) = p i (x) + v i (t i,x) Vickrey’s auction: if player i wins the auction then her utility is equal to u i = -r j +t i ≥ 0, where r j is the second highest bid, otherwise it is u i = 0+0=0

Mechanism Design Problem: the goal Given all the above ingredients, design a mechanism M=, where: g:T 1  …  T N  X is an algorithm which computes an outcome g(r)  X as a function of the reported types r p=(p 1,…,p N )  N is a payment scheme w.r.t. outcome g(r) that specifies a payment for each player which implements the SCF f(t,x) in equilibrium w.r.t. players’ utilities (according to a given solution concept, e.g., dominant strategy equilibrium, Nash equilibrium, etc.). (In other words, there exists a reported type vector r * for which the mechanism provides a solution g(r * ) and a payment scheme p(g(r * )) such that players’ utilities u i (t i,g(r * )) = p i (g(r * )) + v i (t i,g(r * )) are in equilibrium, and f(t,g(r * )) is optimal (either minimum or maximum))

Mechanism Design: a picture System player 1 player N “I propose to you the following mechanism M= ” p1p1 pNpN tNtN t 1t 1 r 1r 1 r Nr N Private “types” Reported types Payments Output which should implement the SCF in equilibrium w.r.t. players’ utilities Each player reports strategically to maximize her well-being… …in response to a payment which is a function of the output!

Our focus: implementation with dominant strategies Def.: A mechanism is an implementation with dominant strategies if the SCF f(t,x) is implemented in dominant strategy equilibrium, i.e., there exists a reported type vector r * =(r 1 *, r 2 *, …, r N * ) such that: 1. for each player i and for each reported type vector r =(r 1, r 2, …, r N ), it holds: u i (t i,g(r -i,r i * )) ≥ u i (t i,g(r)) where g(r -i,r i * )=g(r 1, …, r i-1, r i *, r i+1,…, r N ); 2. f(t,g(r*)) is optimized.

Strategy-Proof Mechanisms If truth telling is the dominant strategy in a mechanism then this is called Strategy-Proof or truthful  r * =t, and so players report their true types instead of strategically manipulating it, and the algorithm of the mechanism runs on the true input Strategy-proof mechanisms are very desirable from a system-wide point of view, since they disclose the true preferences of individuals. Notice, however, that f(t,x) could be also implemented in non-truthful dominant strategy. However, the revelation principle states that if there exists such an implementation, then there exists a truthful implementation as well.

Truthful Mechanism Design: Economics Issues QUESTION: How to design a truthful mechanism? Or, in other words: 1. How to design algorithm g, and 2. How to define the payment scheme p in such a way that the underlying SCF is implemented truthfully? Under which conditions can this be done?

Mechanism Design: Algorithmic Issues QUESTION: What is the time complexity of the mechanism? Or, in other words: What is the time complexity of computing g(r)? What is the time complexity to calculate the N payment functions? What does it happen if it is NP-hard to implement the underlying SCF? Question: What is the time complexity of the Vickrey auction? Answer: Θ(N), where N is the number of players. Indeed, it suffices to check all the offers, by keeping track of the largest one and of the second largest one.

Utilitarian problems: A problem is utilitarian if its SCF is such that f(t,x) =  i v i (t i,x), i.e., the SCF is separately-additive w.r.t. players’ valuations. Remark 1: the auction problem is utilitarian, in that f(t,x) is the type associated with the winner x, and the valuation of a player is either her type or 0, depending on whether she wins or not. Then, f(t,x) =  i v i (t i,x). Remark 2: in many network optimization problems (which are of our special interest) the SCF is separately-additive Good news: for utilitarian problems there exists a class of truthful mechanisms A prominent class of problems

Vickrey-Clarke-Groves (VCG) Mechanisms A VCG-mechanism is (the only) strategy-proof mechanism for utilitarian problems: Algorithm g computes: g(r) = arg max y  X  i v i (r i,y) Payment function for player i: p i (g(r)) = h i (r -i ) +  j≠i v j (r j,g(r)) where h i (r -i ) is an arbitrary function of the types reported by players other than player i. What about non-utilitarian problems? Strategy- proof mechanisms are known only when the type is a single parameter (we will see them later on).

Theorem VCG-mechanisms are truthful for utilitarian problems proof Fix i, r -i, t i. Let ř=(r -i,t i ) and consider a strategy r i  t i x=g(r -i,t i ) =g(ř) x’=g(r -i,r i ) u i (t i,x) = u i (t i,x’) = [h i (r -i )+  j  i v j (r j,x)] + v i (t i,x) [h i (r -i )+  j  i v j (r j,x’)] + v i (t i,x’) = h i (r -i ) +  j v j (ř j,x) +  j v j (ř j,x’) but x is an optimal solution w.r.t. ř =(r -i,t i ), i.e., x = arg max y  X  j v j (ř j,y)  j v j (ř j,x) ≥  j v j (ř j,x’) u i (t i,x)  u i (t i,x’). ř j =r j if j  i, and ř i =t i

How to define h i (r -i )? Remark: not all functions make sense. For instance, what does it happen in our Vickrey’s auction if we set for every player h i (r -i )=-1000 (notice this is independent of reported value r i of player i, and so it obeys to the definition)? Answer: It happens that players’ utility become negative; more precisely, the winner’s utility is u i (t i,x) = p i (x) + v i (t i,x) = h i (r -i ) +  j≠i v j (r j,x) + v i (t i,x) = = -988 while utility of losers is u i (t i,x) = p i (x) + v i (t i,x) = h i (r -i ) +  j≠i v j (r j,x) + v i (t i,x) = = -988  This is undesirable in reality, since with such perspective players would not participate to the auction!

Voluntary participation A mechanism satisfies the voluntary participation condition if agents who reports truthfully never incur a net loss, i.e., for all agents i, true values t i, and other agents’ bids r -i u i (t i,g(t i,r -i ))  0.

The Clarke payments This is a special VCG-mechanism in which h i (r - i ) = -  j≠i v j (r j,g(r - i ))  p i (g(r)) = -  j≠i v j (r j,g(r - i )) +  j≠i v j (r j,g(r)) With Clarke payments, one can prove that players’ utility are always non-negative  players are interested in playing the game solution maximizing the sum of valuations when player i doesn’t play

The Vickrey’s auction is a VCG mechanism with Clarke payments Recall that auctions are utilitarian problems. Then, the VCG-mechanism associated with the Vickrey’s auction is: g(r) = arg max y  X  i v i (r i,y) …this is equivalent to allocate to the bidder with highest reported cost (in the end, the highest type, since it is strategy-proof) p i (g(r)) = -  j≠i v j (r j,g(r - i )) +  j≠i v j (r j,g(r)) …this is equivalent to say that the winner pays the second highest offer, and the losers pay 0, respectively Remark: the difference between the second highest offer and the highest offer is unbounded (frugality issue)

VCG-Mechanisms: Advantages For System Designer: The goal, i.e., the optimization of the SCF, is achieved with certainty For players: players have truth telling as the dominant strategy, so they need not require any computational systems to deliberate about other players strategies

VCG-Mechanisms: Disadvantages For System Designer: The payments may be sub-optimal (frugality) Apparently, with Clarke payments, the system may need to run the mechanism’s algorithm N+1 times: once with all players (for computing the outcome g(r)), and once for every player (indeed, for computing the payment p i associated with player i, we need to know g(r -i ))  If the problem is hard to solve then the computational cost may be very heavy For players: players may not like to tell the truth to the system designer as it can be used in other ways

Assignment: redefine the Vickrey auction in the minimization version t 1 =10 t 2 =12 t 3 =7 r 1 =10 r 2 =12 r 3 =7 I want to allocate the job to the true cheapest machine Once again, the second price auction works: the cheapest bid wins and the winner will get the second cheapest bid The winner is machine 3 and it will receive 10 job to be allocated to machines t i : cost incurred by i if she does the job if machine i is selected and receives a payment of p her utility is p-t i

Thanks for your attention! Questions?