Nash’s Theorem Theorem (Nash, 1951): Every finite game (finite number of players, finite number of pure strategies) has at least one mixed-strategy Nash equilibrium. Nash, John (1951) "Non-Cooperative Games" The Annals of Mathematics 54(2):286-295. (John Nash did not call them “Nash equilibria”, that name came later.) He shared the 1994 Nobel Memorial Prize in Economic Sciences with game theorists Reinhard Selten and John Harsanyi for his work on Nash equilibria. He suffered from schizophrenia in the 1950s and 1960s, as depicted in the 1998 film, “A Beautiful Mind”. He nevertheless recovered enough to return to academia and continue his research.

More on Constant-Sum Games Minimax Theorem (John von Neumann, 1928): For every two-person, zero-sum game with finitely many pure strategies, there exists a mixed strategy for each player and a value V such that: Given player 2’s strategy, the best possible payoff for player 1 is V Given player 1’s strategy, the best possible payoff for player 2 is –V. The existence of strategies part is a special case of Nash’s theorem, and a precursor to it. This basically says that player 1 can guarantee himself a payoff of at least V, and player 2 can guarantee himself a payoff of at least –V. If both players play optimally, that’s exactly what they will get. It’s called “minimax” because the players get this value by pursuing a strategy that tries to minimize the maximum payoff of the other player. We’ll come back to this. Definition: The value V is called the value of the game. Eg: The value of Rock-paper-scissors is 0; the best that P1 can hope to achieve, assuming P2 plays optimally (1/3 probability of each action), is a payoff of 0.

Computing Nash Equilibria In general, it’s quite expensive, although it’s not known exactly how this relates to P or NP. For two-person, constant-sum games, this problem reduces to another problem called “Linear Programming”, which is in P.

Computing Nash Equilibria: 2-person, Zero-Sum Games The Odds and Evens game has no pure-strategy Nash equilibria. By Nash’s theorem, it must have a mixed-strategy Nash equilibrium. How can we find it? Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Let’s start by making some definitions. Let p1 be the probability that the Even player plays 1 finger, in the Nash equilibrium. So with probability 1-p1, Even will play 2 fingers. Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Let’s start by making some definitions. Likewise, let q1 be the probability that the Odd player plays 1 finger, in the Nash equilibrium. So with probability 1-q1, Odd will play 2 fingers. Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Next, let’s write down what we know about the outcomes, in terms of p1 and q1. In equilibrium, Odd’s expected payoff is: q1*p1*(-2) + q1*(1-p1)*(+3) + (1-q1)*p1*(+3) + (1-q1)*(1-p1)*(-4) Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Next, let’s write down what we know about the outcomes, in terms of p1 and q1. In equilibrium, Even’s expected payoff is: q1*p1*(+2) + q1*(1-p1)*(-3) + (1-q1)*p1*(-3) + (1-q1)*(1-p1)*(+4) Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Observation: If Even selects p1 so that Odd gets a higher utility by playing 1 finger instead of 2 fingers, then Odd will always select 1 finger. But that can’t be an equilibrium! (Why not?) Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Observation: Likewise, if Even selects p1 so that Odd gets a higher utility by playing 2 fingers instead of 1 fingers, then Odd will always select 2 fingers. But that can’t be an equilibrium, either! Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Observation: So, the only possible equilibrium has Even selecting p1 so that Odd’s payoff for selecting 1 finger equals Odd’s payoff for selecting 2 fingers. Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games In algebra: Odd’s payoff when Even plays 1 finger with probability p1, and Odd always plays 1 finger: p1*(-2) + (1-p1)*(+3) Odd’s payoff when Even plays 1 finger with probability p1, and Odd always plays 2 fingers: p1*(+3) + (1-p1)*(-4) Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games Our observation says these should be equal: p1*(-2) + (1-p1)*(+3) = p1*(+3) + (1-p1)*(-4) => -2p1 + 3 – 3p1 = 3p1 -4 + 4p1 7 = 12p1 p1 = 7 / 12 Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games We could have done this for either player; here it is from Odd’s perspective: q1*(+2) + (1-q1)*(-3) = q1*(-3) + (1-q1)*(+4) => 2q1 – 3 + 3q1 = -3 q1 +4 -4q1 12q1 = 7 q1 = 7/12 Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Computing Nash Equilibria: 2-person, Zero-Sum Games So now we know a mixed-strategy Nash equilibrium: POdd(1 finger) = 7/12 POdd(2 fingers) = 5/12 PEven(1 finger) = 7/12 PEven(2 fingers) = 5/12 Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Quiz: 2-person, Zero-Sum Games What is the value of this game for Even? (Remember, the value of the game is the expected payoff for the player in equilibrium.) Likewise, what is the value of the game for Odd? Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Answer: 2-person, Zero-Sum Games You can get the value for Even three ways: Recall: In equilibrium, Even’s expected payoff is: q1*p1*(+2) + q1*(1-p1)*(-3) + (1-q1)*p1*(-3) + (1-q1)*(1-p1)*(+4) Or, q1*(+2) + (1-q1)*(-3) or, q1*(-3) + (1-q1)*(+4) These all equal: -1/12 Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

Answer: 2-person, Zero-Sum Games You can get the value for Odd the same three ways, or you can just say that this is a zero-sum game, so the value for Odd must be opposite the value for Even: +1/12 In other words, it’s better to be the Odd player than the Even player, since Odd will win, on average. Odd Player 1 finger 2 fingers -2, +2 +3, -3 -4, +4 1 finger Even Player 2 finger The Odds and Evens Game

2-person games with more actions When there are more actions available than 2 per person, the simple algorithm I gave will no longer work. However, it is still possible to compute Nash equilibria for zero-sum games in polynomial time using a technique called Linear Programming. Linear Programming is a well-known kind of problem with existing solvers, and I won’t cover it in detail here.

Quiz: Computing an Equilibrium for Zero-Sum Games In equilibrium, What is the probability that P1 plays X? What is the probability that P2 plays X? What is the value of the game for P1? Player 1 X Y +5, -5 +2, -2 +3, -3 +6, -6 X Player 2 Y

Answer: Computing an Equilibrium for Zero-Sum Games In equilibrium, What is the probability that P1 plays X? 2/3 What is the probability that P2 plays X? 0.5 What is the value of the game for P1? 4 Player 1 X Y +5, -5 +2, -2 +3, -3 +6, -6 X Player 2 Y

Games beyond this class’s limits There are MANY aspects of games and Game Theory in AI that we will not cover. I’ll briefly mention some of them: Repeated games and Learning Communication between agents Mechanism Design: How to create games so that agents have the incentives to behave in desirable ways (eg, voting and auctions)

1. Repeated Games and Learning Many games (e.g., Rock-Paper-Scissors) are typically played multiple times. These are called repeated games. This can change the incentive structure and the best strategies: E.g., in the Prisoner’s Dilemma, it might be better to say nothing if you believe you can teach your opponent to cooperate and say nothing as well.

Learning and Teaching in Repeated Games This history of play in repeated games offer examples of your opponent’s strategy. This provides an opportunity for learning. It also provides an opportunity for teaching! In multi-agent settings with repeated games, every agent is both a learner and a teacher.

Example learning strategy: “Fictitious Play” Idea: build a model of what the opponent’s strategy is, and then play a best response. Fictitious Play Learning Create an array A that has an entry for each of the opponent’s actions. Initialize with prior beliefs. Repeat: Assuming the counts in A represent the opponents mixed strategy, play a best response to A. Observe the opponent’s action, and update the appropriate count in A.

Some Theoretical Results about Fictitious Play Theorem: If both players use fictitious play, and if the empirical distribution of their chosen actions converges, then it converges to a Nash equilibrium. Theorem: In zero sum games, if both players use fictitious play, they will converge on a Nash equilibrium.

2. Communication in Games Sometimes, communication can improve player outcomes. Player 1 says: “I will play C”. Response? Player 1 says: “I will play C”. Response? D C D C +1, +1 0, +5 +5, 0 +3, +3 +1, +1 0, 0 D D C C Prisoner’s Dilemma Coordination game

2. Communication in Games In the coordination game, P1’s statement is self-commiting and self-revealing, so believable. Player 1 says: “I will play C”. Response? Player 1 says: “I will play C”. Response? D C D C +1, +1 0, +5 +5, 0 +3, +3 +1, +1 0, 0 D D C C Prisoner’s Dilemma Coordination game

3. Mechanism Design: Creating Games with Desired Outcomes Elections and auctions are examples of games: they involve multiple agents, possible actions for each agent (who to vote for, how much to bid), and outcomes that depend on all of the agents’ outcomes. “Mechanism Design” is the study of creating a reward structure so that we have good outcomes, such as that the most popular politician gets elected, or that the person who benefits most from a good wins the auction.

Arrow’s Theorem Definition: A voting mechanism is dictatorial if it exactly follows the preferences of a single voter (called the dictator). Theorem (Arrow, 1951) (Informally): Any voting mechanism in which voters express their true preferences for the outcomes (candidates) that has at least 3 outcomes that always selects the most popular outcome and where the choice between two outcomes is not affected by other less-popular outcomes must be dictatorial. Note: This is a well-known example of an impossibility theorem: a theorem that says it is impossible to design a game with a certain list of desirable properties. This theorem and many like it don’t apply to certain kinds of voting, like rating systems (where voters rate each outcome, for example on a scale of 1-10, rather than specifying preferences.) But it does apply to most voting mechanisms in modern democracies. Which property does the US presidential voting system fail on?

Second-Price Auctions Definition: A second-price auction awards the good to the highest bidder, and charges a price equal to the second-highest bid.

Quiz: Second-Price Auctions Fill in the matrix of payoffs Let v=10 be your value for a good. Let b be your bid. Let c be the highest bid by anyone else in the auction. Your payoff is: v-c if b > c (you win the auction) 0 if b <= c (you lose the auction) C=7 C=9 C=11 C=13 B=12 B=10 B=8 Is there a dominant strategy? If yes, is the strategy “truth-revealing”? (That is, does the strategy make you bid exactly how much you value the good?)

Answer: Second-Price Auctions Fill in the matrix of payoffs Let v=10 be your value for a good. Let b be your bid. Let c be the highest bid by anyone else in the auction. Your payoff is: v-c if b > c (you win the auction) 0 if b <= c (you lose the auction) C=7 C=9 C=11 C=13 B=12 3 1 -1 B=10 B=8 Is there a dominant strategy? Yes, bid b=10 If yes, is the strategy “truth-revealing”? Yes, the dominant strategy matches the value v=10

Second-Price Auctions Definition: A second-price auction awards the good to the highest bidder, and charges a price equal to the second-highest bid. Some properties (under a bunch of assumptions that I won’t get into): They are pareto efficient They are dominant strategy-truthful: the best strategy is to bid exactly what you think the good is worth to you. It is always worth it for agents to take part in the auction. The auctioneer will never lose money. These auctions come from a family of auctions called Vickrey-Clarke-Groves mechanisms, and these are the only possible mechanisms that have the first two properties.