Presentation on theme: "The Structure of Networks"— Presentation transcript:
1 The Structure of Networks with emphasis on information and social networksGame Theory: Chapter 6&7Ýmir Vigfússon
2 Mixed strategies Do Nash equilibria always exist? Matching Pennies gamePlayer #1 wins if mismatch, #2 if matchExample of a zero-sum gameWhat one player gains, the other losesE.g. Allied landing in Europe on June 6, 1944How would you play this game?HeadsTails-1, +1+1, -1
3 Mixed strategies You randomize your strategy Instead of choosing H/T directly, choose a probability you will choose H.Player 1 commits to play H with some probability pSimilarly, player 2 plays H with probability qThis is called a mixed strategyAs opposed to a pure strategy (e.g. p=0)What about the payoffs?
4 Mixed strategies Suppose player 1 evaluates pure strategies Player 2 meanwhile chooses strategy qIf Player 1 chooses H, he gets a payoff of -1 with probability q and +1 with probability 1-qIf Player 1 chooses T, he gets -1 with probability 1-q and +1 with probability qIs H or T more appealing to player 1?Rank the expected valuesPick H: expect (-1)(q) + (+1)(1-q) = 1-2qPick T: expect (+1)(q) + (-1)(1-q) = 2q -1
5 Mixed strategies Def: Nash equilibrium for mixed strategies A pair of strategies (now probabilities) such that each is a best response to the other.Thm: Nash proved that this always exists.In Matching Pennies, no Nash equilibrium can use a pure strategyPlayer 2 would have a unique best response which is a pure strategyBut this is not the best response for player 1...What is Player 1‘s best response to strategy q?If 1-2q ≠2q-1, then a pure strategy (either H or T) is a unique best response to player 1.This can‘t be part of a Nash equilibrium by the aboveSo must have 1-2q=2q-1 in any Nash equilibriumWhich gives q=1/2. Similarly p=1/2 for Player 1.This is a unique Nash equilibrium (check!)
6 Mixed strategiesIntuitively, mixed strategies are used to make it harder for the opponent to predict what will be playedBy setting q=1/2, Player 2 makes Player 1 indifferent between playing H or T.How do we interpret mixed equilibria?In sports (or real games)Players are indeed randomizing their actionsCompetition for food among speciesIndividuals are hardwired to play certain strategiesMixed strategies are proportions within populationsPopulation as a whole is a mixed equilibriumNash equilibrium is an equilibrium in beliefsIf you believe other person will play a Nash equilibrium strategy, so will you.It is self-reinforcing – an equilibrium
7 Mixed strategies: Examples American footballOffense can run with the ball, or pass forwardWhat happens?Suppose the defense defends against a pass with probability qP: expect (0)(q) + (10)(1-q) = 10-10qR: expect (5)(q) + (0)(1-q) = 5qOffense is indifferent when q=2/3Defend runDefend passPass0, 010, -10Run5, -5
8 Mixed strategies: Examples American footballOffense can run with the ball, or pass forwardWhat happens?Suppose offense passes with probability pSimilarly, defense is indifferent when p=1/3(1/3,2/3) is a Nash equilibriumExpected payoff to offense: 10/3 (yard gain)Defend runDefend passPass0, 010, -10Run5, -5
9 Mixed strategies: Examples Penalty-kick gameSoccer penalties have been studied extensivelySuppose goalie defends left with probability qKicker indifferent when(0.58)(q) + (0.95) (1-q) = (0.93)(q) + (0.70) (1-q)Get q =0.42. Similarly p=0.39True values from data? q=0.42 , p=0.40 !!The theory predicts reality very wellDefend leftDefend rightLeft0.58, -0.580.95, -0.95Right0.93, -0.930.70, -0.70
10 Pareto optimalityEven playing best responses does not always reach a good outcome as a groupE.g. prisoner‘s dilemmaWant to define a socially good outcomeDef:A choice of strategies is Pareto optimal if no other choice of strategies gives all players a payoff at least as high, and at least one player gets a strictly higher payoffNote: Everyone must do at least as well
11 Social optimality Def: Example: A choice of strategies is a social welfare maximizer (or socially optimal) if it maximizes the sum of the players‘ payoffs.Example:The unique Nash equilibrium in this game is socially optimalPresentationExam98,9894,9696,9492,92
12 Game theory Regular game theory Evolutionary game theory Individual players make decisionsPayoffs depend on decisions made by allThe reasoning about what other players might do happens simultaneouslyEvolutionary game theoryGame theory continues to apply even if no individual is overtly reasoning or making explicit decisionsDecisions may thus not be consciousWhat behavior will persist in a population?
13 Background Evolutionary biology The idea that an organism‘s genes largely determine its observable characteristics (fitness) in a given environmentMore fit organisms will produce more offspringThis causes genes that provide greater fitness to increase their representation in the populationNatural selection
14 Evolutionary game theory Key insightMany behaviors involve the interaction of multiple organisms in a populationThe success of an organism depends on how its behavior interacts with that of othersCan‘t measure fitness of an individual organismSo fitness must be evaluated in the context of the full population in which it livesAnalogous to game theory!Organisms‘s genetically determined characteristics and behavior = StrategyFitness = PayoffPayoff depends on strategies of organisms with which it interacts = Game matrix
15 Motivating example Let‘s look at a species of a beetle Each beetle‘s fitness depends on finding and processing food effectivelyMutation introducedBeetles with mutation have larger body sizeLarge beetles need more foodWhat would we expect to happen?This makes them less fit for the environmentThe mutation will thus die out over timeBut there is more to the story...
16 Motivating example Beetles compete with each other for food Large beetles more effective at claiming above-average share of the foodAssume food competition is among pairsSame sized beetles get equal shares of foodA large beetle gets the majority of food from a smaller beetleLarge beetles always experience less fitness benefit from given quantity of foodNeed to maintain their expensive metabolism
17 Motivating example The body-size game between two beetles Something funny about thisNo beetle is asking itself: “Do I want to be small or large?“Need to think about strategy changes that operate over longer time scalesTaking place as shifts in population under evolutionary forcesSmallLarge5, 51, 88, 13, 3
18 Evolutionary stable strategies The concept of a Nash equilibrium doesn‘t work in this settingNobody is changing their personal strategyInstead, we want an evolutionary stable strategyA genetically determined strategy that tends to persist once it is prevalent in a populationNeed to make this precise...
19 Evolutionarily stable strategies Suppose each beatle is repeatedly paired off with other beetles at randomPopulation large enough so that there are no repeated interactions between two beetlesA beetle‘s fitness = average fitness from food interactions = reproductive successMore food thus means more offspring to carry genes (strategy) to the next generationDef:A strategy is evolutionarily stable if everyone uses it, and any small group of invaders with a different strategy will die off over multiple generations
20 Evolutionarily stable strategies Def: More formallyFitness of an organism in a population = expected payoff from interaction with another member of populationStrategy T invades a strategy S at level x (for small x) if:x fraction of population uses T1-x fraction of population uses SStrategy S is evolutionarily stable if there is some number y such that:When any other strategy T invades S at any level x < y, the fitness of an organism playing S is strictly greater than the fitness of an organism playing T
21 Motivating example Is Small an evolutionarily stable strategy? Let‘s use the definitionSuppose for some small number x, a 1-x fraction of population use Small and x use LargeIn other words, a small invader population of Large beetlesWhat is the expected payoff to a Small beetle in a random interaction?With prob. 1-x, meet another Small beetle for a payoff of 5With prob. x, meet Large beetle for a payoff of 1Expected payoff: 5(1-x) + 1x = 5-4x
22 Motivating example Is Small an evolutionarily stable strategy? Let‘s use the definitionSuppose for some small number x, a 1-x fraction of population use Small and x use LargeIn other words, a small invader population of Large beetlesWhat is the expected payoff to a Large beetle in a random interaction?With prob. 1-x, meet a Small beetle for payoff of 8With prob. x, meet another Large beetle for a payoff of 3Expected payoff: 8(1-x) + 3x = 8-5x
23 Motivating example Expected fitness of Large beetles is 8-5x Expected fitness of Small beetles is 5-4xFor small enough x (and even big x), the fitness of Large beetles exceeds the fitness for SmallThus Small is not evolutionarily stableWhat about the Large strategy?Assume x fraction are Small, rest Large.Expected payoff to Large: 3(1-x) + 8x = 3+5xExpected payoff to Small: 1(1-x) + 5x = 1+4xLarge is evolutionarily stable
24 Motivating example Summary A few large beetles introduced into a population consisting of small beetlesLarge beetles will do really well:They rarely meet each otherThey get most of the food in most competitionsPopulation of small beetles cannot drive out the large onesSo Small is not evolutionarily stable
25 Motivating example Summary The structure is like prisoner‘s dilemma Conversely, a few small beetles will do very badlyThey will lose almost every competition for foodA population of large beetles resists the invasion of small beetlesLarge is thus evolutionarily stableThe structure is like prisoner‘s dilemmaCompetition for food = arms raceBeetles can‘t change body sizes, but evolutionarily forces over multiple generations are achieving analogous effect
26 Motivating example Even more striking feature! Start from a population of small beetlesEvolution by natural selection is causing the fitness of the organisms to decrease over timeDoes this contradict Darwin‘s theory?Natural selection increases fitness in a fixed environmentEach beetle‘s environment includes all other beetlesThe environment is thus changingIt is becoming increasingly more hostile for everyoneThis naturally decreases the fitness of the population
27 Evolutionary arms races Lots of examplesHeight of trees follows prisoner‘s dilemmaOnly applies to a particular height rangeMore sunlight offset by fitness downside of heightRoots of soybean plants to claim resourcesConserve vs. ExploreHard to truly determine payoffs in real- world settings
28 Evolutionary arms races One recent example with known payoffsVirus populations can play an evolutionary version of prisoner‘s dilemmaVirus AInfects bacteriaManifactures products required for replicationVirus BMutated version of ACan replicate inside bacteria, but less efficientlyBenefits from presence of AIs B evolutionarily stable?
29 Virus game Look at interactions between two viruses Viruses in a pure A population do better than viruses in pure B populationBut regardless of what other viruses do, higher payoff to be BThus B is evolutionarily stableEven though A would have been betterSimilar to the exam-presentation gameAB1.00, 1.000.65, 1.991.99, 0.650.83, 0.83
30 What happens in general? Under what conditions is a strategy evolutionarily stable?Need to figure out the right form of the payoff matrixHow do we write the condition of evolutionary stability in terms of these 4 variables, a,b,c,d?Organism 2STa, ab, cc, bd, dOrganism 1
31 What happens in general? Look at the definition againSuppose again that for some small number x:A 1-x fraction of the population uses SAn x fraction of the population uses TWhat is the payoff for playing S in a random interaction in the population?Meet another S with prob. 1-x. Payoff = aMeet T with prob. x. Payoff = bExpected payoff = a(1-x)+bxAnalogous for playing TExpected payoff = c(1-x)+dx
32 What happens in general? Therefore, S is evolutionarily stable if for all small values of x:a(1-x)+bx > c(1-x)+dxWhen x is really small (goes to 0), this isa > cWhen a=c, the left hand side is larger whenb > dIn other wordsIn a two-player, two-strategy symmetric game, S is evolutionarily stable precisely when eithera > c, ora = c, and b > d
33 What happens in general? IntuitionIn order for S to be evolutionarily stable, then:Using S against S must be at least as good as using T against SOtherwise, an invader using T would have higher fitness than the rest of the populationIf S and T are equally good responses to SS can only be evolutionarily stable if those who play S do better against T than what those who play T do with each anotherOtherwise, T players would do as well against the S part of the population as the S players
34 Relationship with Nash equilibria Let‘s look at Nash in the symmetric gameWhen is (S,S) a Nash equilibrium?S is a best response to S: a ≥ cCompare with evolutionarily stable strategies:(i) a > c or (ii) a = c and b > dVery similar!STa, ab, cc, bd, d
35 Relationship with Nash equilibria We get the following conclusionThm: If strategy S is evolutionary stable, then (S,S) is a Nash equilibriumDoes the other direction hold?What if a = c, and b < d?Can we construct such an example?
36 From last time Stag Hunt Two equilibria, but “riskier“ to hunt stag If hunters work together, they can catch a stagOn their own they can each catch a hareIf one hunter tries for a stag, he gets nothingTwo equilibria, but “riskier“ to hunt stagWhat if other player hunts hare? Get nothingSimilar to prisoner‘s dilemmaMust trust other person to get best outcome!Hunt StagHunt Hare4, 40, 33, 03, 3
37 Counterexample Modify the game a bit Want: a = c, and b < d 4, 4 Hunt StagHunt Hare4, 40, 33, 03, 3STa, ab, cc, bd, d
38 Counterexample Modify the game a bit Want: a = c, and b < d We‘re done!Hunt StagHunt Hare4, 40, 44, 03, 3STa, ab, cc, bd, d
39 Relationship with Nash equilibria We get the following conclusion:Thm: If strategy S is evolutionarily stable, then (S,S) is a Nash equilibriumDoes the other direction hold?Won‘t hold if some game has a = c, and b < dCan we construct such an example? Yes!However! Look at a strict Nash equilibriumThe condition gives a > cThm: If (S,S) is a strict Nash equilibrium, then strategy S is evolutionarily stableThe equilibrium concepts refine one another
40 Summary Nash equilibrium Evolutionarily stable strategies Rational players choosing mutual best responses to each other‘s strategyGreat demands on the ability to choose optimally and coordinate on strategies that are best responses to each otherEvolutionarily stable strategiesNo intelligence or coordinationStrategies hard-wired into players (genes)Successful strategies produce more offspringYet somehow they are almost the same!
41 Mixed strategiesIt may be the case that no strategy is evolutionarily stableThe Hawk-Dove game is an exampleHawk does well in all-Dove populationDove does well in population of all HawksThe game even has two Nash equilibria!We introduced mixed strategies to study this phenomenonHow should we define this in our setting?
42 Mixed strategies Suppose: Expected payoff for organism 1 In total: Organism 1 plays S with probability pPlays T with probability 1-pOrganism 2 plays S with probability qPlays T with probability 1-qExpected payoff for organism 1Probability pq of (S,S) pairing, giving aProbability p(1-q) of (S,T) pairing, giving bProbability (1-p)q of (T,S) pairing, giving cProbability p(1-q) of (T,T) pairing, giving dIn total:V(p,q) = pqa+p(1-q)b+(1-p)qc+(1-p)(1-q)d
43 Mixed strategiesFitness of an organism = expected payoff in a random interactionMore precisely:Def: p is an evolutionary stable mixed strategy if there is a small positive number y s.t.when any other mixed strategy q invades p at any level x<y, thenthe fitness of an organism playing p is strictly greater than the fitness of an organism playing q
44 Mixed strategies Let‘s dig into this condition p is an evolutionarily stable mixed strategy if:For some y and any x < y, the following holds for all mixed strategies q ≠ p:(1-x)V(p,p) + xV(p,q) > (1-x)V(q,p) + xV(q,q)This parallels what we saw earlier for mixed Nash equilibriaIf p is an evolutionarily stable mixed strategy then V(p,p) ≥ V(q,p),Thus p is a best response to qSo (p,p) is a mixed Nash equilibrium
46 Interpretation Can interpret this in two ways All participants in the population are mixing over two possible pure strategies with given probabilityMembers genetically the samePopulation level: 1/3 of animals hard-wired to play D and 2/3 are hard-wired to always play H