Presentation on theme: "Review: Decision Theory"— Presentation transcript:
1 Review: Decision Theory Structure of ProblemChoicesRandom events with probabilityPayoffsDiagram itBox for choice, circle for random, linesProbabilities and payoffsSolve it, right to leftEvaluate circlesAt square, lop off inferior choicesContinue until left with only one best choice
2 Game theory: The Problem Strategic behaviorOutcome depends on choices by both (or all) playersSo what I should do depends on what you doWhat you should do depends on what I doThis describes, among other thingsMuch of economicsPoliticsDiplomacy…If we had a general solution, it might be usefulThe Theory of Games and Economic Behavior
3 Strategic BehaviorA lot of what we do involves optimizing against natureShould I take an umbrella?What crops should I plant?How do we treat this disease or injury?How do I fix this car?We sometimes imagine it as a game against a malevolent opponentsFinagle's Law: If Something Can Go Wrong, It Will"The perversity of inanimate objects"Yet we know it isn'tBut consider a two person zero sum game, where what I win you lose.From my standpoint, your perversity is a fact not an illusionBecause you are acting to maximize your winnings, hence minimize mine
4 Consider a non-fixed sum game My apple is worthNothing to me (I'm allergic)One dollar to you (the only customer)If I sell it to you, the sum of our gains is … ?If bargaining breaks down and I don't sell it, the sum of our gains is … ?So we have both cooperation--to get a deal--and conflict over the terms.Giving us the paradox thatIf I will not accept less than $.90, you should pay that, but …If you will not offer more than $.10, I should accept that.Bringing in the possibility of bluffs, commitment strategies, and the like.
5 Consider a Many Player Game Many playersEach choosing movesTrying maximize his returnsThis adds a new element: CoalitionsEven if the game is fixed sum for all of us put togetherIt can be positive sum for a group of playersAt the cost of those outside the groupSo we really have three games at onceForming coalitionsThe coalition playing against othersThe coalition dividing its gains among its members
6 To “Solve” a game we need A way of describing a game—Better a way of describing all gamesA definition of a solutionA way of finding itSome possible descriptions:Decision treeStrategy matrix
7 Decision TreeA sequence of choices, except that now some are made by player 1, some by player 2 (and perhaps 3, 4, …)May still be some random elements as wellCan rapidly become unmanageably complicated, but …Useful for one purpose: Subgame Perfect EquilibriumA solution conceptBack to our basketball player--this timea two person game. Team, playerTo solveUpper blue box: Doesn't playLower: PlaysRepeat for other fourBack to decision theoryAgain we are going right to leftAt each last choice, take thealternative with higher payoffGiven that A knows B’s last choiceWhat is now A’s best last choice?Continue until only one sequence remainsThis is called “Subgame Perfect Equilibrium”
8 But … Tantrum/No Tantrum game Child ignores the logic of subgame perfect equilibriumBecause he knows it is a repeated gameAnd if he is going to ignore it, parent should let him stay upSo Subgame Perfect works only if commitment strategies are not available
9 Strategy Matrix Scissors/Paper/Stone The rules Payoff Scissors cut paperStone breaks scissorsPaper covers stonePayoffLoser pays winner a dollarNo payment in case of tieDescription: Matrix of strategies and outcomes
10 Strategy Matrix Each player chooses a strategy Player 1 picks a column Player 2 picks a rowThe intersection shows the payoffs to the two players
11 Any Two Player Game Can be Described This Way My strategy is a complete descriptionOf what I am going to do, includingHow it depends on observed random eventsAnd on what I see the other player doingYour strategy similarlyGiven my strategy and yoursWe can see what happens when they are played out togetherWith the outcome possibly probabilistic depending on random elements in the gameSo any game can be represented as a matrixA list of all possible strategies for player 1A list for player 2At the intersection, the outcome if that pair is chosenThe book never explains this, which is whyIt claims some games can't be represented on a matrixAnd that "don't sue/go to court" is a Nash EquilibriumBoth of which are wrong if we think in terms of strategies, not actsWhat about die rolls, card shuffles, and the like?What does “zero sum” mean in this description?
12 Prisoner’s DilemmaThere is a dominant pair of strategies--confess/confessMeaning that whatever Player 1 does, Player 2 is better off confessing, andWhatever Player 2, does Player 1 is better off confessingEven though both would be better off if neither confessedOne possible sense of “solution” to a game.
13 Dealing with Prisoner’s Dilemma Simple example of a general problemWhere individual rational behavior by two or moreMakes them worse off than irrational (don’t betray)Other examples?If you are in a PD, how might you get out?Enforceable contractI won't confess if you won'tIn that case, using nonlegal mechanisms to enforceCommitment strategy--you peach on me and when I get out …Repeat plays?Double cross me this timeI’ll double cross you nextDoesn’t work for a fixed number of plays!Might for an unknown numberWhich is the game all of us are in
14 Examples? Athletes taking steroids. Is it a PD? Countries engaging in an arms raceStudents studying in order to get better grades?
15 Defining a Solution What does “solution of a game” mean? “What will happen”“How to play?”What will happen if everyone plays perfectly?Dominant strategy (as in prisoner’s dilemma)Whatever he does, I should do XWhatever I do, he should do YReasonable definition--if it exists. But …Consider scissors/paper/stoneVon Neumann solution for 2 person zero sumNash equilibrium for many person
16 Von Neumann SolutionVon Neumann proved that for any 2 player zero sum gameThere was a pair of strategies, one for player A, one for B,And a payoff P for A (-P for B)Such that if A played his strategy, he would (on average) get at least P whatever B did.And if B played his, A would get at most P whatever he didHow can this be true for scissors/paper/stone?You can choose a “mixed strategy”Roll a die where the other player can’t see it1,2: scissors3,4: paper5,6: stoneWhat is my payoff to this, whatever you do? Yours, whatever I do?Including mixed strategies, there is always a VN solution
17 Nash EquilibriumCalled that because it was invented by Cournot, in accordance with Stigler's LawWhich holds that scientific laws are never named after their real inventorsCournot invented the two player version long before NashPuzzle:Consider a many player game.Each player chooses a strategyGiven the choices of the other players, my strategy is best for meAnd similarly for each other playerNash EquilibriumDriving on the right side of the road is a Nash EquilibriumIf everyone else drives on the right, I would be wise to do the sameSimilarly if everyone else drives on the leftSo multiple equilibria are possibleWho invented Stigler's Law?
18 Other Problems with Nash Equilibrium One problem: It assumes no coordinated changesA crowd of prisoners are escaping from Death RowFaced by a guard with one bullet in his gunGuard will shoot the first one to charge himStanding still until they are captured is a Nash EquilibriumIf everyone else does it, I had better do it too.Are there any others?But if I and my buddy jointly charge him, we are both better off.One reason why it doesn't work well for two player gamesSecond problem: Definition of Strategy is ambiguous.Consider an oligopoly--an industry with (say) 6 firmsEach chooses how much to produce, what price to askBut that’s really a single choice, sinceThe price I charge determines how much I can sellNash equilibrium: I choose the best strategy given what others chooseBut am I choosing the best price, given their priceOr the best quantity, given their quantity?It turns out that the solutions in the two cases are entirely different
19 Solution ConceptsSubgame Perfect equilibrium--if it exists and no commitment is possibleStrict dominance--"whatever he does …" Prisoner's DilemmaVon Neumann solution to 2 player gameNash EquilibriumAnd there are more
20 Von Neumann Solution to Many Player Games Outcome--how much each player ends up withDominance: Outcome A dominates B if there is some group of players, all of whom do better under A (end up with more) and who, by working together, can get A for themselvesA solution is a set of outcomes none of which dominates another, such that every outcome not in the solution is dominated by one in the solutionConsider, for example, …
21 Three Player Majority Vote A dollar is to be divided among Ann, Bill and Charles by majority vote.Ann and Bill propose (.5,.5,0)--they split the dollar, leaving Charles with nothingCharles proposes (.6,0,.4). Ann and Charles both prefer it, so it beats the first proposal, but …Bill proposes (0, .5, .5), which beats that …And so around we go.One Von Neumann solution is the set: (.5,.5,0), (0, .5, .5), (.5,0,.5) (check)There are others--lots of others.
22 Applied Schelling ("Focal") Points In a bargaining situation, people may end up with a solution because it is perceived as unique, hence better than continued (costly) bargainingWe can go on forever as to whether I am entitled to 61% of the loot or 62%Whether to split 50/50 or keep bargaining is a simpler decision.But what solution is unique is a function of how people think about the problemThe bank robbery was done by your family (you and your son) and mine (me and my wife and daughter)Is the Schelling point 50/50 between the families, or 20% to each person?Obviously the latter (obvious to me--not to you).It was only a two person job--but I was the one who bribed a clerk to get inside informationShould we split the loot 50/50 orThe profit 50/50--after paying me back for the bribe?In bargaining with a union, when everyone gets tired, the obvious suggestion is to "split the difference."But what the difference is depends on each party's previous offersWhich gives each an incentive to make offers unrealistically favorable to itself.What is the strategic implication?If you are in a situation where the outcome is likely to be agreement on a Schelling pointHow might you improve the outcome for your side?
23 Experimental Game Theory Computers work cheapSo Axelrod set up a tournamentHumans submit programs defining a strategy for many times repeated prisoner's dilemmaPrograms are randomly paired with each other to play (say) 100 timesWhen it is over, which program wins?In the first experiment, the winner was "tit for tat"Cooperate in the first roundIf the other player betrays on any round, betray him the next round (punish), but cooperate thereafter if he does (forgive)In fancier versions, you have evolutionStrategies that are more successful have more copies of themselves in the next roundWhich matters, since whether a strategy works depends in part on what everyone else is doing.Some more complicated strategies have succeeded in later versions of the tournament,but tit for tat does quite wellHis book is The Evolution of Cooperation
24 Threats, bluffs, commitment strategies A nuisance suit.Plaintiff's cost is $100,000, as is defendant's cost1% chance that plaintiff wins and is awarded $5,000,000What happens?How might each side try to improve the outcomeAirline hijacking, with hostagesThe hijackers want to be flown to Cuba (say)Clearly that costs less than any serious risk of having the plane wrecked and/or passengers killedShould the airline give in?When is a commitment strategy believable?Suppose a criminal tries to commit to never plea bargaining?On the theory that that makes convicting him more costly than convicting other criminalsSo he will be let go, or not arrested
25 Game Theory: Review Problem: Strategic behavior What I should do depends on what you doAnd vice versaAbstract representations of gamesDecision tree for sequential gamesStrategy matrix for all games (2D for 2 player)Solution conceptsSubgame perfect equilibriumDominanceVon Neumann solution to 2 player gameNash equilibriumVon Neumann solution to many player game
26 Subgame perfect equilibrium Treat the final choice (subgame) as a decision theory problemThe solution to which is obviousSo plug it inContinue right to left on the decision treeAssumes no way of committing andNo coalition formationIn the real world, A might pay B not to take what would otherwise be his ideal choice--because that will change what C does in a way that benefits A.One criminal bribing another to keep his mouth shut, for instanceBut it does provide a simple way of extending the decision theory approachTo give an unambiguous answerIn at least some situationsConsider our basketball player problem
27 Dominant StrategyEach player has a best choice, whatever the other doesMight not exist in two sensesIf I know you are doing X, I do Y—and if you know I am doing Y, you do X. Nash equilibrium. Driving on the right. The outcome may not be unique, but it is stable.If I know you are doing X, I do Y—and if you know I am doing Y, you don't do X. Unstable. Scissors/paper/stone.
28 Nash EquilibriumBy freezing all the other players while you decide, we reduce it to decision theory for each player--given what the rest are doingWe then look for a collection of choices that are consistent with each otherMeaning that each person is doing the best he can for himselfGiven what everyone else is doingThis assumes away all coalitionsit doesn't allow for two ore more people simultaneously shifting their strategy in a way that benefits bothLike my two escaping prisonersIt ignores the problem of defining “freezing other players”Their alternatives partly depend on what you are doingSo “freezing” really means “adjusting in a particular way”For instance, freezing prize, varying quantity, or vice versaIt also ignores the problem of how to get to that solutionOne could imagine a real world situation whereA adjusts to B and CWhich changes B's best strategy, so he adjustsWhich changes C and A's best strategies …Forever …A lot of economics is like this--find the equilibrium, ignore the dynamics that get you there
29 Von Neumann Solution aka minimax aka saddlepoint aka ….? It tells each player how to figure out what to doA value VA strategy for one player that guarantees winning at least VAnd for the other that guarantees losing at most VDescribes the outcome if each follows those instructionsBut it applies only to two person fixed sum games.
30 VN Solution to Many Player Game Outcome--how much each player ends up withDominance: Outcome A dominates B if there is some group of players, all of whom do better under A (end up with more) and who, by working together, can get A for themselvesA solution is a set of outcomes none of which dominates another, such that every outcome not in the solution is dominated by one in the solutionConsider, to get some flavor of this, three player majority voteA dollar is to be divided among Ann, Bill and Charles by majority vote.Ann and Bill propose (.5,.5,0)--they split the dollar, leaving Charles with nothingCharles proposes (.6,0,.4). Ann and Charles both prefer it, to it beats the first proposal, but …Bill proposes (0, .5, .5), which beats that …And so around we go.One solution is the set: (.5,.5,0), (0, .5, .5), (.5,0,.5)
31 Schelling aka Focal Point Two people have to coordinate without communicatingEither can’t communicate (students to meet)Or can’t believe what each says (bargaining)They look for a unique outcomeBecause the alternative is choosing among many outcomesWhich is worse than thatWhile the bank robbers are haggling the cops may show upBut “unique outcome” is a factNot about nature butAbout how people thinkWhich implies thatYou might improve the outcome by how you frame the decisionCoordination may break down on cultural boundariesBecause people frame decisions differentlyHence each thinks the other is unreasonably refusing the obvious compromise.
32 Conclusion Game theory is helpful as a way of thinking “Since I know his final choice will be, I should …”Commitment strategiesPrisoner’s dilemmas and how to avoid themMixed strategies where you don’t want the opponent to know what you will doNash equilibriumSchelling pointsBut it doesn’t provide a rigorous answerTo either the general question of what you should doIn the context of strategic behaviorOr of what people will do
33 Insurance Risk Aversion Moral Hazard Adverse selection I prefer a certainty of paying $1000 to a .001 chance of $1,000,000Why?Moral HazardSince my factory has fire insurance for 100%Why should I waste money on a sprinkler system?Adverse selectionSomeone comes running into your office“I want a million dollars in life insurance, right now”Do you sell it to him?Doesn’t the same problem exist for everyone who wants to buy?Buying signals that you think you are likely to collectI.e. a bad riskSo price insurance assuming you are selling to bad risksWhich means it’s a lousy deal for good risksSo good risks don’t get insuredeven if they would be willing to pay a priceAt which it is worth insuring them
34 Risk Aversion The standard explanation for insurance By pooling risks, we reduce riskI have a .001 chance of my $1,000,000 house burning downA million of us will have just about 1000 houses burn down each yearFor an average cost of $1000/person/yearWhy do I prefer to reduce the risk?The more money I have, the less each additional dollar is worth to meWith $20,000/year, I buy very important thingsWith $200,000/year, the last dollar goes for something much less importantSo I want to transfer moneyTo futures where I have little, because my house burned downFrom futures where I have lots--house didn’t burn
35 “Risk Aversion” a Misleading Term Additional dollars are probably worth less to me the more I haveIt doesn’t follow that (say) additional years of life areWithout the risky operation I live fifteen yearsIf it succeeds I live thirty, but …Half the time the operation kills the patientAnd I always wanted to have childrenSo really “risk aversion in money”Aka “declining marginal utility of income.”
36 Moral hazardI have a ten million dollar factory and am worried about fireIf I can take ten thousand dollar precaution that reduces the risk by 1% this year, I will—(.01x$10,000,000=$100,000>$10,000)But if the precaution costs a million, I won't.insure my factory for $9,000,000It is still worth taking a precaution that reduces the chance of fire by %1But only if it costs less than …?Of course, the price of the insurance will take account of the fact that I can be expected to take fewer precautions:Before I was insured, the chance of the factory burning down was 5%So insurance should have cost me about $450,000/year, but …Insurance company knows that if insured I will be less carefulRaising the probability to (say) 10%, and the price to $900,000There is a net loss here—precautions worth taking that are not getting taken, because I pay for them but the gain goes mostly to the insurance company.
37 Solutions?Require precautions (signs in car repair shops—no customers allowed in, mandated sprinkler systems)The insurance company gives you a lower rate if you take the precautionsOnly works for observable precautionsMake insurance only cover fires not due to your failure to take precautions (again, if observable)Only insure for (say) 50% of valueThere are still precautions you should take and don’tBut you take the ones that matter mostI.e. the ones where benefit is at least twice costSo moral hazard remains, but is cost is reducedOf course, you also now have more risk to bear
38 A PuzzleThe value of a dollar changes a lot between $20,000/year and $200,000/yearBut very little between $200,000 and $200,100So why do people insure against small losses?Service contract on a washing machineEven a toaster!Medical insurance that covers routine checkupsFilling cavities, and the like
39 Is Moral Hazard a Bug or a Feature? Big company, many factories, they insureWhy? They shouldn't be risk averseSince they can spread the loss across their factories.Consider the employee running one factory without insuranceHe can spend nothing, have 3% chance of a fireOr spend $100,000, have 1%--and make $100,000 less/year for the companyWhich is it in his interest to do?Insure the factory to transfer cost to insurance companyWhich then insists on a sprinkler systemMakes other rulesIs more competent than the company to prevent the fire!
40 Put Incentive Where It Does the Most Good Insurance transfers loss from owner to the insurance companySometimes the owner is the one best able to prevent the lossIn which case moral hazard is a cost of insuranceTo be weighed against risk spreading gainSometimes the insurance company is best ableIn which case “moral hazard” is the objectiveSears knows more about getting their washing machines fixed than I doSo I buy a service contract to transfer the decision to themSometimes each party has precutions it is best atSo coinsurance--say 50%--gives each an incentive to takeThose precautions that have a high payoff
41 Health Insurance If intended as risk spreading But maybe it’s intended Should be a large deductibleSo I pay for all minor thingsGiving me an incentive to keep costs downSince I am paying themBut cover virtually 100% of rare high ticket itemsIf my life is at stake, I want itBut I don’t want to risk paying even 10% of a million dollar procedureBut maybe it’s intendedTo transfer to the insurance companyThe incentive to find me a good doctorNegotiate a good priceRobin Hanson’s versionI decide how much my life is worthI buy that much life insurance, from a company that alsoMakes and pays for my medical decisionsAnd now has the right incentive to keep me alive
42 Adverse Selection The problem: The market for lemons AssumptionsUsed car in good condition worth $10,000 to buyer, $8000 to sellerLemon worth $5,000, $4,000Half the cars are creampuffs, half lemonsFirst try:Buyers figure average used car is worth $7,500 to them, $6,000 to seller, so offer something in betweenWhat happens?What is the final result?How might you avoid this problem—due to asymmetric informationMake the information symmetric—inspect the car. Or …Transfer the risk to the party with the information—seller insures the carWhat problems does the latter solution raise?
43 Plea Bargaining A student raised the following question: Suppose we include adverse selection in our analysis of plea bargainingWhat does the D.A. signal by offering a deal?What does the defendant signal by accepting?Which subset of defendants end up going to trial?
44 Why insurance matters?Most of you won’t be working for insurance companiesOr even negotiating contracts with themBut the analysis of insurance will be importantAlmost any contract is in part insuranceDo you pay salesmen by the month or by the sale?Is your house built for a fixed price, or cost+?Do you take the case for a fixed price, contingency fee, or hourly charge?