1. Frank Cowell: Microeconomics Game Theory: Dynamic MICROECONOMICS Principles and Analysis Frank Cowell November 2006 Almost essential Game Theory: Strategy and Equilibrium Almost essential Game Theory: Strategy and Equilibrium Prerequisites

2. Frank Cowell: Microeconomics Overview... Game and subgame Equilibrium Issues Applications Game Theory: Dynamic Mapping the temporal structure of games

3. Frank Cowell: Microeconomics Time Why introduce “time” into model of a game? Why introduce “time” into model of a game? Without it some concepts meaningless… Without it some concepts meaningless…  can we really speak about reactions?  an equilibrium path?  threats? “Time” involves structuring economic decisions “Time” involves structuring economic decisions  model the sequence of decision making  consideration of rôle of information in that sequence Be careful to distinguish strategies and actions Be careful to distinguish strategies and actions  see this in a simple setting

4. Frank Cowell: Microeconomics A simple game   Stage 1: Alf’s decision   Stage 2: Bill’s decision following [LEFT]   The payoffs (1a,1b)(1a,1b) Alf Bill [LEFT] [RIGHT] [left][right] [left][right]   Stage 2: Bill’s decision following [RIGHT] (2a,2b)(2a,2b) (3a,3b)(3a,3b) (4a,4b)(4a,4b)

5. Frank Cowell: Microeconomics [left-right] (4a,4b)(4a,4b) (1a,1b)(1a,1b) [left-left] (3a,3b)(3a,3b) (1a,1b)(1a,1b) (4a,4b)(4a,4b)(3a,3b)(3a,3b)[RIGHT] (2a,2b)(2a,2b)(2a,2b)(2a,2b)Alf Bill [right-left][right-right] A simple game: Normal form   Alf has two strategies   Bill has four strategies   The payoffs [LEFT] Always play [right] whatever Alf chooses Always play [left] whatever Alf chooses Play [left] if Alf plays [LEFT]. Play [right] if Alf plays [RIGHT] Play [right] if Alf plays [LEFT]. Play [left] if Alf plays [RIGHT]   Alf moves first: strategy set contains just two elements   Bill moves second: strategy set contains four elements

6. Frank Cowell: Microeconomics The setting Take a game in strategic form Take a game in strategic form If each player has exactly one round of play… If each player has exactly one round of play…  game is extremely simple…  …simultaneous or sequential? Otherwise need a way of describing structure Otherwise need a way of describing structure  imagine a particular path through the tree diagram  characterise unfolding decision problem Begin with some reminders Begin with some reminders

7. Frank Cowell: Microeconomics Structure: 1 (reminders) Decision nodes Decision nodes  in the extensive form…  …represent points where a decision is made by a player Information set Information set  where is player (decision maker) located?  may not have observed some previous move in the game  knows that he is at one of a number of decision nodes  collection of these nodes is the information set

8. Frank Cowell: Microeconomics Structure: 2 (detail) Stage Stage  a particular point in the logical time sequence of the game  payoffs come after the last stage Direct successor nodes Direct successor nodes  take the decision branches (actions) that follow from node *  if the branches lead to other decision nodes at the next stage…  …then these are direct successor nodes to node * Indirect successor nodes Indirect successor nodes  repeat the above through at least one more stage…  … to get indirect successor nodes How can we use this structure? How can we use this structure?  break overall game down…  …into component games?

9. Frank Cowell: Microeconomics Subgames (1) A subgame of an extensive form game A subgame of an extensive form game  a subset of the game…  …satisfying three conditions 1. Begins with a “singleton” information set  contains a single decision node  just like start of overall game 2. Contains all the decision nodes that…  are direct or indirect successors,…  …and no others 3. If a decision node is in the subgame then  any other node in the same information set…  …is also in the subgame.

10. Frank Cowell: Microeconomics Alf [LEFT] [RIGHT] Alf [LEFT][RIGHT]  1  2  3  4  5  6  7  8  Alf [LEFT][RIGHT] Alf [LEFT] [RIGHT] [left][right] [left][right] Bill Subgames (2)   Stage 1: (Alf’)   Stage 2: (Bill)   The payoffs   Add a Stage (Alf again)...   A subgame...   Another subgame...

11. Frank Cowell: Microeconomics Alf [LEFT] [RIGHT] Alf [LEFT][RIGHT] Alf [LEFT][RIGHT] Alf [LEFT] [RIGHT] [left][right] [left][right] Bill Subgames (3)   The previous structure   Additional strategy for Alf   Ambiguity at stage 3   A subgame   Not a subgame (violates 2) [MID] Alf [LEFT] [RIGHT] [left][right] [left][right] Alf [LEFT] [RIGHT] Alf [LEFT] Alf [LEFT][RIGHT] Bill [RIGHT] [left][right] Bill Alf  1  2  3  4  5  6  7  8  9  10  11  12  Alf   Not a subgame (violates 3)

12. Frank Cowell: Microeconomics Game and subgame: lessons “Time” imposes structure on decision-making “Time” imposes structure on decision-making Representation of multistage games Representation of multistage games  requires care  distinguish actions and strategies  normal-form representation can be awkward Identifying subgames Identifying subgames  three criteria  cases with non-singleton information sets are tricky

13. Frank Cowell: Microeconomics Overview... Game and subgame Equilibrium Issues Applications Game Theory: Dynamic Concepts and method

14. Frank Cowell: Microeconomics Equilibrium Equilibrium raises issues of concept and method Equilibrium raises issues of concept and method  both need some care…  …as with the simple single-shot games Concept Concept  can we use the Nash Equilibrium again?  clearly requires careful specification of the strategy sets Method Method  a simple search technique?  but will this always work? We start with an outline of the method… We start with an outline of the method…

15. Frank Cowell: Microeconomics Backwards induction Suppose the game has N stages Suppose the game has N stages Start with stage N Start with stage N  suppose there are m decision nodes at stage N Pick an arbitrary node … Pick an arbitrary node …  suppose h is player at this stage  …determine h’s choice, conditional on arriving at that node  …note payoff to h and to every other player arising from this choice Repeat for each of the other m−1 nodes Repeat for each of the other m−1 nodes  this completely solves stage N  gives m vectors of [  1 ],…,[  m ] Re-use the values from solving stage N Re-use the values from solving stage N  gives the payoffs for a game of N−1 stages Continue on up the tree… Continue on up the tree… An example:

16. Frank Cowell: Microeconomics Backwards induction: example Examine the last stage of the 3-stage game used earlier Examine the last stage of the 3-stage game used earlier Suppose the eight payoff-levels for Alf satisfy Suppose the eight payoff-levels for Alf satisfy  υ 1 a > υ 2 a (first node)  υ 3 a > υ 4 a (second node)  υ 5 a > υ 6 a (third node)  υ 7 a > υ 8 a (fourth node) If the game had in fact reached the first node: If the game had in fact reached the first node:  obviously Alf would choose [LEFT]  so value to (Alf, Bill) of reaching first node is [υ 1 ] = (υ 1 a,υ 1 b ) Likewise the value of reaching other nodes at stage 3 is… Likewise the value of reaching other nodes at stage 3 is…  [υ 3 ] (second node)  [υ 5 ] (third node)  [υ 7 ] (fourth node) Backwards induction has reduced the 3-stage game… Backwards induction has reduced the 3-stage game…  …to a two-stage game  …with payoffs [υ 1 ], [υ 3 ], [υ 5 ], [υ 7 ]

17. Frank Cowell: Microeconomics Alf [LEFT] [RIGHT] Alf [LEFT][RIGHT] Alf [LEFT][RIGHT] Alf [LEFT] [RIGHT] [left][right] [left][right] Bill Backwards induction: diagram   3-stage game as before   Payoffs to 3-stage game   Alf would play [LEFT] at this node υ 1 a > υ 2 a υ 3 a > υ 4 a υ 5 a > υ 6 a υ 7 a > υ 8 a 11  2   3   4  55 6677 8  11   …and here  3  5577   The 2-stage game derived from the 3-stage game

18. Frank Cowell: Microeconomics Equilibrium: questions Backwards induction is a powerful method Backwards induction is a powerful method  accords with intuition  usually leads to a solution But what is the appropriate underlying concept? But what is the appropriate underlying concept? Does it find all the relevant equilibria? Does it find all the relevant equilibria? What is the role for the Nash Equilibrium (NE) concept? What is the role for the Nash Equilibrium (NE) concept? Begin with the last of these… Begin with the last of these… A simple example:

19. Frank Cowell: Microeconomics Equilibrium example (0,0)(2,1)(1,2) [LEFT] [RIGHT] [left][right] [left][right]   The extensive form   Bill’s choices in final stage   Values found through backwards induction   Alf’s choice in first stage   Backwards induction finds equilibrium payoff of 2 for Alf, 1 for Bill Bill (2,1)(1,2)   The equilibrium path Alf Bill (2,1)   But what is/are the NE here?   Look at the game in normal form…

20. Frank Cowell: Microeconomics [left-right] 1,2 0,0 [left-left] 1,2 0,0 1,2 [RIGHT] 2,1 s2as2as2as2a Alf Bill [right-left][right-right] Equilibrium example: Normal form   Alf’s two strategies   Bill’s four strategies   Payoffs s4bs4bs4bs4b s1bs1bs1bs1b s2bs2bs2bs2b s3bs3bs3bs3b [LEFT] s1as1as1as1a   Best replies to s 1 a   Best reply to s 3 b or to s 4 b Nash equilibria: (s 1 a,s 3 b ), (s 1 a,s 4 b )   Best replies to s 2 a   Best reply to s 1 b or to s 2 b (s 2 a,s 1 b ), (s 2 a,s 2 b ),

21. Frank Cowell: Microeconomics Equilibrium example: the set of NE The set of NE include the solution already found The set of NE include the solution already found   backwards induction method…   (s 1 a, s 3 b ) yields payoff (2,1)   (s 1 a, s 4 b ) yields payoff (2,1) What of the other NE? What of the other NE?   (s 2 a, s 1 b ) yields payoff (1,2)   (s 2 a, s 2 b ) yields payoff (1,2) These suggest two remarks   First, Bill’s equilibrium strategy may induce some odd behaviour   Second could such an NE be sustained in practice? We follow up each of these in turn

22. Frank Cowell: Microeconomics Equilibrium example: odd behaviour? (0,0)(2,1)(1,2) [LEFT] [RIGHT] [left][right] [left][right] Bill Alf Bill Take the Bill strategy s 1 b = [left-left] Take the Bill strategy s 1 b = [left-left]  “Play [left] whatever Alf does” If Alf plays [RIGHT] on Monday If Alf plays [RIGHT] on Monday  On Tuesday it’s sensible for Bill to play [left] But if Alf plays [LEFT] on Monday But if Alf plays [LEFT] on Monday  what should Bill do on Tuesday?  the above strategy says play [left]  but, from Tuesday’s perspective, it’s odd Given that the game reaches node * Given that the game reaches node *  Bill then does better playing [right] Yet… s 1 b is part of a NE?? Yet… s 1 b is part of a NE?? Tuesday Monday *

23. Frank Cowell: Microeconomics Equilibrium example: practicality Again consider the NE not found by backwards induction Again consider the NE not found by backwards induction  Give a payoff of 1 to Alf, 2 to Bill Could Bill “force” such a NE by a threat? Could Bill “force” such a NE by a threat?  Imagine the following pre-play conversation.  Bill: “I will play strategy [left-left] whatever you do”  Alf: “Which means?”  Bill: “To avoid getting a payoff of 0 you had better play [RIGHT]” The weakness of this is obvious The weakness of this is obvious  Suppose Alf goes ahead and plays [LEFT]  Would Bill now really carry out this threat?  After all Bill would also suffer (gets 0 instead of 1) Bill’s threat seems incredible Bill’s threat seems incredible  So the “equilibrium” that seems to rely on it is not very impressive

24. Frank Cowell: Microeconomics Equilibrium concept Some NEs are odd in the dynamic context Some NEs are odd in the dynamic context  So there’s a need to refine equilibrium concept Introduce Subgame-Perfect Nash Equilibrium (SPNE) Introduce Subgame-Perfect Nash Equilibrium (SPNE) A profile of strategies is a SPNE for a game if it A profile of strategies is a SPNE for a game if it  …is a NE  …induces actions consistent with NE in every subgame

25. Frank Cowell: Microeconomics NE and SPNE All SPNE are NE All SPNE are NE  reverse is not true  some NE that are not SPNE involve agents making threats that are not credible Definition of SPNE is demanding Definition of SPNE is demanding  it says something about all the subgames  even if some subgames do not interesting  or are unlikely to be actually reached in practice Backward induction method is useful Backward induction method is useful  but not suitable for all games with richer information sets

26. Frank Cowell: Microeconomics Equilibrium issues: summary Backwards induction provides a practical method Backwards induction provides a practical method Also reveals weakness of NE concept Also reveals weakness of NE concept Some NE may imply use of empty threats Some NE may imply use of empty threats  given a node where a move by h may damage opponent  but would cause serious damage h himself  h would not rationally make the move  threatening this move should the node be reached is unimpressive Discard these as candidates for equilibria? Discard these as candidates for equilibria? Focus just on those that satisfy subgame perfection Focus just on those that satisfy subgame perfection See how these work with two applications See how these work with two applications

27. Frank Cowell: Microeconomics Overview... Game and subgame Equilibrium Issues Applications Game Theory: Dynamic Industrial organisation (1) Market leadership Market entry

28. Frank Cowell: Microeconomics Market leadership: an output game Firm 1 (leader) gets to move first: chooses q¹ Firm 1 (leader) gets to move first: chooses q¹ Firm 2 (follower) observes q¹ and then chooses q² Firm 2 (follower) observes q¹ and then chooses q² Nash Equilibria? Nash Equilibria?  given firm 2’s reaction function χ²(∙)…  any (q¹, q²) satisfying q² = χ²(q¹) is the outcome of a NE  many such NE involve incredible threats Find SPNE by backwards induction Find SPNE by backwards induction  start with follower’s choice of q² as best response to q¹  this determines reaction function χ²(∙)  given χ² the leader's profits are p(q² + χ²(q¹))q¹ − C¹(q¹)  max this w.r.t. q¹ to give the solution

29. Frank Cowell: Microeconomics Market leadership q1q1 q2q2 2(·)2(·) 0 (q S, q S ) ll ll 1 2   Leader max profits using follower’s stage-2 response   Follower’s isoprofit curves   Follower max profits conditional on q 1   Leader’s opportunity set   Leader’s isoprofit curves   Firm 2’s reaction function gives set of NE   Stackelberg solution as SPNE

30. Frank Cowell: Microeconomics Overview... Game and subgame Equilibrium Issues Applications Game Theory: Dynamic Industrial organisation (2) Market leadership Market entry

31. Frank Cowell: Microeconomics Entry: reusing an example Take the example used to illustrate equilibrium Take the example used to illustrate equilibrium  recall the issue of non-credible threats Modify this for a model of market entry Modify this for a model of market entry  rework the basic story…  …of a monopolist facing possibility of another firm entering  will there be a fight (e.g. a price war) after entry?  should such a fight be threatened? Replace Alf with the potential entrant firm Replace Alf with the potential entrant firm  [LEFT] becomes “enter the industry”  [RIGHT] becomes “stay out” Replace Bill with the incumbent firm Replace Bill with the incumbent firm  [left] becomes “fight a potential entrant”  [right] becomes “concede to a potential entrant”

32. Frank Cowell: Microeconomics Entry: reusing an example (more) Payoffs: potential entrant firm Payoffs: potential entrant firm  if it enters and there’s a fight: 0  if stays out: 1 (profit in some alternative opportunity)  if enters and there’s no fight: 2 Payoffs: incumbent firm Payoffs: incumbent firm  if it fights an entrant: 0  if concedes entry without a fight: 1  if potential entrant stays out: 2 (monopoly profit) Use the equilibrium developed earlier Use the equilibrium developed earlier  Find the SPNE  We might guess that outcome depends on “strength” of the two firms  Let’s see…

33. Frank Cowell: Microeconomics (0,0)(2,1)(1,2) [LEFT] [RIGHT] [left][right] [left][right] Bill Alf Bill Entry example (0,0)(2,1)(1,2) [IN] [OUT] [fight][concede] [fight][concede]   The modified version   Incumbent’s choice in final stage   Remove part of final stage that makes no sense   Entrant’s choice in first stage   SPNE is clearly (IN, concede)   A threat of fighting would be incredible Inc (1,2)   The equilibrium path Ent Inc (2,1)   The original example (2,1)

34. Frank Cowell: Microeconomics Entry: modifying the example The simple result of the SPNE in this case is striking The simple result of the SPNE in this case is striking  but it rests on an assumption about the “strength” of the incumbent  suppose payoffs to the incumbent in different outcomes are altered…  specifically, suppose that it’s relatively less costly to fight  what then? Payoffs of the potential entrant just as before Payoffs of the potential entrant just as before  if it enters and there’s a fight: 0  if stays out: 1  if enters and there’s no fight: 2 Lowest two payoffs for incumbent are interchanged Lowest two payoffs for incumbent are interchanged  if it fights an entrant: 1 (maybe has an advantage on “home ground”)  if concedes entry without a fight: 0 (maybe dangerous to let newcomer establish a foothold)  if potential entrant stays out: 2 (monopoly profit) Take another look at the game and equilibrium… Take another look at the game and equilibrium…

35. Frank Cowell: Microeconomics (0,1)(2,0) [IN] [OUT] [fight][concede] (1,2) Inc (1,2) (0,1) Ent Entry example (revised)   Incumbent’s choice in final stage   Entrant’s choice in first stage   The equilibrium path is trivial   SPNE involves potential entrant choosing [OUT]   The example revised

36. Frank Cowell: Microeconomics Entry model: development Approach has been inflexible Approach has been inflexible  relative strength of the firms are just hardwired into the payoffs  can we get more economic insight? What if the rules of the game were amended? What if the rules of the game were amended?  could an incumbent make credible threats? Introduce a “commitment device” Introduce a “commitment device”  example of this is where a firm incurs sunk costs  the firm spends on an investment with no resale value A simple version of the commitment idea A simple version of the commitment idea  introduce an extra stage at beginning of the game  incumbent might carry out investment that costs k  advertising? First, generalise the example:

37. Frank Cowell: Microeconomics Entry deterrence: two subgames [FIGHT][CONCEDE] 1 [In][Out] 2 (  F,0) (  J,  J ) (  M,  ) _   Payoffs if there had been pre-play investment   Firm 2 chooses whether to enter   Firm 1 chooses whether to fight   Π M : monopoly profit for incumbent   Π > 0: reservation profit for challenger   Π J : profit for each if they split the market   Π F : incumbent’s profit if there’s a fight (  J – k,  J ) (  M – k,  ) _   Investment cost k hits incumbent’s profits at each stage Now fit the two subgames together

38. Frank Cowell: Microeconomics Entry deterrence: full model [FIGHT`][CONCEDE] 1 1 1 [NOT INVEST] [In][Out] [In][Out] 2 [INVEST] [FIGHT`][CONCEDE] (  F,0) (  J,  J )(  J – k,  J ) (  M,  ) _ (  M – k,  ) _ (  F,0)   Firm 1 chooses whether to invest   Firm 2 chooses whether to enter 2   Firm 1 chooses whether to fight

39. Frank Cowell: Microeconomics Entry deterrence: equilibrium Suppose the incumbent has committed to investment: Suppose the incumbent has committed to investment:  if challenger enters…  it’s more profitable for incumbent to fight than concede if  Π F > Π J – k Should the incumbent precommit to investment? Should the incumbent precommit to investment?  it pays to do this rather than just allow the no-investment subgame if …  …profit from deterrence exceeds that available without investment:  Π M – k > Π J The SPNE is (INVEST, out) if: The SPNE is (INVEST, out) if:  both Π F > Π J – k  and Π M – k > Π J.  i.e. if k satisfies Π J – Π F < k < Π M – Π J  in this case deterrence “works” So it may be impossible for the incumbent to deter entry So it may be impossible for the incumbent to deter entry  in this case if 2Π J > Π M + Π F …  …then there is no k that will work

40. Frank Cowell: Microeconomics Summary New concepts New concepts  Subgame  Subgame-perfect Nash Equilibrium  Backwards induction  Threats and credibility  Commitment What next? What next?  Extension of time idea…  …repeated games