1. Achieving Network Optima Using Stackelberg Routing Strategies Yannis A. Korilis, Member, IEEE Aurel A. Lazar, Fellow, IEEE & Ariel Orda, Member IEEE IEEE/ACM transactions on networking, vol. 5, No. 1, February 1997 Sanjeev Kohli EE 228A

2. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

3. Non-cooperative Networks

4.  Users take control decisions individually to max own performance

5. Non-cooperative Networks  Users take control decisions individually to max own performance  Similar to non cooperative games

6. Non-cooperative Networks  Users take control decisions individually to max own performance  Similar to non cooperative games  Operating points of such networks are determined by Nash equilibria

7. Non-cooperative Networks  Users take control decisions individually to max own performance  Similar to non cooperative games  Operating points of such networks are determined by Nash equilibria  Nash Equilibria – Unilateral deviation doesn’t help any user

8. Non-cooperative Networks  Users take control decisions individually to max own performance  Similar to non cooperative games  Operating points of such networks are determined by Nash equilibria  Nash Equilibria – Unilateral deviation doesn’t help any user  Inefficient, leads to sub optimal performance

9. Non-cooperative Networks  Users take control decisions individually to max own performance  Similar to non cooperative games  Operating points of such networks are determined by Nash equilibria  Nash Equilibria – Unilateral deviation doesn’t help any user  Inefficient, leads to sub optimal performance  Better solution needed !

10. Network Manager

11.  Architects the n/w to achieve efficient equilibria

12. Network Manager  Architects the n/w to achieve efficient equilibria  Run time phase

13. Network Manager  Architects the n/w to achieve efficient equilibria  Run time phase  Awareness of users behavior

14. Network Manager  Architects the n/w to achieve efficient equilibria  Run time phase  Awareness of users behavior  Aims to improve overall system performance through maximally efficient strategies

15. Network Manager  Architects the n/w to achieve efficient equilibria  Run time phase  Awareness of users behavior  Aims to improve overall system performance through maximally efficient strategies  Maximally efficient strategy  Optimizes overall performance

16. Network Manager  Architects the n/w to achieve efficient equilibria  Run time phase  Awareness of users behavior  Aims to improve overall system performance through maximally efficient strategies  Maximally efficient strategy  Optimizes overall performance  Individual users are well off at this operating point [Pareto Efficient]

17. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

18. Overview of this approach

19.  Total flow: Flow of users + Flow of manager

20. Overview of this approach  Total flow: Flow of users + Flow of manager  Example of manager’s flow Traffic generated by signaling/control mechanism

21. Overview of this approach  Total flow: Flow of users + Flow of manager  Example of manager’s flow Traffic generated by signaling/control mechanism Users traffic that belongs to virtual networks

22. Overview of this approach  Total flow: Flow of users + Flow of manager  Example of manager’s flow Traffic generated by signaling/control mechanism Users traffic that belongs to virtual networks  Manager optimizes system performance by controlling its portion of flow

23. Overview of this approach  Total flow: Flow of users + Flow of manager  Example of manager’s flow Traffic generated by signaling/control mechanism User traffic that belongs to virtual networks  Manager optimizes system performance by controlling its portion of flow  Investigates manager’s role using routing as a control paradigm

24. Non Cooperative Routing Scenario  IPv4/IPv6 allow source routing User determines the path its flow follows from source- destination

25. Goal of Manager  Optimize overall network performance according to some system wide efficiency criterion Capability of Manager  It is aware of non cooperative behavior of users and performs its routing based on this information

26. Central Idea

27.  Manager can predict user responses to its routing strategies

28. Central Idea  Manager can predict user responses to its routing strategies  Allows manager to choose a strategy that leads of optimal operating point

29. Central Idea  Manager can predict user responses to its routing strategies  Allows manager to choose a strategy that leads of optimal operating point  Example of Leader-Follower Game [Stackelberg]

30. MAN Org1 Org2 Org n VP’s k User 1 User 2 User 3 User p

31. Need to derive  A necessary and sufficient condition that guarantees that the manager can enforce an equilibrium that coincides with the network optimum  Above condition requires –  Manager’s flow Control > Threshold

32. Need to derive  A necessary and sufficient condition that guarantees that the manager can enforce an equilibrium that coincides with the network optimum  Above condition requires –  Manager’s flow Control > Threshold If the above criterion is met, we can show that the maximally efficient strategy of manager is unique and we will specify its structure explicitly

33. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

34. Model and Problem Formulation  User set I = {1,…..,I}  Communication Links L= {1,.....,L} SourceDestination 1 2 L

35. Model and Problem Formulation (contd)  Manager is referred at user 0  I 0 = I U {0}  c l = capacity of link l  c = (c 1,….c L ) : capacity configuration  C =  l  L c l : total capacity of the system of parallel links  c 1 >= c 2 >= …. >= c L  Each i  I 0 has a throughput demand r i > 0  r 1 >= r 2 >= …. >= r I r =  i  I r i  R = r + r 0  Demand is less than capacity of links  R < C

36. Model and Problem Formulation (contd)  User i  I 0 splits its demand r i over the set of parallel links to send its flow  Expected flow of user i on link l is f l i  Routing strategy of user i  f i = (f 1 i,….f L i )  Strategy space of user i  F i = {f i  IR L : 0 <= f l i <= c l, l  L;  l  L f l i = r i }  Routing strategy profile  f = {f 0, f 1,….,f I )  System strategy space  F =  i  I o F i

37. Model and Problem Formulation (contd)  Cost function quantifying GoS of user i’s flow is J i : F  IR i  I 0  Cost of user i under strategy profile f is J i (f)  J i (f) =  l  L f l i T l (f l ); T l (f l ) is the average delay on link l, depends only on the total flow f l =  i  I o f l i on that link  T l (f l ) = (c l - f l ) -1,f l < c l = , f l >= c l  Total cost J(f) =  i  I o J i (f) =  l  L f l / (c l - f l )  Higher cost  lower GoS provided to the user’s flow, higher average delay

38. Model and Problem Formulation (contd)  is a convex function of (f 1, …, f L )  a unique link flow configuration exists – min cost (f 1 *,….f L * ) ;  Above is solution to classical routing opt problem, routing of all flow (users+manager) is centrally controlled; referred to as network optimum.

39. Kuhn – Tucker Optimality conditions  (f 1 *,….f L * ) is the network optimum if and only if there exists a Lagrange Multiplier, such that for every link l  L

40. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

41. Non cooperative users

42.  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)

43. Non cooperative users  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)  This minimization depends on strategies of the manager and other users, described by strategy profile f -i = (f 0, f 1,… f i-1, f i+1,… f I )

44. Non cooperative users  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)  This minimization depends on strategies of the manager and other users, described by strategy profile f -i = (f 0, f 1,… f i-1, f i+1,… f I )  R outing strategy of manger is FIXED  f 0

45. Non cooperative users  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)  This minimization depends on strategies of the manager and other users, described by strategy profile f -i = (f 0, f 1,… f i-1, f i+1,… f I )  R outing strategy of manger is FIXED  f 0  Each user adjusts its strategy to other users actions

46. Non cooperative users  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)  This minimization depends on strategies of the manager and other users, described by strategy profile f -i = (f 0, f 1,… f i-1, f i+1,… f I )  R outing strategy of manger is FIXED  f 0  Each user adjusts its strategy to other users actions  Can be modeled as a non cooperative game, any operating point is Nash Equilibrium; dependent on f 0 !

47. Non cooperative users  Each user tries to find a routing strategy f i  F i that minimizes its cost J i (average time delay)  This minimization depends on strategies of the manager and other users, described by strategy profile f -i = (f 0, f 1,… f i-1, f i+1,… f I )  R outing strategy of manger is FIXED  f 0  Each user adjusts its strategy to other users actions  Can be modeled as a non cooperative game, any operating point is Nash Equilibrium; dependent on f 0 !  From users view point, manager reduces capacity on each link l by f l 0, the system reduces to a set of parallel links with capacity configuration c – f 0  has a unique Nash Equilibrium f 0  f -0 ……. N 0 (f 0 )

48. Non cooperative users  For a given strategy profile f -i of other users in I 0, the cost of i, J i (f) =  l  L f l i T l (f l ), is a convex fn of its strategy f i, hence the following min problem has a unique solution

49. Kuhn – Tucker Optimality conditions  f i is the optimal response of user i if and only if there exists a (Lagrange Multiplier), such that for every link l  L, we have

50. Non cooperative users  f -0  F -0 is a Nash Equilibrium of the self optimizing users induced by strategy f 0 of the manger.  The function N 0 : F 0  F -0 that assigns the induced equilibrium of the user routing game (to each strategy of the manger) is called the Nash Mapping. It is continuous.

51. Role of the Manager  It has knowledge of non cooperative behavior of users; determines the Nash Equilibrium N 0 (f 0 ) induced by any routing strategy it f 0 chosen by him  Acts as Stackelberg leader, that imposes its strategy on the self optimizing users that behave as followers  Aims to optimize the overall network performance, plays a social rather than selfish role  To find f 0 such that if f -0 = N 0 (f 0 ), then  i  I o f l i = f l * for all l This f 0 is called maximally efficient strategy of manager It is Pareto efficient !

52. Outline of Results  In case of a single user, the manager can always enforce network optimum; its MES is specified explicitly  In case of any no of users, the manager can enforce the network optimum iff its demand is higher that some threshold r 0, in which case the MES is specified explicitly  r 0 is feasible if total demand of users plus r 0 is less than C  It is easy for manager to optimize heavily loaded networks as r 0 is small  As the no of user increases, threshold increases i.e. harder for manager to enforce network optimum  The higher the difference in throughput demands of any two users, the easier it is for manager to enforce network optimum

53.  Network optimum: (f 1 *,….f L * )  Flow on link l, f l * is decreasing in link no l  L  There exists some link L *, such that f l * > 0 for l L * ; L * is determined by (from [1] & [2]), where and G 1 =0, G L+1 =  l n=1 c n = C  c l >= c l+1  G l <= G l+1

54.  Using Lagrange Multiplier’s equations, we get,  Network Optimum is given by [2]

55.  Best reply f i of user i  I 0 to the strategies of manager and other users, described by f -i, can be determined as network optimum for a system of parallel links with capacity configuration (c 1 i,…, c L i )  Assuming c l i >= c l+1 i, l=1,…,L-1 the flow f l i is decreasing in the link no l  L  There exists some link L i, such that f l i > 0 for l L i ; The threshold L i is determined by

56.  Best reply f i of user i to strategy profile f -i of the other users in I 0 is given by  Best reply doesn’t depend on detailed description of f -i but only on residual capacity c l i seen by user on every link l  L  In practice, residual capacity info can be acquired by measuring the link delays using an appropriate estimation technique

57. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

58. Single Follower Stackelberg Routing Game

59. In this game, there exists a MES of the manager then it is unique and is given by

60. Single Follower Stackelberg Routing Game  The best reply f 1 of the follower is  Therefore, {1,…,L 1 } is the set of links over which the follower sends its flow when manager implements f 0.  For manager: Send flow f l * on every link l that will not receive any flow from the follower Split the rest of its flow among the links that will receive user flow proportional to their capacities

61. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

62. Multi Follower Stackelberg Routing Game

63.  An arbitrary number I of self optimizing users share the system of parallel links

64. Multi Follower Stackelberg Routing Game  An arbitrary number I of self optimizing users share the system of parallel links  Maximally Efficient Strategy of manager (if it exists) and the corresponding Nash Equilibrium of non cooperative users is:

65. Multi Follower Stackelberg Routing Game  Equilibrium strategy f i of user i  I is described by  If a MES exists, then the induced Nash equilibrium of the followers has precisely the same structure with the best reply follower in the single follower case

66. Remarks - M F Stackelberg Routing Game

67.  {1,…., L i } is the set of links that receive flow from follower i  I

68. Remarks - M F Stackelberg Routing Game  {1,…., L i } is the set of links that receive flow from follower i  I  I l is the set of followers that send flow on link l. Since H 1 = 0 < r i, i  I, all users send flow on link 1  I 1 = I

69. Remarks - M F Stackelberg Routing Game  {1,…., L i } is the set of links that receive flow from follower i  I  I l is the set of followers that send flow on link l. Since H 1 = 0 < r i, i  I, all users send flow on link 1  I 1 = I  For f 0 to be admissible, f l 0 >= 0, for all l  L

70. Remarks - M F Stackelberg Routing Game  {1,…., L i } is the set of links that receive flow from follower i  I  I l is the set of followers that send flow on link l. Since H 1 = 0 < r i, i  I, all users send flow on link 1  I 1 = I  For f 0 to be admissible, f l 0 >= 0, for all l  L  If f l 0 < 0  f l-1 0 < 0

71. Remarks - M F Stackelberg Routing Game  {1,…., L i } is the set of links that receive flow from follower i  I  I l is the set of followers that send flow on link l. Since H 1 = 0 < r i, i  I, all users send flow on link 1  I 1 = I  For f 0 to be admissible, f l 0 >= 0, for all l  L  If f l 0 < 0  f l-1 0 < 0  Admissible condition reduces to f 1 0 >= 0

72. Remarks - M F Stackelberg Routing Game  {1,…., L i } is the set of links that receive flow from follower i  I  I l is the set of followers that send flow on link l. Since H 1 = 0 < r i, i  I, all users send flow on link 1  I 1 = I  For f 0 to be admissible, f l 0 >= 0, for all l  L  If f l 0 < 0  f l-1 0 < 0  Admissible condition reduces to f 1 0 >= 0  f 1 0 is an increasing function of the throughput demand r 0 of leader, r 0  [0, C - r] ………. [3]

73. Theorem  There exists some r 0, with 0 < r 0 < C – r, such that the leader in multi follower Stackelberg routing game can enforce the network optimum if and only if its throughput demand r 0 satisfies r 0 < r 0 < C – r. The maximally efficient strategy of leader is given by

74. Presentation Outline  Introduction to non cooperative networks  Overview of approach  Model and Problem Formulation  Non cooperative User & Manager  Single Follower Stackelberg Routing game  Multi Follower Stackelberg Routing game  Issues

75. Properties of Leader Threshold r 0

76.  r 0 of the leader is a unique solution of the equation “f 1 0 (r 0 ) = 0” in r 0  [0, C - r] Properties of Leader Threshold r 0

77.  r 0 of the leader is a unique solution of the equation “f 1 0 (r 0 ) = 0” in r 0  [0, C - r]  When r  C, r 0  0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum Properties of Leader Threshold r 0

78.  r 0 of the leader is a unique solution of the equation “f 1 0 (r 0 ) = 0” in r 0  [0, C - r]  When r  C, r 0  0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum  With throughput demand r fixed, the leader threshold r 0 increases with increase in no of users. Properties of Leader Threshold r 0

79.  r 0 of the leader is a unique solution of the equation “f 1 0 (r 0 ) = 0” in r 0  [0, C - r]  When r  C, r 0  0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum  With throughput demand r fixed, the leader threshold r 0 increases with increase in no of users.  Leader threshold r 0 decreases with increase in difference in user demands Properties of Leader Threshold r 0

80. Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by I identical followers with total demand r

81. Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager r 0 = r 0

82. Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager r 0 = r 0

83. Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager r 0 = r 0

84. Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager r 0 = r 0

85. Scalability

86.  To determine maximally efficient strategy, manager needs throughput demand r i of every user. Scalability

87.  To determine maximally efficient strategy, manager needs throughput demand r i of every user.  In many networks, user declare average rate r i during negotiation phase Scalability

88.  To determine maximally efficient strategy, manager needs throughput demand r i of every user.  In many networks, user declare average rate r i during negotiation phase  Alternatively, the manager can estimate average rates by monitoring the behavior of users Scalability

89.  To determine maximally efficient strategy, manager needs throughput demand r i of every user.  In many networks, user declare average rate r i during negotiation phase  Alternatively, the manager can estimate average rates by monitoring the behavior of users  Manager can adjust its strategy to maximally efficient one whenever a user departs or a new one joins the network Scalability

90.  To determine maximally efficient strategy, manager needs throughput demand r i of every user.  In many networks, user declare average rate r i during negotiation phase  Alternatively, the manager can estimate average rates by monitoring the behavior of users  Manager can adjust its strategy to maximally efficient one whenever a user departs or a new one joins the network  User not necessarily mean a single user, it can be a group of users joining the network as an organization. It also reduces threshold r 0 Scalability

91. References [1]A. Orda, R. Rom, and N. Shimkin, “Competitive routing in multi- user communication networks,” IEEE/ACM Trans. Networking, vol. 1, pp. 510-521, Oct. 1993. [2]Y.A. Korilis, A.A. Lazar, and A. Orda, “Capacity allocation under non cooperative routing,” IEEE Trans. Automat. Contr. [3] Y.A. Korilis, A.A. Lazar, and A. Orda, “Achieving network optima using Stackelberg routing strategies,” Center for Telecommunications Research, Columbia University, NY, CTR Tech. Rep. 384-94-31, 1994.

92. THANK YOU