1. Introduction to Game Theory application to networks Joy Ghosh CSE 716, CSE@UB 25 th April, 2003

2. What is Game Theory ? Study of problems of conflict and cooperation amongst independent decision makers Formal way of analyzing interactions among a group of rational agents who behave strategically Games of Strategy rather than Games of Chance! Ingredients: Players / decision makers Choices / feasible actions / pure strategies Payoffs / benefits / utilities Preferences to payoffs

3. Some basic concepts Group – Any game consisting of more than one player with single player the game becomes a decision problem! Interaction – Actions of one affects the other else it would become simple sequence of independent decisions Strategic – Players account for interdependence Rationality – Players consistently opt for best choices Common Knowledge All players know that all players are rational Equilibrium – a point of best shared interest for all

4. Classification of Problems Static vs. Dynamic In Dynamic problems the sequence of choices are relevant Cooperative vs. Non-cooperative In non-cooperative games players watch out for their own interests. In cooperative games, players form coalitions with shared objectives.

5. Decision Theory under Certainty Decision problem (A, ≤) Finite set of outcomes A = {a 1, a 2, …. a n } Preference relation ‘≤’ : a ≤ b => ‘b’ is at least as good as ‘a’ Completeness – for all a, b in A, either a≤b, or b≤a Transitivity – if a≤b and b≤c, then a≤c Utility function u: A  R (consist with preference relation) For all a,b in A, u(a) <= u(b) iff a ≤ b Rational decision maker tries to maximize utility Choose outcome a* in A s.t. for all a in A, a ≤ a*

6. Decision Theory under Uncertainty Lottery L = {(a 1, p 1 ), (a 2, p 2 ), …… (a n, p n )} Σ iεn p i = 1, 0 ≤ p i ≤ 1 Outcome a i occurs with probability p i Infinitely many different possible lotteries Large number of lottery comparisons Preference relation unobservable “Under additional restrictions on preferences over lotteries there exists a utility function over outcomes such that the expected utility of a lottery provides a consistent ranking of all lotteries” - John von Neumann and Oscar Morgenstern u(L) = Σ iεn u(a i ).p i

7. Allais Paradox Lottery A A1 (sure win of 3000) vs. A2 (80% chance to win 4000) A1 strictly preferred to A2 Lottery B B1 (90% chance to win 3000) vs. B2 (70% chance to win 4000) B1 might still be preferred to B2 Lottery C C1 (25% chance to win 3000) vs. C2 (20% chance to win 4000) Most people start preferring C2 over C1 even though these two lotteries are variations of the 1 st pair in Lottery A

8. Game Theory – multi agent decision problem A normal (strategic) form game G consists of: A finite set of agents D = {1, 2, ….. N} Strategy sets S 1, S 2,... S N = set of feasible actions for agents Strategy profile S = S 1 x S 2 x... x S N Payoff function u i : S  R (i = 1, 2, …. N) NOTE: The preference an agent has is to the outcome and not to the individual action

9. Some standard games in normal form Matching Pennies Row: gains if pennies match Col: gains if there is no match Tough vs. Chicken A game of head-on collision

10. Iterated Deletion of Dominated Strategies Common Knowledge assumptions about other people’s rational behavior Some more definitions: S -i = S 1 x S 2.... x S (i-1) x S (i+1).... x S N (strategy sets of others) utility function of player i for a given pure strategy: u i (s  S) = u i (s i, s -i ) Belief  i of agent i = probability distribution over S -i for pure strategies the probability distribution is a point distribution Player i is rational with beliefs  i if: s i  arg max  s -i  S -i u i (s’ i, s -i ).  i (s -i ) for all s’ i  S i Note: as  i gets fixed, player i faces a simple decision problem

11. Dominated Strategies Strongly Dominated si  Si is strictly dominated if:  s’i  Si s.t. ui(s’i, s-i) > ui(si, s-i) for all s-i  S-i Weakly Dominated if the inequality is weak (  ) for all s-i  S-i, and strong (>) for at least one Rational players do not play strongly dominated strategy

12. Iterated Dominance (deletion) With common knowledge about rationality of players U,L is the outcome M is strictly dominated by L. Rational column player ignores M If row player knows column player is rational, he will ignore D If column player knows the above, then he will choose L

13. Iterated Dominance – Formal Definition The game is solvable by pure strategy iterated strict dominance only if S S contains a single strategy profile

14. Does the order of elimination matter? In games that are solvable by iterated dominance, the speed and order or elimination doesn’t matter. This is however not true for weakly dominated strategies. Deletion Sequence #1: T, L - (2,1) is the playoff Deletion Sequence #2: B, R - (1,1) is the playoff

15. Nash Equilibrium for pure strategy No incentive for a player to deviate from his best response to his/her belief about other player’s strategy U,L was the NE in the example of strongly dominated strategies Definitions: A strategy profile s* is a pure strategy Nash equilibrium of G iff u i (s i *, s -i *) ≥ u i (s i, s -i *) for all players i and for all s i  S i A pure strategy NE is strict if the inequality is strict There can be multiple Nash equilibria for a particular G Two people trying to meet at one out of 2 places (NY Game!)

16. Do pure strategies always work? Most games are not solvable by dominance Coordination game, zero-sum game Penny matching Game Whatever pure strategy one player chooses, the other can win by choosing a better strategy Players have to consider mixed strategies

17. Mixed strategies - definitions Mixed strategy  i for player i is a probability distribution over his strategy space S i  i : S i  R + s.t.  s i  S i  i (s i ) = 1  i is the set of probability distributions on S i  =  1 x  2 x … x  N Player i’s expected payoff with mixed strategies u i (  i,  -i ) =  s i, s -i u i (s i, s -i )  i (s i )  -i (s -i )

18. Mixed strategies – more definitions Mixed strategy NE of G is a  *   such that: u i (  i *,  -i *) ≥ u i (  i,  -i *) for all i and for all  i   i In a finite game, support of a mixed strategy  i: supp (  i ) = { s i  S i |  i (s i ) > 0 } Proposition if  i * is a mixed strategy NE and s i ’, s i ’’  supp (  i *), then u i (s i ’,  -i *) = u i (s i ’’,  -i *)

19. Proof of previous proposition

20. A mixed strategy example game There is no pure strategy NE Row plays U with probability  Column plays L with probability  Players need to be indifferent to their choice of strategies: u 1 (U,  2 *) = u 1 (D,  2 *)  = 2 (1 -  ) u 2 (L,  1 *) = u 2 (R,  1 *)  + 2 (1 -  ) = 4  + (1 -  )  = 1/4 ;  = 2/3 Unique mixed NE  1 * = 1/4 U + 3/4 D  2 * = 2/3 L + 1/3 R

21. Two People Zero Sum Games – Pure Strategy One player’s winnings is another player’s loss! Each player does the following: For each of his/her strategies, compute the maximum of losses that he could incur. Choose the strategy with the minimum max loss

22. Example 2 people 0 sum Game Row is player 1; Column is player 2 If a ij > 0, player 1 wins, else player 2 Player 1: i* = arg max i (min j (a ij )) V(A) = min j a i*j is the gain-floor for the game A In this case, V(A) = -2, with i*  {2, 3} Player 2: j* = arg min j (max i (a ij ))  (A) = max i a ij* is the loss-ceiling for the game A In this case,  (A) = 0, with j* = 3

23. Two People Zero Sum Game – Mixed Strategy If  (A) = V(A) then A has a point of equilibrium Else we need to develop mixed strategy Consider the following game: For player 1, we have V(A) = 0, with i* = 2 For player 2, we have  (A) = 1, with j* = 2 No saddle point or equilibrium Let players 1, 2 play strategy i with probability x i, y i

24. Best Choice Analysis

25. Optimization Problem In a nutshell, the players are solving the following pair of dual linear programming problems Player 1 Player 2

26. Application to networks Formulation for n users competing for fixed resources Generic non-cooperative game Each user has access control /parameter  n Each user receives certain amount  n (  ) of network resources   (  1,  2, ….  n )  n  [0,  n max ] for some  n max > 0  n (  ) is a non-decreasing function of  n  n (.) is continuous in n=1 N [0,  n max ] and is differentiable with respect to  n If  n = 0,  n (  ) =0 for all   n (  ) maybe interpreted as the QOS received by the n th user

27. Formulation (contd.) Let network charges be fixed at M / unit resources Each user tries to maximize his/her net utility U n (  n (  )) – M.  n (  ) Un is non-decreasing and Un(0) = 0; U’ n is non-increasing, i.e. U n is concave Maximum net benefit of n th user y n = arg max  (U n (  n (  )) – M.  n (  )) = (U’ n ) -1.(M) Action of n th user Modify  n to make received QOS  n (  ) equal to desired y n

28. User iterations and equilibrium After the j th iteration/step, access parameter of user n:  n j+1 = min (G (y n,  n (  j ),  n j ),  n max ) G (y, ,  )  , if  = y > , if  < y y Nash equilibria A fixed or equilibrium point of this iteration is any  *  [0,  n max ]  n * = min (G (y n,  n (  *),  n *),  n max ) By Brouwer’s fixed point theorem there exists at least one such fixed point.

29. Non-cooperative game for circuit switched network N users compete for K circuits n th user’s connection setup request is Poisson with intensity  n and arbitrary holding time distribution with mean 1/  n Total traffic intensity:   .1/  Aggregate arrival rate    n=1 N  n Mean holding time over all connections 1/  =  n=1 N 1/  n  n /  Hence,  =  n=1 N  n /  n Per user connection blocking probability (Erlang’s form)  K (  )  (  K /K!) / (  k=0 K  k /k!)

30. Formulation leading to equilibrium Net arrival rate of n th user:  n (1 -  K (  )) Mean number of occupied circuits for the n th user:  n (  )  1/  n  n (1 -  K (  (  ))) Thus,  n and  depend on all arrival rates  Iteration using multiplicative increase and decrease  n j+1 = min {y n /  n.  n,  n max } or,  n j+1 = min {y n.  n / (1 -  K (  (  j ))),  n max } By previous formulation we can find an equilibrium!

31. References Game Theory.NET - college lecture notes (http://www.gametheory.net) IE675: Game Theory - Dr. Wayne Bialas, Dept. of IE, SUNY Buffalo, (http://www.acsu.buffalo.edu/~bialas/IE675.html ) “Computational Finance: Game and Information Theoretic Approach” – Dr. B. Mishra, Dept. of CS, NYU (http://www.cs.nyu.edu/mishra/COURSES/GAME/game.html) Introduction to Game Theory – Markus Mobius, Dept. of Economics, Harvard (http://icg.fas.harvard.edu/~ec1052/lecture/index.html) Infocom 2003 - “Nash equilibria of a generic networking game with applications to circuit-switched networks” - Youngmi Jin and George Kesidis, Dept. of EE & CS, Pennsylvania State University.