1. Game and Evolutionary Game in Communication Networks Yuedong Xu 2013.12.04

2. Outline Game Theory: A Premier Evolutionary Game Applications to Networks Potential Research Fields Using as less math as possible ! Using as less math as possible !

3. Game Theory: A Premier What is game about? Game of Chicken – driver who swerves away looses What should drivers do? – To swerve or to stay? 2 2

4. Game Theory: A Premier What is game about? Game of Chicken – driver who swerves away looses 2 2 swervestay swerve0, 0-1, 5 stay5, -1-10, -10 Driver 1 Driver 2 Drivers want to do opposite of one another

5. Game Theory: A Premier A Game consists of – at least two players – a set of strategies for each player – a payoff for each strategy profile Basic assumption (rationality of players) Nash Equilibrium – no player can improve its payoff by unilaterally changing its strategy Pareto optimality, price of anarchy

6. Game Theory: A Premier Non-Cooperative (Competitive) Games – individualized play Cooperative Games – play as a group Repeated, Stochastic and Evolutionary Games – not one shot Classification 1:

7. Game Theory: A Premier Classification 2: Non-cooperativeCooperative StaticDynamic (repeated) Strategic-formExtensive-form Perfect informationImperfect information Complete informationIncomplete information

8. Game Theory: A Premier Internet Application Selfish Routing game s v w t C(x) = 1 C(x) = x C(x) = 1 C(x) = x C(x) = 0

9. Game Theory: A Premier Internet Application P2P Networks: Bittorrent, Xunlei, Pplive, PPStream, QQLive …

10. Game Theory: A Premier Internet Application Internet Ecosystem (Business Models)

11. Game Theory: A Premier Internet Application Cloud Computing game

12. Game Theory: A Premier Internet Application Online Social Networks

13. Game Theory: A Premier Internet Application Network Security Game

14. Game Theory: A Premier Wireless Application 802.11 multiple access game

15. Game Theory: A Premier Wireless Application 3G/4G Power Control Game

16. Game Theory: A Premier Wireless Application Packet forwarding game ? ? Blue Green

17. Game Theory: A Premier Wireless Application Cognitive radio network game

18. Game Theory: A Premier Wireless Application Wireless jamming and eavesdrop games E

19. Outline Game Theory: A Premier Evolutionary Game Applications to Networks Potential Research Fields

20. Recap Follow the crowd!

21. Evolutionary game theory Evolutionary game theory (EGT) – refinement of CGT – game in a population – dynamics of strategy adoption – mutual learning among players Evolutionary game theory differs from classical game theory by focusing more on the dynamics of strategy change as influenced not solely by the quality of various competing strategies, but by the effect of frequency with which the various competing strategies are found in the population.

22. Evolutionary game theory Evolutionary game theory (EGT) – Usually two types of game: games against the field and games with pairwise contests A game against the eld is one in which there is no specic opponent for a given individual - their payoff depends on what everyone in the population is doing. Ex: Choice of Gender A pairwise contest game describes a situation in which a given individual plays against an opponent that has been randomly selected (by nature) from the population and the payoff depends just on what both individual do. Ex: Hawk-Dove Game

23. Evolutionary game theory A profile of evolutionary game Payoff (fitness)

24. Evolutionary game theory Evolutionary stable strategy (ESS) Theorem (ESS) An evolutionarily stable strategy is a strategy which, if adopted by a population of players, cannot be invaded (or replaced) by any alternative strategy that is initially rare.

25. Evolutionary game theory

26. Example (Hawk-Dove Game) cont – In the population, the payoff of a mutant is

27. Evolutionary game theory

29. Lyapunov stability vs asymptotic stability Lyapunov function and Engenvalue approach

30. Evolutionary game theory ESS vs NE in associated two-player game – An ESS is a (mixed) NE – A NE might not be an ESS Asymmetric NE in monomorphic population Unstable NE

31. Evolutionary game theory Replicator dynamics and NE – In a two-strategy game Any NE is a fixed point of replicator dynamics Not every fixed point corresponds to a NE Replicator dynamics and ESS – ESS is an asymptotically stable fixed point – Two strategy pair-wise contest – More than two strategies

32. Outline Game Theory: A Premier Evolutionary Game Applications to Networks Potential Research Fields Peer-to-peer file sharing Wireless networks

33. Peer-to-peer file sharing File Piece (e.g. chunk, block) – A content is split in pieces – Each piece can be independently downloaded Leecher – A peer that is client and server – In the context of content delivery Has a partial copy of the content Seed – A peer that is only server – In the context of content delivery Has a full copy of the content

34. Great improvement over customer-server mode Ideal system: single chunk, fully cooperative Big System: many peers, many chunks, stochastic system time t=0 t=T t=2T Seed 34 Peer-to-peer file sharing Which peers shall I serve in each time slot?

35. Peer-to-peer file sharing If no good incentive strategy – Slow service – Even overwhelmed by requests Incentive model – A strategy is the behavior (providing/rejecting a service) of a peer against other peers – A policy is the set of rules of for incentivization – A point is a utility measure of peers – A system is robust : convergence and cooperation Q. Zhao, J. Lui, D. ChiuA Mathematical Framework for Analyzing Adaptive Incentive Protocols in P2P Networks, IEEE/ACM Trans. Networking, 2012

36. Peer-to-peer file sharing Incentive model (cont) – Strategy = type of peer – Finite strategies {cooperator, defector, reciprocator} Always serve Always reject Serve cooperators and reciprocators with certain probabilities, reject defectors

37. Peer-to-peer file sharing Incentive model (cont) – System description: – Incentive scheme (esp. for reciprocators) At the beginning of each time slot, each peer randomly selects another peer to request for service. The selected peer chooses to serve the request based on his current strategy. A peer obtains α points if its request is served and loses β (=1) points if it provides service to others.

38. Peer-to-peer file sharing Utility model – After a long way, the points gained by a type-i peer We can now study – equilibrium state (given G) – is the equilibrium stable? – how to reach this equilibrium? – how good is the incentive scheme Type-i payoff Network payoff Is this enough?

39. Peer-to-peer file sharing Learning model in P2P networks – Current best learning model At the end of each slot, a peer chooses to switch to another strategy s with certain prob. To decide which strategy to choose, the peer learns from other peers. Needs to compute the gains of all other peers !

40. Peer-to-peer file sharing Learning model in P2P networks – Opportunistic learning model At the end of each slot, each peer chooses another peer as its teacher with certain prob. If the teacher is of a different type and performs better, this peer adapts to the teachers strategy with another prob. Simpler !

41. Peer-to-peer file sharing Now we can study – Robustness of incentive scheme Mirror incentive policy – reciprocators are tit-for-tat Proportional incentive policy – A reciprocators always serves any other reciprocator Linear incentive policy Each scheme generates a different matrix G ! Prob. That reciprocators serve other types of peers!

42. Peer-to-peer file sharing

43. In relation to EG – pair-wise contest population game – peers players; chunk exchange 2 players games Opp. Learning After some efforts Replicator dynamics Curr. Best Learning

44. Large-scale wireless networks Random multiple access (slotted ALOHA) – A node transmits with prob. p in each slot – Simultaneous transmission collisions

45. Large-scale wireless networks Power control game (signal to noise interference ratio, SINR) – Large power better throughput – Large power more interference to other receivers

46. Large-scale wireless networks

47. Sad facts: – Selfishness is unsuccessful – Optimal cooperation is hard in a large distributed networks ( bargaining, Shapley value ) Evolutionary game kicks in! H. Tembine, E. Altman, Evolutionary Games in Wireless Networks, IEEE Trans. Syst. Man Cyber. B, 2010 What if wireless nodes learn from each other in local interactions? What if wireless nodes learn from each other in local interactions?

48. Large-scale wireless networks Challenges – Standard EGT: a player interacts with all other players (or average population) – Large-scale wireless networks: no longer strategic pair-wise competition random number of local players non-reciprocal interactions – Finite strategies of a player {transmit, stay quiet} in multiple access game {high power, low power} in power control game Non standard EGT Standard EGT Non standard EGT Standard EGT

49. Large-scale wireless networks WCDMA power control game – SINR with distance r between transmitter and receiver of node i is given by PHPH PLPL PHPH Pi : the strategy of node i (i.e., PH or PL) x : the proportion of the population choosing PH g : channel gain, r0 is the radius-of-reception circle of receiver α : the attenuation order with value between 3 and 6, σ : the noise power, and β : the inverse of processing gain I(x) : total interference from all nodes to the receiver of node i

50. Large-scale wireless networks WCDMA power control game – Payoff of node i is as follows: R : transmission range wp : cost weight due to adopting power Pi (e.g. energy consumption) ζ(r) : probability density function given the density of receiver

51. Large-scale wireless networks WCDMA power control game – Existence of uniqueness of ESS Replicator dynamics This function is continuous and strictly monotonic, which is required for the proof of stability based on sufficient condition

52. Large-scale wireless networks Some other related works – Extensions to EGT – Applications P. Coucheney, C. Touati. Fair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks, IEEE Infocom 2009 (mini) S. Shakkottai, E. Altman. The Case for Non-cooperative Multihoming of Users to Access Points in IEEE 802.11 WLANs, IEEE Infocom 2006 C. Jiang, K. Liu, Distributed Adaptive Networks: A Graphical Evolutionary Game-Theoretic View, IEEE Trans. Signal Processing, 2013 P. Coucheney, C. Touati. Fair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks, IEEE Infocom 2009 (mini) S. Shakkottai, E. Altman. The Case for Non-cooperative Multihoming of Users to Access Points in IEEE 802.11 WLANs, IEEE Infocom 2006 C. Jiang, K. Liu, Distributed Adaptive Networks: A Graphical Evolutionary Game-Theoretic View, IEEE Trans. Signal Processing, 2013 E. Altman, Y. Hayel. Markov Decision Evolutionary Games, IEEE Trans. Auto. Ctrl. 2010 X. Luo and H. Tembine. Evolutionary Coalitional Games for Random Access Control, IEEE Infocom 2013 (mini) E. Altman, Y. Hayel. Markov Decision Evolutionary Games, IEEE Trans. Auto. Ctrl. 2010 X. Luo and H. Tembine. Evolutionary Coalitional Games for Random Access Control, IEEE Infocom 2013 (mini)

53. Large-scale wireless networks Summary – P2P : practical problem EG theory – WCDMA: EG theory practical problem – Common Challenges: difficult to find important problem difficult to have theoretical contributions to EGT Two different styles !

54. Thank you! Q & A