Zero Determinant Strategy Game Theory in Wireless and Communication Networks: Theory, Models, and Applications Lecture 14 Zero Determinant Strategy Zhu Han, Dusit Niyato, Walid Saad, and Tamer Basar Thanks for Dr. Huaqing Zhang’s slides

Overview of Lecture Notes Introduction to Game Theory: Lecture 1, book 1 Non-cooperative Games: Lecture 1, Chapter 3, book 1 Bayesian Games: Lecture 2, Chapter 4, book 1 Differential Games: Lecture 3, Chapter 5, book 1 Evolutionary Games: Lecture 4, Chapter 6, book 1 Cooperative Games: Lecture 5, Chapter 7, book 1 Auction Theory: Lecture 6, Chapter 8, book 1 Matching Game: Lecture 7, Chapter 2, book 2 Contract Theory, Lecture 8, Chapter 3, book 2 Learning in Game, Lecture 9, Chapter 6, book 2 Stochastic Game, Lecture 10, Chapter 4, book 2 Game with Bounded Rationality, Lecture 11, Chapter 5, book 2 Equilibrium Programming with Equilibrium Constraint, Lecture 12, Chapter 7, book 2 Zero Determinant Strategy, Lecture 13, Chapter 8, book 2 Mean Field Game, Lecture 14, UCLA course, book 2 Network Economy, Lecture 15, Dr. Jianwei Huang, book 2

Overview Zero-Determinant Method Examples Cheating Management of Wireless Communication Power Control of Multiple Wireless Operators in LTE Unlicensed Huaqing Zhang, Dusit Niyato, Lingyang Song, Tao Jiang, and Zhu Han, “Zero-determinant Strategy for Resource Sharing in Wireless Cooperations," IEEE Transactions on Wireless Communications, vol. 15, no. 3, pp. 2179-2192, March 2016. Huaqing Zhang, Xianfu Chen, and Zhu Han, \A Zero-Determinant Approach for Power Control of Multiple Wireless Operators in LTE Unlicensed," IEEE Globecom 2016 Huaqing Zhang, Dusit Niyato, Lingyang Song, Tao Jiang, and Zhu Han, “Zero-Determinant Strategy in Cheating Management of Wireless Cooperation," IEEE Global Communications Conference, December, Austin, TX, 2014. 2

Game Theory (5,5) (3,6) (6,3) (0,0) Player Strategy Utility X: Macrocell Y: Small cell Chicken Dare (5,5) (3,6) (6,3) (0,0) Y X Strategy Chicken: Low transmit power Dare: High transmit power Utility Chicken-Dare Game Revenue for both X and Y. 12

Nash Equilibrium (5,5) (3,6) (6,3) (0,0) If each player has chosen a strategy and no player can benefit by changing strategies while the other players keep theirs unchanged, then the current set of strategy choices and the corresponding payoffs constitute a Nash equilibrium. A B C E Chicken Dare (5,5) (3,6) (6,3) (0,0) 13

Correlated Equilibrium In order to improve the social welfare of the game, both players not only are aware of their own strategies, but also consider the other’s behaviors to seek if there are mutual benefits to explore. Chicken Dare (5,5) (3,6) (6,3) (0,0) Chicken Dare 0.6 0.2 Maximum social welfare achieves 9.6 14

Correlated Equilibrium Zero-Determinant Method Is there still an improvement? B A One player is responsible to maintain high social welfare. The Game is played in an iterated way. D C E Zero-Determinant Method 15

Utility in one Iteration The four outcomes of the previous move are labeled 1, 2, 3, 4 for the respective outcomes 𝑥𝑦∈(𝑐𝑐,𝑐𝑑,𝑑𝑐,𝑑𝑑). Y Chicken Dare X X’s payoff matrix is R, R S, T T, S P, P 𝑺 𝑋 = 𝑅 𝑆 𝑇 𝑃 𝑇 Chicken Y’s payoff matrix is Dare 𝑺 𝑌 = 𝑅 𝑇 𝑆 𝑃 𝑇 16

Strategy Profile X’s strategy is p=(p1; p2; p3; p4) Y’s strategy is q=(q1; q2; q3; q4) 17

Markov Process Unnecessary to simulate the play of strategies p against q move by move, we adopt Markov process: The transition matrix M(p,q) = The stationary vector of Markov process(implying reaching the equilibrium of the game) is v: v=Mv 18

Expected Utilities 𝑠 𝑋 = 𝑣 T ⋅ 𝑆 𝑋 𝑣 T ⋅1 = 𝐷(𝑝,𝑞, 𝑆 𝑋 𝐷(𝑝,𝑞,1 Therefore, in the stationary state, the expected utilities of both players X and Y are, respectively, 𝑠 𝑋 = 𝑣 T ⋅ 𝑆 𝑋 𝑣 T ⋅1 = 𝐷(𝑝,𝑞, 𝑆 𝑋 𝐷(𝑝,𝑞,1 𝑠 𝑌 = 𝑣 T ⋅ 𝑆 𝑌 𝑣 T ⋅1 = 𝐷(𝑝,𝑞, 𝑆 𝑌 𝐷(𝑝,𝑞,1 The scores s depend linearly on their corresponding payoff matrices S. 19

Zero-Determinant Method Cramer’s rule 𝒗 𝑇 𝐌= 𝒗 𝑇 Adj 𝑴 ′ 𝑴 ′ = 𝑑𝑒𝑡 𝑴 ′ 𝑰 ( Adj 𝑴 ′ is the adjugate matrix of 𝑴 ′ ) We set 𝐌 ′ =𝐌−𝐈 The matrix 𝑴 ′ ≡ 𝑴−𝑰 is singular, with thus zero determinant. 𝒗 𝑇 𝐌′=𝟎 Adj 𝑴 ′ 𝑴 ′ = 𝟎 Every row of Adj 𝑴 ′ is proportional to 𝒗 𝑇 . 20

Zero-Determinant Method We do dot product of an arbitrary four-vector 𝒇= 𝑓 1 𝑓 2 𝑓 3 𝑓 4 𝑇 with the stationary vector v: 𝒗 𝑇 ⋅𝒇=𝐷(𝒑,𝒒,𝒇)=𝑑𝑒𝑡 −1+ 𝑝 1 𝑞 1 −1+ 𝑝 1 −1+ 𝑞 1 𝑓 1 𝑝 2 𝑞 3 −1+ 𝑝 2 𝑞 3 𝑓 2 𝑝 3 𝑞 2 𝑝 3 −1+ 𝑞 2 𝑓 3 𝑝 4 𝑞 4 𝑝 4 𝑞 4 𝑓 4 = 𝒑 = 𝒒 Notably, The second column is solely under the control of 𝑿; The third column is solely under the control of 𝒀; The fourth column is simply 𝒇. 21

Zero-Determinant Method We suppose 𝒇=𝛼 𝑺 𝑿 +𝛽 𝑺 𝒀 +𝛾𝟏 For any linear combination of scores, it is true that Both X and Y have the possibility of choosing unilateral strategies that will make the determinant in the numerator vanish. i.e. If X chooses a strategy that satisfies or if Y chooses a strategy that satisfies then the determinant vanishes and a linear relation between the two scores, We call these zero-determinant (ZD) strategies. 22

Constraints 𝛼 𝑠 𝑋 +𝛽 𝑠 𝑌 −𝛾=0 We suppose 𝛼 𝛽 <0 We suppose In the iterated Chicken-Dare game, when player X takes the zero-determinant strategy, achieving 𝛼 𝑠 𝑋 +𝛽 𝑠 𝑌 −𝛾=0 𝛼 𝛽 ≥ 𝑈 4 𝑌 − 𝑈 1 𝑌 𝑈 1 𝑋 − 𝑈 4 𝑋 the scalars 𝛼 and 𝛽 are required to meet the constraint the scalars 𝛾 is required to satisfy or 23

Improvements 𝛼 𝑠 𝑋 +𝛽 𝑠 𝑌 −𝛾=0 Player Y: A greedy player Player X: Zero-determinant strategy 𝛼 𝑠 𝑋 +𝛽 𝑠 𝑌 −𝛾=0 The final utilities of both players can fall on the line segments AC. Player X is able to adopt the different zero-determinant strategy to determine any specific point on line segment AC unilaterally. 24

Zero-Determinant Strategy When player X aims to maximize the social welfare of both players, and takes zero-determinant strategies The game achieves an equilibrium where the social welfare of both players is 𝑠 𝑋 + 𝑠 𝑌 =𝑈 1 𝑋 + 𝑈 1 𝑌 and none of the players would like to deviate from the current strategy. 25

Overview Zero-Determinant Method Examples Cheating Management of Wireless Communication Power Control of Multiple Wireless Operators in LTE Unlicensed 2

Wireless Resource Sharing Heterogeneous Network Introduction Wireless Resource Sharing becomes both heterogeneous and dynamic. Device-to-Device Cognitive Radio

Motivation we define the participant who is responsible to maintain the high social welfare as the administrator of cooperation (AoC), Introduction the other rational selfish participant as the regular participant of cooperation (PoC). However, because of Weak communication signals Cheating strategies Specially, Deal with the interaction of decision makers with conflicting interests Some PoCs may unexpectedly stop cooperation on resource sharing unilaterally. AoC is responsible to maintain the social welfare in high values

Problem Formulation 𝑈 all =𝛼 𝑠 𝐴 +𝛽 𝑠 𝑃 =−𝛾 In wireless communication networks, we assume AoC as player X, PoC as player Y. We set 𝑈 all =𝛼 𝑠 𝐴 +𝛽 𝑠 𝑃 =−𝛾 ZD method in Wireless The maximum social welfare the AoC can be maintained is obtained by solving the problem

Zero-determinant Strategy When the strategy of AoC is ZD method in Wireless where The social welfare can be maintained at the high value of 𝛼, 𝛽 is required to satisfy

Simulation Results C D C D Parameter Setting (Without special clarification) Initial State: 𝑣 0 =[0.25, 0.25, 0.25, 0.25] Utility in each situation C D Simulation C U R A =2 U R P =2 U 𝑆 A =−1 U 𝑇 P =4 D U T A =4 U S P =−1 U P A =0 U P P =0 Scalar Setting in Zero-determinant strategy 𝛼=1 𝛽=3

Simulation Results Strategy of AoC Zero-determinant Strategy Tit-for-Tat Strategy ( 𝒑= 1 0 1 0 ) Pavlov Strategy ( 𝒑= 1 0 0 1 ) Cooperative Strategy ( 𝒑= 1 1 1 1 ) Non-cooperative Strategy ( 𝒑= 0 0 0 0 ) Strategy of PoC Simulation Any Strategy ( 𝒒= 0.75 0.75 0.75 0.75 ) Tit-for-Tat Strategy ( 𝒒= 1 0 1 0 ) Pavlov Strategy ( 𝒒= 1 0 0 1 ) Cooperative Strategy ( 𝒒= 1 1 1 1 ) Non-cooperative Strategy ( 𝒒= 0 0 0 0 )

Simulation Results The social welfare versus number of iterations

Simulation Results The utility of AoC and PoC versus number of iterations Simulation Utility in Zero-determinant strategy is higher than all the others, except

Overview Zero-Determinant Method Examples Cheating Management of Wireless Communication Power Control of Multiple Wireless Operators in LTE Unlicensed 2

Introduction LTE is the most advanced mobile telecommunication technology. Introduction To further expand LTE capacity to meet the traffic demands, a natural way is to integrate unlicensed carrier into the overall LTE system by adapting LTE air interface to operate in the unlicensed spectrum, i.e., LTE unlicensed (LTE-U)

Challenge Single-Floor Wi-Fi and LTE operator Performance overview from “Performance Evaluation of LTE and Wi-Fi Coexistence in Unlicensed Bands” Single-Floor Sparse deployed Dense deployed

Challenge Multiple LTE operators

Architecture System Model MU: Mobile User WU: Wi-Fi User WAP: Wi-Fi Access Point Data signal WCO: Wireless Cellular Operator Interference

Relations between the WAP and each WCO Relations among all WCOs Game Analysis Relations between the WAP and each WCO Relations among all WCOs WCO WAP WCO WCO WCO …… Game Analysis Zero-Determinant Strategy

Simulation Results There are 2 WCOs trying to share the unlicensed spectrum with the WAP in a two-dimensional area The WCOs are located at coordinates (50,0) and (25,43) MUs are located at coordinates (90,0) and (−5,43) The WAP is assumed to be located at the origin, and its serving WU at coordinates (0,10) Simulation The power levels for both WCOs are, respectively, {600,1200} mW and {450,900} mW, and the power levels for the WAP are chosen from {400,800} mW. The power of the additive white noise is σ = −105dBm

Simulation Results Zero-determinant strategy is guaranteed to converge; The social welfare is determined by the behaviors of the WAP

Simulation Results Increasing the lower power level of WAP is able to improve the social welfare

Conclusion Chicken Game played repeatedly A dominant player wants to control a selfish user’s behaviors Zero Determined Strategy A linear performance constraint can be enforced Potential Applications in Future Networks