1. “mean-Field games with strategic servers”
Asaf Cohen
(joint with Erhan Bayraktar and Amarjit Budhiraja)
Department of Mathematics
University of Michigan
Math Finance, Probability, and PDE Conference
Rutgers University
May 2017
add

2. Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme

3. Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme

4. Literature The Queueing Model Rate control in communication systems:
H. J. Kushner. Heavy traffic analysis of controlled queueing and communication networks,
2001.
(2) B. Ata, J. M. Harrison, and L. A. Shepp. Drift rate control of a Brownian processing system. AAP, 2005
(3) A. Budhiraja, A. P. Ghosh, and C. Lee. Ergodic rate control problem for single class queueing networks, SICON, 2011
Rate control in limit order books:
E. Bayraktar and M. Ludkovski. Liquidation in limit order books with controlled intensity.
Math. Finance, 2014.
(2) O. Gueant and C.A. Lehalle. General intensity shapes in optimal liquidation. Math. Finance,
2015.
(3) A. Lachapelle, J. Lasry, C.A. Lehalle, and P. Lions. Efficiency of the price formation process in presence of high frequency participants: a mean field game analysis, Arxiv , 2015.
Strategic servers:
(1) R. Gopalakrishnan, S. Doroudi, A. R. Ward, and A. Wierman. Routing and staffing when servers are strategic. Oper. Res., 2016.
(2) J. Li, R. Bhattacharyya, S. Paul, S. Shakkottai, and V. Subramanian. Incentivizing sharing in real time d2d streaming networks: A mean field game perspective. IEEE(INFOCOM), 2015

5. Basic definitions The Queueing Model queues rejected
controlled arrival
process
controlled service
process
queues
size of the
i-th queue
rejected
capacity

6. Basic definitions The Queueing Model queues rejected
controlled arrival
process
controlled service
process
queues
size of the
i-th queue
rejected
capacity

7. Scaling The Queueing Model queues arrival rates service rates where
controlled arrival
process
controlled service
process
queues
size of the
i-th queue
arrival rates
service rates
where
is the control of Server
heavy traffic
is the scaled queue length
is the empirical distribution of the scaled queue lengths

8. Scaling The Queueing Model queues arrival rates service rates where
controlled arrival
process
controlled service
process
queues
size of the
i-th queue
arrival rates
service rates
where
is the control of Server
heavy traffic
is the scaled queue length
is the empirical distribution of the scaled queue lengths

9. reflections from below and above
The Queueing Model
Scaling
reflections from below and above
where

10. The Queueing Model
The control problem

11. The control problem MIN The Queueing Model
Representation by the Skorokhod map:
Given and
the cost is
initial states
strategies
MIN

12. The Queueing Model Assumptions a)-b) Main Result
a) The functions and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
Main Result
There exists an asymptotic Nash equilibrium. That is, there is a sequence of admissible strategies , such that for every player , and every sequence of admissible strategies for that player, one has

13. Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme

14. Literature The Mean-Field Game Mean-field games:
(1) J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. I. Le cas stationnaire. C. R. Math. Acad. Sci. Paris,
(2) J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. II. Horizon ni et contrôle optimal. C. R.
Math. Acad. Sci. Paris, 2006.
(3) J.-M. Lasry and P.-L. Lions. Mean field games. Jpn. J. Math., 2007.
(4) M. Huang, P. E. Caines, and R. P. Malhame. The Nash certainty equivalence principle and Mckean-Vlasov systems: An invariance principle and entry adaptation. Decision and Control, IEEE, 2007.
(5) M. Huang, R. P. Malhame, and P. E. Caines. Large population stochastic dynamic games: Closed loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst., 2006.
(6) P. Cardaliaguet. Notes on mean field games. 2013
(7) R. Carmona and F. Delarue. Probabilistic analysis of mean-field games. SICON, 2013.
(8) R. Carmona and D. Lacker. A probabilistic weak formulation of mean field games and applications. AAP, 2015.
Brownian control problem:
(1) J.M. Harrison, Brownian models of queueing networks with heterogeneous customer populations, IMA Volumes in Mathematics and its Applications, 1988.

15. Intuition MIN The Mean-Field Game
The idea is to approximate the queueing process
By the drift-controlled reflected diffusion process
weak formulation
In the heavy-traffic literature, the limit problem is often called Brownian control problem. We now define a Brownian control problem of mean-field type, that is, a mean-field game.
MIN
Notice that the processes in the pre-limit are discrete, whereas the limit is continuous!

16. The Mean-Field Game
Scheme
The fixed point is called a solution of the mean-field game
Fix a flow of measures
Solve the control problem with replacing the flow of empirical
measures Denote the value function by
(2) Find the optimal control and the distribution of under that control
(3) Find a fixed point of the mapping
The fixed point is called a solution of the MFG
Theorem 1
Under Assumptions a)-b), there is a solution to the MFG.
Remark
Under standard assumptions in MFG theory uniqueness of the MFG solution is attained.

17. Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme

18. Asymptotic Nash Equilibrium
Assumptions a)-d)
a) The functions and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
c) The function is independent of the mean-field term.
d) There exists s.t. for every ,
Theorem 2 (main result)
There exists an asymptotic Nash equilibrium. That is, there is a sequence of admissible controls , such that for every player , and every sequence of admissible strategies for that player, one has

19. (1) Same control for all the players
Asymptotic Nash Equilibrium
(1) Same control for all the players
The controls are defined by where is a solution of the MFG.
derivative
of the value
of the MFG
We show that the sequence
is tight and by reducing to a subsequence using de-Finetti type of argument,
Weak formulation
and
Using the regularity of and the continuity of the Skorokhod mapping, we get
We use a diagonalization argument to show that the sequence
is tight and by reducing to a subsequence and using de-Finetti type of argument, the regularity of , continuity of the Skorokhod mapping, and that is a MFG solution we get that
By the SLLN, we get , and by the continuity of the cost functions we get that
Value of the MFG with

20. (1) Same control for all the players
Asymptotic Nash Equilibrium
(1) Same control for all the players
(2) Deviation of one player (Player 1)
The controls are defined by where is a solution of the MFG.
Server 1 uses , while the others
We show that the sequence
is tight and by reducing to a subsequence using de-Finetti type of argument,
We show that the sequence
is tight and by reducing to a subsequence using de-Finetti type of argument,
relaxed controls
Using the regularity of and the continuity of the Skorokhod mapping, we get
Using the convergence of and the continuity of the Skorokhod mapping, we get
We use a diagonalization argument to show that the sequence
is tight and by reducing to a subsequence and using de-Finetti type of argument, the regularity of , continuity of the Skorokhod mapping, and that is a MFG solution we get that
By the SLLN, we get , and by the continuity of the cost functions we get that
One player has no impact on the limit of the empirical distribution, so as before, we get , and by the continuity of the cost functions we get that
Value of the MFG with
Value of the MFG with

21. Contents
The Queueing Model
The Mean-Field Game
Asymptotic Nash Equilibrium
Numerical Scheme

22. Literature Numerical Scheme
Y. Achdou and I. Capuzzo-Dolcetta. Mean-Field Games: Numerical Methods, SIAM J.
Numer. Anal, 2010
(2) Y. Achdou, F. Camilli, and I. Capuzzo-Dolcetta. Mean-Field Games: Numerical Methods for
the Planning Problem, SICON, 2012
(3) S. Cacace, F. Camilli, C.A. Marchi. A Numerical Method for Mean-Field Games on Networks., ESAIM: Mathematical Modelling and Numerical Analysis, 2017

23. Approximating Markov chain
Numerical Scheme
Approximating Markov chain
Fix and
buffer
size
time
MDP
MIN

24. Assumptions Numerical Scheme Assumptions a)-e) Fix a sequence
a) The functions and are uniformly Lipschitz with respect to all of their
arguments.
b) The Hamiltonian has a unique minimizer:
c) The function is independent of the mean-field term.
d) There exists s.t. for every ,
is Lipschitz
e) The MFG has a unique solution.

25. First attempt Numerical Scheme Fix a sequence Iterations:
optimal
(1) Fix an arbitrary function and set
tightness
(subsequence)
(2) Define
Now, the tightness of implies that is relatively tight.
Finally, we need to make sure that is optimal. To this end, we fix an arbitrary control and by an additional discretization argument, and using the optimality of we show that is suboptimal.
If is the subsequence, then is
However, this is not working! Tightness forces us to reduce to subsequences and we cannot obtain the limit .

26. Not contracting! Solution Numerical Scheme
Using the same scheme for a fixed h:
(a) show contraction of the mapping
Not contracting!
(b) find a fixed point of (for the fixed h), denoted by So,
and
(c) take and use the tightness argument.
(2) Using the same scheme for a fixed h:
(a) show “almost” contraction of the mapping
(b) after enough iterations as before, there is such that
and
(c) take and use the tightness argument and the last inequality.

27. Numerical study Numerical Scheme
We studied a linear quadratic MFG with penalty for rejections:
and
Also,
where is the mean of .

28. Numerical Scheme
Numerical study

29. Thank you!

30. Summary
(1) We consider a rate control problem for large symmetric queuing systems in heavy-traffic with strategic servers.
(2) We introduce an MFG for controlled reflected diffusions, establish its solvability and prove unique solvability under additional conditions.
(3) We use the solution of MFG to construct an asymptotically optimal Nash equilibrium for the n-player game.
(4) We develop a convergent numerical scheme to solve the MFG.

31. Scheme MIN Theorem 1 The Mean-Field Game
The fixed point is called a solution of the mean-field game
Fix a flow of measures
Solve the control problem
s.t.
Find the optimal control and the distribution of under that control.
(2) Find a fixed point of the mapping
MIN
The fixed point is called a solution of the MFG
Theorem 1
Under Assumptions a)-b), there is a solution to the MFG.