An Architectural View of Game Theoretic Control Raga Gopalakrishnan and Adam Wierman California Institute of Technology Jason R. Marden University of Colorado at Boulder 6/18/2010Hotmetrics 2010

Distributed Resource Allocation Sensor CoverageWireless Access Point Selection Wireless Channel SelectionPower Control (sensor networks)

Resource Allocation Problem – A Simple Model Set of (distributed) agents, N = {1, 2,..., n} Set of resources, R Action sets, A i µ 2 R for agents i 2 N – Set of action profiles, A = A 1 £ A 2 £... £ A n – Set of agents choosing resource r in action profile a, {a} r Objective function, W : A! R – Linearly separable, i.e., W(a) =  r 2 R W r ( {a} r ) Goal: Find an allocation a 2 A that maximizes W(a)

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Game-theoretic Control

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Game-theoretic Control  Promising new approach  Model the agents as “self-interested” players in a non-cooperative game  Still being explored  The solution to the problem emerges as the equilibrium of the game

Modeling the problem as a game Set of players, N = {1, 2,..., n} Action sets, A i µ 2 R for players i 2 N – Set of action profiles, A = A 1 £ A 2 £  £ A n – Set of players choosing resource r in action profile a, {a} r Utility functions, U i : A! R for players i 2 N – Linearly separable, i.e., U i (a) =  r 2 R f r ( i, {a} r ) Welfare function W : A! R – Linearly separable, i.e., W(a) =  r 2 R W r ( {a} r ) Resource Allocation ProblemResource Allocation Game Set of agents, N = {1, 2,..., n} Set of resources, R Action sets, A i µ 2 R for agents i 2 N – Set of action profiles, A = A 1 £ A 2 £... £ A n – Set of agents choosing resource r in action profile a, {a} r Objective function, W : A! R – Linearly separable, i.e., W(a) =  r 2 R W r ( {a} r )

Game Theoretic Control (GTC) Setup the game 1 Design the players 2  decision makers/players  action sets  utility functions agent decision rules (learning rules) Desirable global behavior emerges as equilibrium of the game Goal:

Game Theoretic Control (GTC) Setup the game 1 Design the players 2  decision makers/players  action sets  utility functions agent decision rules (learning rules) Desirable properties  Existence of an equil.  Efficiency of an equil.  Tractability  Locality of information  Budget balance  … Desirable properties  Locality of information  Fast convergence  Equilibrium selection  Robust convergence  … Learning Design Utility Design Inherited Designed

Many other applications: [Akella et al. 2002, Kaumann et al. 2007, Marden et al. 2007, 2008, Mhatre et al. 2007, Komali and MacKenzie 2007, Zou and Chakrabarty 2004, Campos-Nanez 2008, Marden & Effros 2009] [Marden, Wierman 2008] [Campos-Nanez, Garcia, Li 2008] Applications of GTC Utility Design Learning Design Sensor Coverage Power Control (sensor networks) Is there a way to view Game Theoretic Control from an application-independent perspective?

Architectural View for GTC Utility Design Learning Design Class of Games “Virtualization” layer IP Network Apps Network hardware OS software hardware Potential Games are games for which there exists a potential function  : A! R such that ∀ i 2 N, ∀ a –i 2 A –i, ∀ a i, a i ’ 2 A i, it holds that  (a i, a –i ) –  (a i ’, a –i ) = U i  (a i, a –i ) – U i (a i ’, a –i ) Key Property: Local maxima of  are Nash equilibria

Potential Games-based Architecture Utility Design Learning Design Potential Games Unifying view of several existing designs: [Akella et al. 2002] [Kaumann et al. 2007] [Marden et al. 2007, 2008] [Mhatre et al. 2007] [Komali and MacKenzie 2007] [Zou and Chakrabarty 2004] [Campos-Nanez 2008] [Marden & W 2008] [Marden & Effros 2009] and many others…

Utility Design (examples) Wonderful Life Utility (WLU) [Wolpert et al. 1999] – Potential game with © = W (hence, price of stability = 1) – Price of anarchy = ½ for sub-modular games Shapley Value Utility (SVU) [Shapley 1953] – Potential game – Price of anarchy = Price of stability = ½ for sub-modular games Weighted SVU [Shapley 1953] – Similar properties as SVU Adapted from cost-sharing literature in economic theory [Marden, Wierman]

Learning Design (examples) Gradient Play [Ermoliev et al. 1997, Shamma et al. 2005] – Convergence to a Nash equilibrium Joint Strategy Fictitious Play (JSFP) [Marden et al. 2009] – Convergence to a Nash equilibrium Log-Linear Learning [Blume 1993, Marden et al.] – Convergence to the best Nash equilibrium Many others... [Ozdaglar et al. 2009, Shah et al. 2010]

Potential Games-based Architecture Utility Design Learning Design Potential Games SVU Wonderful Life WSVU Gradient Play Log-Linear Learning JSFP +Modularity / Decoupling +Flexibility ?Relationships to other approaches ?Limitations +Modularity / Decoupling +Flexibility ?Relationships to other approaches ?Limitations

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Potential Games Utility Design Learning Design Relationships to Other Approaches Game-theoretic Control

Distributed Constraint Optimization Problem (DCOP) – Utility Design: WLU – Learning Design: Variety Chapman, Rogers, Jennings – Benchmarking hybrid algorithms for distributed constraint optimization games [OptMAS ‘08] Potential Games WLU Variety Distributed Optimization

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Potential Games Utility Design Learning Design Game-theoretic Control Relationships to Other Approaches

Gibbs-sampler-based control ―Utility Design: WLU ―Learning Design: Log-Linear Learning Access Point SelectionChannel Selection Kauffmann, Baccelli, Chaintreau, Mhatre, Papagiannaki, Diot – Measurement-based self organization of interfering wireless access networks [INFOCOM ‘07] Potential Games WLU Log-Linear Learning Physics-inspired Control We prove that

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Potential Games Utility Design Learning Design Game-theoretic Control Relationships to Other Approaches

Distributed Approaches Distributed Optimization Lyapunov-based Control Physics-inspired Control Potential Games Utility Design Learning Design Game-theoretic Control Relationships to Other Approaches

Potential Games-based Architecture Utility Design Learning Design Potential Games SVU Wonderful Life WSVU Gradient Play Log-Linear Learning JSFP +Modularity / Decoupling +Flexibility Relationships to other approaches ?Limitations +Modularity / Decoupling +Flexibility Relationships to other approaches ?Limitations

Do Potential Games Suffice? No utility design with all the desirable properties Utility Design Learning Design POTENTIAL GAMES Desirable properties  Existence of an equil.  Efficiency of an equil.  Budget balance  Tractability  Locality of information  … Not always! Open Question: What other limitations are there? Any linearly separable, budget-balanced utility design that guarantees equilibrium existence has PoS · ½ [Marden, Wierman 2009]

Summary Utility Design Learning Design Potential Games SVU Wonderful Life WSVU Gradient Play Log-Linear Learning JSFP +Modularity / Decoupling +Flexibility Relationships to other approaches ―Not all desirable properties can be achieved +Modularity / Decoupling +Flexibility Relationships to other approaches ―Not all desirable properties can be achieved ?Beyond Potential Games

Conclusion Utility Design Learning Design Potential Games SVU Wonderful Life WSVU Gradient Play Log-Linear Learning JSFP +Modularity / Decoupling +Flexibility Relationships to other approaches ―Not all desirable properties can be achieved ? Other choices for virtualization layer [MW’09,AJWG’09,Sv’09] Strengths and Limitations A library of architectures

Thank You