Potential Functions and the Inefficiency of Equilibria Tim Roughgarden Stanford University

Pigou's Example Example: one unit of traffic wants to go from s to t Question: what will selfish network users do? assume everyone wants smallest-possible cost [Pigou 1920] cost depends on congestion c(x)=x s t c(x)=1 no congestion effects

Motivating Example Claim: all traffic will take the top link. Reason: Є > 0  traffic on bottom is envious Є = 0  equilibrium all traffic incurs one unit of cost Flow = 1-Є c(x)=x s t c(x)=1 this flow is envious! Flow = Є

Can We Do Better? Consider instead: traffic split equally Improvement: half of traffic has cost 1 (same as before) half of traffic has cost ½ (much improved!) Flow = ½ c(x)=x s t c(x)=1 Flow = ½

Braess’s Paradox Initial Network: s t x 1 ½ Cost = 1.5

Braess’s Paradox Initial Network: Augmented Network: s t x 1 ½ ½ ½ x 1 s t ½ ½ 1 x Cost = 1.5 Now what?

Braess’s Paradox Initial Network: Augmented Network: s t x 1 ½ x 1 s t s t 1 x Cost = 1.5 Cost = 2

Braess’s Paradox Initial Network: Augmented Network: All traffic incurs more cost! [Braess 68] also has physical analogs [Cohen/Horowitz 91] s t x 1 ½ x 1 s t 1 x Cost = 1.5 Cost = 2

High-Level Overview Motivation: equilibria of noncooperative network games typically inefficient e.g., Pigou's example + Braess's Paradox don't optimize natural objective functions Price of anarchy: quantify inefficiency w.r.t some objective function Our goal: when is the price of anarchy small? when does competition approximate cooperation? benefit of centralized control is small

Selfish Routing Games directed graph G = (V,E) source-destination pairs (s1,t1), …, (sk,tk) ri = amount of traffic going from si to ti for each edge e, a cost function ce(•) assumed continuous and nondecreasing Examples: (r,k=1) c(x)=x c(x)=x c(x)=1 ½ ½ c(x)=0 s1 t1 s1 t1 c(x)=1 c(x)=1 c(x)=x ½ ½

Outcomes = Network Flows Possible outcomes of a selfish routing game: fP = amount of traffic choosing si-ti path P outcomes of game flow vectors f flow vector: nonnegative and total flow  fP on si-ti paths equals traffic rate ri (for all i) s t

Outcomes = Network Flows Possible outcomes of a selfish routing game: fP = amount of traffic choosing si-ti path P outcomes of game flow vectors f flow vector: nonnegative and total flow  fP on si-ti paths equals traffic rate ri (for all i) Question: What are the equilibria (natural selfish outcomes) of this game? s t

Nash Flows Def: [Wardrop 52] A flow is at Nash equilibrium (or is a Nash flow) if no one can switch to a path of smaller cost. I.e., all flow is routed on min-cost paths. [given current edge congestion] Examples: 1 ½ x x s t s t 1 1 ½ x 1 ½ x 1 1 s t s t 1 x ½ 1 x

Our Objective Function Definition of social cost: total cost C(f) incurred by the traffic in a flow f. Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then: C(f) = P fP • cP(f) s t

Our Objective Function Definition of social cost: total cost C(f) incurred by the traffic in a flow f. Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then: C(f) = P fP • cP(f) Example: s t x ½ s t Cost = ½•½ +½•1 = ¾ 1 ½

The Price of Anarchy Defn: price of anarchy of a game definition from [Koutsoupias/Papadimitriou 99] price of anarchy of a game obj fn value of selfish outcome = optimal obj fn value Example: POA = 4/3 in Pigou's example 1 ½ x x s t s t 1 1 ½ Cost = 3/4 Cost = 1

A Nonlinear Pigou Network Bad Example: (d large) equilibrium has cost 1, min cost  0 s t xd 1 1-Є Є

A Nonlinear Pigou Network Bad Example: (d large) equilibrium has cost 1, min cost  0  price of anarchy unbounded as d -> infinity Goal: weakest-possible conditions under which P.O.A. is small. s t xd 1 1-Є Є

When Is the Price of Anarchy Bounded? Examples so far: Hope: imposing additional structure on the cost functions helps worry: bad things happen in larger networks s t x 1 xd x 1 s t s t 1 1 x

Polynomial Cost Functions Def: linear cost fn is of form ce(x)=aex+be Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions: ≤ 4/3 × cost of Nash flow cost of opt flow s t x 1

Polynomial Cost Functions Def: linear cost fn is of form ce(x)=aex+be Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions: ≤ 4/3 × Bounded-deg polys: (w/nonneg coeffs) replace 4/3 by Θ(d/log d) cost of Nash flow cost of opt flow s t x 1 xd tight example s t 1

A General Theorem Thm: [Roughgarden 02], [Correa/Schulz/Stier Moses 03] fix any set of cost fns. Then, a Pigou-like example 2 nodes, 2 links, 1 link w/constant cost fn) achieves worst POA s t xd 1 tight example

Interpretation Bad news: inefficiency of selfish routing grows as cost functions become "more nonlinear". think of "nonlinear" as "heavily congested" recall nonlinear Pigou's example Good news: inefficiency does not grow with network size or # of source-destination pairs. in lightly loaded networks, no matter how large, selfish routing is nearly optimal s t xd 1 tight example

Benefit of Overprovisioning Suppose: network is overprovisioned by β > 0 (β fraction of each edge unused). Then: Price of anarchy is at most ½(1+1/√β). arbitrary network size/topology, traffic matrix Moral: Even modest (10%) over-provisioning sufficient for near-optimal routing.

Potential Functions potential games: equilibria are actually optima of a related optimization problem has immediate consequences for existence, uniqueness, and inefficiency of equilibria see [Beckmann/McGuire/Winsten 56], [Rosenthal 73], [Monderer/Shapley 96], for original references see [Roughgarden ICM 06] for survey

The Potential Function Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f ce(x)dx (over all flows). ce(fe) e fe

The Potential Function Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f ce(x)dx (over all flows). Lemma 1: locally optimal solutions are precisely the Nash flows (derivative test). Lemma 2: all locally optimal solutions are also globally optimal (convexity). Corollary: Nash flows exist, are unique. ce(fe) e fe

Consequences for the Price of Anarchy Example: linear cost functions. Compare cost + potential function: C(f) = e fe • ce(fe) = e [ae fe + be fe] PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe] 2 2 e

Consequences for the Price of Anarchy Example: linear cost functions. Compare cost + potential function: C(f) = e fe • ce(fe) = e [ae fe + be fe] PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe] cost, potential fn differ by factor of ≤ 2 gives upper bound of 2 on price on anarchy C(f) ≤ 2×PF(f) ≤ 2×PF(f*) ≤ 2×C(f*) 2 2 e

Better Bounds? Similarly: proves bound of d+1 for degree-d polynomials (w/nonnegative coefficients). not tight, but qualitatively accurate e.g., price of anarchy goes to infinity with degree bound, but only linearly to get tight bounds, need "variational inequalities" see my ICM survey for details

Variational Inequality Claim: if f is a Nash flow and f* is feasible, then e fe • ce(fe) ≤ e f* • ce(fe) proof: use that Nash flow routes flow on shortest paths (w.r.t. costs ce(fe)) e

Pigou Bound Recall goal: want to show Pigou-like examples are always worst cases. Pigou bound: given set of cost functions (e.g., degree-d polys), largest POA in a network: two nodes, two links one function in given set one constant function constant = cost of fully congested top edge xd s t 1

Pigou Bound (Formally) Let S = a set of cost functions. e.g., polynomials with degree at most d, nonnegative coefficients Definition: the Pigou bound α(S) for S is: max max is over all choices of cost fns c in S, traffic rate r  0, flow y  0 r • c(r) xd y • c(y) + (r-y) • c(r) s t 1

Pigou Bound (Example) Let S = { c : c(x) = ax +b } [linear functions] Recall: the Pigou bound α(S) for S is: max max is over all choices of cost fns c in S, traffic rate r  0, flow y  0 choose c(x) = x; r = 1; y = 1/2  get 4/3 calculus: α(S) = 4/3 [d/ln d for deg-d polynomials] r • c(r) x y • c(y) + (r-y) • c(r) s t 1

Main Theorem (Formally) Theorem: [Roughgarden 02, Correa/Schulz/Stier Moses 03]: For every set S, for every selfish routing network G with cost functions in C, the POA in G is at most α(S). POA always maximized by Pigou-like examples That is, if f and f* are Nash + optimal flows in G, then C(f)/C(f*) ≤ α(S). example: POA ≤ 4/3 if G has affine cost fns

Proof of General Thm Let f and f* are Nash + optimal flows in G.

Proof of General Thm Let f and f* are Nash + optimal flows in G. Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe: α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] e e e e

Proof of General Thm Let f and f* are Nash + optimal flows in G. Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe: α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] Step 2: rearrange and sum over e: C(f*) = e f* • ce(f*) e e e e e e

Proof of General Thm Let f and f* are Nash + optimal flows in G. Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe: α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] Step 2: rearrange and sum over e: C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)] e e e e e e e

Proof of General Thm Let f and f* are Nash + optimal flows in G. Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe: α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] Step 2: rearrange and sum over e: C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)] Step 3: apply VI e e e e e e e  0

Proof of General Thm Let f and f* are Nash + optimal flows in G. Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe: α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] Step 2: rearrange and sum over e: C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S) Step 3: apply VI, done! e e e e e e =C(f)

Recap selfish routing: simple, basic routing game inefficient equilibria: Pigou + Braess examples price of anarchy: ratio of objective fn values of selfish + optimal outcomes potential functions: equilibria actually solving a related optimization problem immediate consequence for existence, uniqueness, and inefficiency of equilibria

Recap variational inequality: inequality based on "first-order condition" satisfied by equilibria Pigou bound: given a set of cost functions, largest POA in a Pigou-like example main result: for every set of cost fns, Pigou bound is tight (all multicommodity networks) POA depends only on complexity of cost functions, not on complexity of network structure

Outline Part I: The Price of Anarchy in Selfish Routing Games Part II: The Price of Stability in Network Connectivity Games

Selfish Network Design Given: G = (V,E), fixed costs ce for all e є E, k vertex pairs (si,ti) Each player wants to build a network in which its nodes are connected. Player strategy: select a path connecting si to ti. [Anshelevich et al 04]

Shapley Cost Sharing How should multiple players on a single edge split costs? Natural choice is fair sharing, or Shapley cost sharing: Players using e pay for it evenly: ci(P) = Σ ce/ke Each player tries to minimize its cost. e є P

Comparison to Selfish Routing Note: like selfish routing, except: finite number of outcomes in selfish routing, outcomes = fractional flows positive (not negative) externalities cost function (per player) = ce/ke Objective: C = Σi ci(Pi) = Σ ce where S = union of Pi's e є S

What's the POA? Example: t1, t2, … tk t 1+ k s s1, s2, … sk

What's the POA? Example: t1, t2, … tk t t 1+ k 1+ k s s s1, s2, … sk OPT (also Nash eq)

What's the POA? Example: t1, t2, … tk t t t 1+ k 1+ k 1+ k s s s s1, s2, … sk OPT (also Nash eq) another Nash eq

Multiple Equilibria Moral: in Shapley network design games, different Nash eq can have different costs. Recall: Note: not well defined if Nash eq not unique. which one do we look at? POA of a game obj fn value of selfish outcome = optimal obj fn value

The Price of Stability General definition of POA: [KP99] POA = k in last example, uninteresting cost(worst NE) cost(OPT) Price of Anarchy =

The Price of Stability Alternative: General definition of POA: [KP99] POA = k in last example, uninteresting Alternative: POS = 1 in last example cost(worst NE) cost(OPT) Price of Anarchy = cost(best NE) cost(OPT) Price of Stability =

The Price of Stability Note: small price of stability only guarantees that some Nash eq has low cost. much weaker guarantee than small POA Interpretation: best solution consistent with self-interested players natural outcome for centralized planner to suggest [e.g., network protocol designer]

Example: High Price of Stability 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability cost(OPT) = 1+ε t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability cost(OPT) = 1+ε …but not a NE: player k pays (1+ε)/k, could pay 1/k t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability so player k would deviate t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability now player k-1 pays (1+ε)/(k-1), could pay 1/(k-1) t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability so player k-1 deviates too t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k

Example: High Price of Stability Continuing this process, all players defect. This is a NE! (the only Nash) cost = 1 + + … + t 1 1 1 1 1 2 3 k k-1 . . . 1+ 1 2 3 k-1 k 1 1 2 k Price of Stability is Hk = Θ(log k)!

The Price of Stability of Selfish Network Design Thus: the price of stability of selfish network design can be as high as ln k. [k = # players] Our goals: in all such games, there is at least one pure-strategy Nash eq one of them has cost ≤ ln k • OPT i.e. price of stability always ≤ ln k [Anshelevich et al 04] Technique: potential function method.

Potential Functions Recall: potential function Փ of a game = function optimized by selfish players not necessarily a natural objective function Defn: Փ (fn from outcomes to reals) is a potential function if for all outcomes S, players i, and deviations by i from S: ΔՓ = Δci

Potential Functions So: potential fn tracks deviations by players Thus: equilibria of game = local optima of Փ so finite potential games have pure-strategy Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96] precursors: [Rosenthal 73], [Beckmann et al 56]

Potential Functions So: potential fn tracks deviations by players Thus: equilibria of game = local optima of Փ so finite potential games have pure-strategy Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96] precursors: [Rosenthal 73], [Beckmann et al 56] Claim: every Shapley network design game has a potential function.

Proof of Potential Function Define Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke] where ke is # players using e in S. Hk Let Ф(S) = Σ Фe(S) Consider some solution S (a path for each player). Suppose player i is unhappy and decides to deviate. What happens to Ф(S)? e e є S

Proof of Potential Function Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke] Suppose player i’s new path includes e. i pays ce/(ke+1) to use e. Фe(S) increases by the same amount. If player i leaves an edge e’, Фe’(S) exactly reflects the change in i’s payment. ce[1+ 1/2 +… +1/ke] e i e’ ce’[1+ 1/2 +… +1/ke’]

Proof of Potential Function Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke] Suppose player i’s new path includes e. i pays ce/(ke+1) to use e. Фe(S) increases by the same amount. If player i leaves an edge e’, Фe’(S) exactly reflects the change in i’s payment. ce[1+ 1/2 +… +1/ke]+ce/(ke+1) e i e’ ce’[1+ 1/2 +… +1/ke’] -ce’/ke’

Bound on Price of Stability Compare cost + potential function: C(S) = e ce PF(S) = e ce[1+ 1/2 + 1/3 + … 1/ke] cost, potential fn differ by factor of ≤ Hk gives upper bound of Hk on price on stability let S = min-potential soln [note: also a Nash eq] let S* = opt solution C(S) ≤ PF(S) ≤ PF(S*) ≤ Hk • C(S*)

Undirected Networks Open Question: what is the POS in undirected graphs? best known lower bound = 12/7 [Fiat et al 06]: O(log log k) for special case 1 k 2 3 = t 1+ . . . k-1

Shapley Cost-Sharing Summary: with Shapley cost sharing, POA = k, even in undirected graphs POS = Hk in directed graphs (unknown in undirected graphs) Question #1: can we do better? Question #2: subject to what?

In Defense of Shapley Essential properties: (non-negotiable) "budget-balanced" (total cost shares = cost) "local" (cost shares computed edge-by-edge) pure-strategy Nash equilibria exist Bonus good properties: (negotiable) "uniform" (same definition for all networks) "fair" (characterizes Shapley)

Other Cost Shares? Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs. Shapley justified on efficiency grounds! non-uniform schemes not well understood

Other Cost Shares? Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs. Shapley justified on efficiency grounds! non-uniform schemes not well understood Theorem: [Chen/Roughgarden/Valiant 07] Can do much better in undirected graphs. can get POA = O(log2 k) better for special cases or non-uniform protocols

Wrap-Up network games arise in many CS applications price of anarchy/stability/etc a flexible tool to measure inefficiency of selfish behavior future direction: inform protocol design potential functions are an easy-to-use, versatile techniques to bound POA/POS many open questions... looking forward to future theorems from you!