1. Potential Functions and the Inefficiency of Equilibria
Tim Roughgarden
Stanford University

2. Pigou's Example Example: one unit of traffic wants to go from s to t
Question: what will selfish network users do?
assume everyone wants smallest-possible cost
[Pigou 1920]
cost depends on congestion
c(x)=x s t
c(x)=1
no congestion effects

3. Motivating Example Claim: all traffic will take the top link. Reason:
Є > 0  traffic on bottom is envious
Є = 0  equilibrium
all traffic incurs one unit of cost
Flow = 1-Є
c(x)=x s t
c(x)=1
this flow is envious!
Flow = Є

4. Can We Do Better? Consider instead: traffic split equally Improvement:
half of traffic has cost 1 (same as before)
half of traffic has cost ½ (much improved!)
Flow = ½
c(x)=x s t
c(x)=1
Flow = ½

5. Braess’s Paradox
Initial Network: s t x 1
Cost = 1.5

6. Braess’s Paradox Initial Network: Augmented Network: s t x 1 ½ ½ ½ x 1s t 1 x
Cost = 1.5
Now what?

7. Braess’s Paradox Initial Network: Augmented Network: s t x 1 ½ x 1 s ts t 1 x
Cost = 1.5
Cost = 2

8. Braess’s Paradox Initial Network: Augmented Network:
All traffic incurs more cost! [Braess 68]
also has physical analogs [Cohen/Horowitz 91] s t x 1 x 1 s t 1 x
Cost = 1.5
Cost = 2

9. High-Level Overview
Motivation: equilibria of noncooperative network games typically inefficient
e.g., Pigou's example + Braess's Paradox
don't optimize natural objective functions
Price of anarchy: quantify inefficiency w.r.t some objective function
Our goal: when is the price of anarchy small?
when does competition approximate cooperation?
benefit of centralized control is small

10. Selfish Routing Games directed graph G = (V,E)
source-destination pairs (s1,t1), …, (sk,tk)
ri = amount of traffic going from si to ti
for each edge e, a cost function ce(•)
assumed continuous and nondecreasing
Examples: (r,k=1)
c(x)=x
c(x)=x
c(x)=1
c(x)=0 s1 t1 s1 t1
c(x)=1
c(x)=1
c(x)=x

11. Outcomes = Network Flows
Possible outcomes of a selfish routing game:
fP = amount of traffic choosing si-ti path P
outcomes of game flow vectors f
flow vector: nonnegative and total flow  fP on si-ti paths equals traffic rate ri (for all i) s t

12. Outcomes = Network Flows
Possible outcomes of a selfish routing game:
fP = amount of traffic choosing si-ti path P
outcomes of game flow vectors f
flow vector: nonnegative and total flow  fP on si-ti paths equals traffic rate ri (for all i)
Question: What are the equilibria (natural selfish outcomes) of this game? s t

13. Nash Flows
Def: [Wardrop 52] A flow is at Nash equilibrium (or is a Nash flow) if no one can switch to a path of smaller cost. I.e., all flow is routed on min-cost paths. [given current edge congestion]
Examples: 1 x x s t s t 1 1 x 1 x 1 1 s t s t 1 x 1 x

14. Our Objective Function
Definition of social cost: total cost C(f) incurred by the traffic in a flow f.
Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:
C(f) = P fP • cP(f) s t

15. Our Objective Function
Definition of social cost: total cost C(f) incurred by the traffic in a flow f.
Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:
C(f) = P fP • cP(f)
Example: s t x s t
Cost = ½•½ +½•1 = ¾ 1

16. The Price of Anarchy Defn: price of anarchy of a game
definition from [Koutsoupias/Papadimitriou 99]
price of
anarchy of a game
obj fn value of selfish outcome =
optimal obj fn value
Example: POA = 4/3 in Pigou's example 1 x x s t s t 1 1
Cost = 3/4
Cost = 1

17. A Nonlinear Pigou Network
Bad Example: (d large)
equilibrium has cost 1, min cost  0 s t xd 1
1-Є Є

18. A Nonlinear Pigou Network
Bad Example: (d large)
equilibrium has cost 1, min cost  0
 price of anarchy unbounded as d -> infinity
Goal: weakest-possible conditions under which P.O.A. is small. s t xd 1
1-Є Є

19. When Is the Price of Anarchy Bounded?
Examples so far:
Hope: imposing additional structure on the cost functions helps
worry: bad things happen in larger networks s t x 1 xd x 1 s t s t 1 1 x

20. Polynomial Cost Functions
Def: linear cost fn is of form ce(x)=aex+be
Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:
≤ 4/3 ×
cost of Nash flow
cost of opt flow s t x 1

21. Polynomial Cost Functions
Def: linear cost fn is of form ce(x)=aex+be
Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:
≤ 4/3 ×
Bounded-deg polys: (w/nonneg coeffs) replace 4/3 by Θ(d/log d)
cost of Nash flow
cost of opt flow s t x 1 xd
tight
example s t 1

22. A General Theorem
Thm: [Roughgarden 02], [Correa/Schulz/Stier Moses 03] fix any set of cost fns. Then, a Pigou-like example 2 nodes, 2 links, 1 link w/constant cost fn) achieves worst POA s t xd 1
tight
example

23. Interpretation
Bad news: inefficiency of selfish routing grows as cost functions become "more nonlinear".
think of "nonlinear" as "heavily congested"
recall nonlinear Pigou's example
Good news: inefficiency does not grow with network size or # of source-destination pairs.
in lightly loaded networks, no matter how large, selfish routing is nearly optimal s t xd 1
tight
example

24. Benefit of Overprovisioning
Suppose: network is overprovisioned by β > 0 (β fraction of each edge unused).
Then: Price of anarchy is at most ½(1+1/√β).
arbitrary network size/topology, traffic matrix
Moral: Even modest (10%) over-provisioning sufficient for near-optimal routing.

25. Potential Functions
potential games: equilibria are actually optima of a related optimization problem
has immediate consequences for existence, uniqueness, and inefficiency of equilibria
see [Beckmann/McGuire/Winsten 56], [Rosenthal 73], [Monderer/Shapley 96], for original references
see [Roughgarden ICM 06] for survey

26. The Potential Function
Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f ce(x)dx (over all flows).
ce(fe) e fe

27. The Potential Function
Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f ce(x)dx (over all flows).
Lemma 1: locally optimal solutions are precisely the Nash flows (derivative test).
Lemma 2: all locally optimal solutions are also globally optimal (convexity).
Corollary: Nash flows exist, are unique.
ce(fe) e fe

28. Consequences for the Price of Anarchy
Example: linear cost functions.
Compare cost + potential function:
C(f) = e fe • ce(fe) = e [ae fe + be fe]
PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe] 2 2 e

29. Consequences for the Price of Anarchy
Example: linear cost functions.
Compare cost + potential function:
C(f) = e fe • ce(fe) = e [ae fe + be fe]
PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]
cost, potential fn differ by factor of ≤ 2
gives upper bound of 2 on price on anarchy
C(f) ≤ 2×PF(f) ≤ 2×PF(f*) ≤ 2×C(f*) 2 2 e

30. Better Bounds?
Similarly: proves bound of d+1 for degree-d polynomials (w/nonnegative coefficients).
not tight, but qualitatively accurate
e.g., price of anarchy goes to infinity with degree bound, but only linearly
to get tight bounds, need "variational inequalities"
see my ICM survey for details

31. Variational Inequality
Claim:
if f is a Nash flow and f* is feasible, then
e fe • ce(fe) ≤ e f* • ce(fe)
proof: use that Nash flow routes flow on shortest paths (w.r.t. costs ce(fe)) e

32. Pigou Bound
Recall goal: want to show Pigou-like examples are always worst cases.
Pigou bound: given set of cost functions (e.g., degree-d polys), largest POA in a network:
two nodes, two links
one function in given set
one constant function
constant = cost of fully congested top edge xd s t 1

33. Pigou Bound (Formally)
Let S = a set of cost functions.
e.g., polynomials with degree at most d, nonnegative coefficients
Definition: the Pigou bound α(S) for S is:
max
max is over all choices of cost fns c in S, traffic rate r  0, flow y  0
r • c(r) xd
y • c(y) + (r-y) • c(r) s t 1

34. Pigou Bound (Example) Let S = { c : c(x) = ax +b } [linear functions]
Recall: the Pigou bound α(S) for S is:
max
max is over all choices of cost fns c in S, traffic rate r  0, flow y  0
choose c(x) = x; r = 1; y = 1/2  get 4/3
calculus: α(S) = 4/3 [d/ln d for deg-d polynomials]
r • c(r) x
y • c(y) + (r-y) • c(r) s t 1

35. Main Theorem (Formally)
Theorem: [Roughgarden 02, Correa/Schulz/Stier Moses 03]: For every set S, for every selfish routing network G with cost functions in C, the POA in G is at most α(S).
POA always maximized by Pigou-like examples
That is, if f and f* are Nash + optimal flows in G, then C(f)/C(f*) ≤ α(S).
example: POA ≤ 4/3 if G has affine cost fns

36. Proof of General Thm
Let f and f* are Nash + optimal flows in G.

37. Proof of General Thm Let f and f* are Nash + optimal flows in G.
Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe:
α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)] e e e e

38. Proof of General Thm Let f and f* are Nash + optimal flows in G.
Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe:
α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)]
Step 2: rearrange and sum over e:
C(f*) = e f* • ce(f*) e e e e e e

39. Proof of General Thm Let f and f* are Nash + optimal flows in G.
Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe:
α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)]
Step 2: rearrange and sum over e:
C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S) [e (f* - fe) • ce(fe)] e e e e e e e

40. Proof of General Thm Let f and f* are Nash + optimal flows in G.
Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe:
α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)]
Step 2: rearrange and sum over e:
C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S) [e (f* - fe) • ce(fe)]
Step 3: apply VI e e e e e e e
 0

41. Proof of General Thm Let f and f* are Nash + optimal flows in G.
Step 1: for each e, invoke Pigou bound with c = ce, y = f*, r = fe:
α(S)  fe • ce(fe)/[f* • ce(f*) + (fe -f* ) • ce(fe)]
Step 2: rearrange and sum over e:
C(f*) = e f* • ce(f*)  [e fe • ce(fe)]/α(S)
Step 3: apply VI, done! e e e e e e
=C(f)

42. Recap selfish routing: simple, basic routing game
inefficient equilibria: Pigou + Braess examples
price of anarchy: ratio of objective fn values of selfish + optimal outcomes
potential functions: equilibria actually solving a related optimization problem
immediate consequence for existence, uniqueness, and inefficiency of equilibria

43. Recap
variational inequality: inequality based on "first-order condition" satisfied by equilibria
Pigou bound: given a set of cost functions, largest POA in a Pigou-like example
main result: for every set of cost fns, Pigou bound is tight (all multicommodity networks)
POA depends only on complexity of cost functions, not on complexity of network structure

44. Outline Part I: The Price of Anarchy in Selfish Routing Games
Part II: The Price of Stability in Network Connectivity Games

45. Selfish Network Design
Given: G = (V,E),
fixed costs ce for all e є E,
k vertex pairs (si,ti)
Each player wants to build a network in which its nodes are connected.
Player strategy: select a path connecting si to ti.
[Anshelevich et al 04]

46. Shapley Cost Sharing How should multiple players
on a single edge split costs?
Natural choice is fair sharing,
or Shapley cost sharing:
Players using e pay for it evenly: ci(P) = Σ ce/ke
Each player tries to minimize its cost.
e є P

47. Comparison to Selfish Routing
Note: like selfish routing, except:
finite number of outcomes
in selfish routing, outcomes = fractional flows
positive (not negative) externalities
cost function (per player) = ce/ke
Objective: C = Σi ci(Pi) = Σ ce
where S = union of Pi's
e є S

48. What's the POA?
Example:
t1, t2, … tk t
1+ k s
s1, s2, … sk

49. What's the POA? Example: t1, t2, … tk t t 1+ k 1+ k s s s1, s2, … sk
OPT
(also Nash eq)

50. What's the POA? Example: t1, t2, … tk t t t 1+ k 1+ k 1+ k s s s
s1, s2, … sk
OPT
(also Nash eq)
another
Nash eq

51. Multiple Equilibria
Moral: in Shapley network design games, different Nash eq can have different costs.
Recall:
Note: not well defined if Nash eq not unique.
which one do we look at?
POA of a game
obj fn value of selfish outcome =
optimal obj fn value

52. The Price of Stability General definition of POA: [KP99]
POA = k in last example, uninteresting
cost(worst NE)
cost(OPT)
Price of Anarchy =

53. The Price of Stability Alternative: General definition of POA: [KP99]
POA = k in last example, uninteresting
Alternative:
POS = 1 in last example
cost(worst NE)
cost(OPT)
Price of Anarchy =
cost(best NE)
cost(OPT)
Price of Stability =

54. The Price of Stability
Note: small price of stability only guarantees that some Nash eq has low cost.
much weaker guarantee than small POA
Interpretation: best solution consistent with self-interested players
natural outcome for centralized planner to suggest [e.g., network protocol designer]

55. Example: High Price of Stability1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

56. Example: High Price of Stability
cost(OPT) = 1+ε t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

57. Example: High Price of Stability
cost(OPT) = 1+ε
…but not a NE:
player k
pays (1+ε)/k,
could pay 1/k t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

58. Example: High Price of Stability
so player k
would deviate t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

59. Example: High Price of Stability
now player k-1
pays (1+ε)/(k-1),
could pay 1/(k-1) t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

60. Example: High Price of Stability
so player k-1
deviates too t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k

61. Example: High Price of Stability
Continuing this process, all players defect.
This is a NE!
(the only Nash)
cost = … + t 1 1 1 1 1 2 3 k
k-1
. . .
1+ 1 2 3
k-1 k k
Price of Stability is Hk = Θ(log k)!

62. The Price of Stability of Selfish Network Design
Thus: the price of stability of selfish network design can be as high as ln k. [k = # players]
Our goals: in all such games,
there is at least one pure-strategy Nash eq
one of them has cost ≤ ln k • OPT
i.e. price of stability always ≤ ln k
[Anshelevich et al 04]
Technique: potential function method.

63. Potential Functions
Recall: potential function Փ of a game = function optimized by selfish players
not necessarily a natural objective function
Defn: Փ (fn from outcomes to reals) is a potential function if for all outcomes S, players i, and deviations by i from S:
ΔՓ = Δci

64. Potential Functions So: potential fn tracks deviations by players
Thus: equilibria of game = local optima of Փ
so finite potential games have pure-strategy Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]
precursors: [Rosenthal 73], [Beckmann et al 56]

65. Potential Functions So: potential fn tracks deviations by players
Thus: equilibria of game = local optima of Փ
so finite potential games have pure-strategy Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]
precursors: [Rosenthal 73], [Beckmann et al 56]
Claim: every Shapley network design game has a potential function.

66. Proof of Potential Function
Define Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
where ke is # players using e in S Hk
Let Ф(S) = Σ Фe(S)
Consider some solution S (a path for each player).
Suppose player i is unhappy and decides to deviate.
What happens to Ф(S)? e
e є S

67. Proof of Potential Function
Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
Suppose player i’s new path includes e.
i pays ce/(ke+1) to use e.
Фe(S) increases by the same amount.
If player i leaves an edge e’,
Фe’(S) exactly reflects the change in i’s payment.
ce[1+ 1/2 +… +1/ke] e i e’
ce’[1+ 1/2 +… +1/ke’]

68. Proof of Potential Function
Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
Suppose player i’s new path includes e.
i pays ce/(ke+1) to use e.
Фe(S) increases by the same amount.
If player i leaves an edge e’,
Фe’(S) exactly reflects the change in i’s payment.
ce[1+ 1/2 +… +1/ke]+ce/(ke+1) e i e’
ce’[1+ 1/2 +… +1/ke’] -ce’/ke’

69. Bound on Price of Stability
Compare cost + potential function:
C(S) = e ce
PF(S) = e ce[1+ 1/2 + 1/3 + … 1/ke]
cost, potential fn differ by factor of ≤ Hk
gives upper bound of Hk on price on stability
let S = min-potential soln [note: also a Nash eq]
let S* = opt solution
C(S) ≤ PF(S) ≤ PF(S*) ≤ Hk • C(S*)

70. Undirected Networks
Open Question: what is the POS in undirected graphs?
best known lower bound = 12/7
[Fiat et al 06]: O(log log k) for special case 1 k 2 3 = t
1+
. . .
k-1

71. Shapley Cost-Sharing Summary: with Shapley cost sharing,
POA = k, even in undirected graphs
POS = Hk in directed graphs
(unknown in undirected graphs)
Question #1: can we do better?
Question #2: subject to what?

72. In Defense of Shapley Essential properties: (non-negotiable)
"budget-balanced" (total cost shares = cost)
"local" (cost shares computed edge-by-edge)
pure-strategy Nash equilibria exist
Bonus good properties: (negotiable)
"uniform" (same definition for all networks)
"fair" (characterizes Shapley)

73. Other Cost Shares?
Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.
Shapley justified on efficiency grounds!
non-uniform schemes not well understood

74. Other Cost Shares?
Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.
Shapley justified on efficiency grounds!
non-uniform schemes not well understood
Theorem: [Chen/Roughgarden/Valiant 07] Can do much better in undirected graphs.
can get POA = O(log2 k)
better for special cases or non-uniform protocols

75. Wrap-Up network games arise in many CS applications
price of anarchy/stability/etc a flexible tool to measure inefficiency of selfish behavior
future direction: inform protocol design
potential functions are an easy-to-use, versatile techniques to bound POA/POS
many open questions...
looking forward to future theorems from you!