MS&E 211, Lecture 11 The dual of Min-Cost Flow Ashish Goel

Taking stock Today: Max-flow Min-Cut Next eight lectures (with possible order changes): – Zero sum games – Interior Point Method – Introduction to convex and quadratic optimization – Two examples from finance – Support vector machines – Gittins index and dynamic programming – Lagrange multipliers – Semi-definite and conic programming

Recap: The MCF

The MCF: A Useful Trick Assume that the sum of all the demands = 0 (Necessary for the problem to be feasible)

The MCF: A Useful Trick

≤ ≤

≤ ≤ - Maximize -

The MCF: A Useful Trick NOW, THE LP IS IN A STANDARD PRODUCTION FORM

Adding Labels to the MCF

The Dual of MCF PRIMAL DUAL

Coefficient of variable z(p) in constraint x(v,w) in Dual? PRIMAL The Transpose rule of thumb: Same as the coefficient of variable x(v,w) in constraint z(p) in primal p=v : x(v,w) appears as part of OUT x (p), i.e. with coefficient 1 p=w : x(v,w) appears as part of IN x (p), i.e. with coefficient -1 p≠v, p≠w : x(v,w) does not appear in z(p), i.e. occurs with coefficient 0

Coefficient of variable z(p) in constraint x(v,w) in Dual? PRIMAL The Transpose rule of thumb: Same as the coefficient of variable x(v,w) in constraint z(p) in primal p=v : x(v,w) appears as part of OUT x (p), i.e. with coefficient 1 p=w : x(v,w) appears as part of IN x (p), i.e. with coefficient -1 p≠v, p≠w : x(v,w) does not appear in z(p), i.e. occurs with coefficient 0

Coefficient of variable z(p) in constraint x(v,w) in Dual? PRIMAL The Transpose rule of thumb: Same as the coefficient of variable x(v,w) in constraint z(p) in primal p=v : x(v,w) appears as part of OUT x (p), i.e. with coefficient 1 p=w : x(v,w) appears as part of IN x (p), i.e. with coefficient -1 p≠v, p≠w : x(v,w) does not appear in z(p), i.e. occurs with coefficient 0

Coefficient of variable z(p) in constraint x(v,w) in Dual? PRIMAL The Transpose rule of thumb: Same as the coefficient of variable x(v,w) in constraint z(p) in primal p=v : x(v,w) appears as part of OUT x (p), i.e. with coefficient 1 p=w : x(v,w) appears as part of IN x (p), i.e. with coefficient -1 p≠v, p≠w : x(v,w) does not appear in z(p), i.e. occurs with coefficient 0

The Dual of MCF PRIMAL DUAL

Coefficient of variable y(q,r) in constraint x(v,w) in Dual? PRIMAL The Transpose rule of thumb: Same as the coefficient of variable x(v,w) in constraint y(q,r) in primal (q,r)=(v,w) : x(v,w) appears in y(q,r) with coefficient 1 (q,r)≠ (v,w): x(v,w) does not appear in y(q,r), i.e. occurs with coefficient 0

The Dual of MCF PRIMAL DUAL

The Dual of MCF PRIMAL DUAL FLIP SIGNS

The Dual of MCF PRIMAL DUAL

A Lot of Work!! Why?

A Lot of Work!! Why? We can now apply this to many problems Next module: Dual of Shortest Path Problem

The Dual of MCF PRIMAL DUAL

Applying to Shortest Path Problem The constraints x(v,w) ≤ u(v,w) don’t exist in the primal ! => No variables y(v,w) in the Dual !! d(s) = -1; d(t) = 1; all other d(v)’s are 0 !!! DUAL

Dual for the Shortest Path Problem Really Simple. Informal Interpretation: z(v) – z(s) = Shortest path distance from s to v

An Algorithmic Interpretation Ford’s Algorithm – Initialize: Set z(s) = 0; z(v) = 1 for all v  s – Repeat: While there exists an edge (v,w) which violates the constraint, set z(w) = z(v) + c(v,w) [Known as relaxation] Intuition: z(v) is the shortest path distance from s to v Most (perhaps all) popular shortest path algorithms are special cases of Ford’s algorithm (if multiple edges violate the constraint, which one to choose)

Dual Complementary Slackness for Shortest Path Example Let x,z be optimum solutions to primal, dual respectively Either z(w) = z(v) + c(v,w) or x(v,w) = 0 Informal Interpretation: Either there exists shortest path to w that goes via v or the flow on edge (v,w) is zero

Example s t p q r What is the value of z(t) – z(s) in any optimum solution of the dual for the shortest path problem?

Example s t p q r One optimum solution to the dual is z(s) = 0 z(p) = 4 z(q) = 3 z(r) = 5 z(t) = 6 Using dual complementary slackness, for which of the following edges can we conclude that the flow in an optimum solution to the primal shortest path is 0? (a) (s,p) (b) (p,s) (c) (s, q) (d) (r,t)

Example s t p q r One optimum solution to the dual is z(s) = 0 z(p) = 4 z(q) = 3 z(r) = 5 z(t) = 6 In fact, we can conclude that edges (p,s), (p,t), and (q,s) can not have positive flow in the primal

Example s t p q r One optimum solution to the dual is z(s) = 0 z(p) = 4 z(q) = 3 z(r) = 5 z(t) = 6 In fact, we can conclude that edges (p,s), (p,t), and (q,s) can not have positive flow in the primal INTRIGUING FACT: The remaining edges form a shortest path tree! Not totally a coincidence, since this corresponds to our informal interpretation, but we will not develop this further in this class. INTRIGUING FACT: The remaining edges form a shortest path tree! Not totally a coincidence, since this corresponds to our informal interpretation, but we will not develop this further in this class.

Applying to Max-Flow Problem The values d(v) and c(v,w) are 0 for max-flow (except for the fake edge, which does not have a capacity constraint) DUAL

Applying to Max-Flow Problem The values d(v) and c(v,w) are 0 for max-flow (except for the fake edge, which does not have a capacity constraint and does not need x t,s ¸ 0) DUAL

The Dual of Max-Flow The values y(v,w) must be 0 or 1 (increasing the capacity of an edge by a small amount ± increases the max-flow by 0 or ±

The Dual of Max-Flow The values y(v,w) must be 0 or 1 (increasing the capacity of an edge by a small amount ± increases the max-flow by 0 or ±  Similarly, z must be either 0 or 1 [Proof omitted]  We have the “min-cut” problem: Find the set of edges of smallest capacity which “cuts” the source from the sink, i.e., removes all paths from source to sink

Theorem: Max-flow = Min-Cut The maximum flow you can send from s to t in a network = The minimum s-t in the same network [Minimum s-t cut is the smallest total capacity of any s-t cut] [An s-t cut is a set of edges such that removing these edges from the network will result in no path remaining from s to t]

2 1 Sin k Sourc e Yinyu Ye, Stanford, MS&E211 Lecture Notes #8 36 Look at Another Max-Flow Problem 3

Let x ij be the flow rate from node i to node j. Then the problem can be formulated as max x 41 x 21 + x 31 + x 41 − x 12 − x 13 x 12 + x 32 + x 42 − x 21 − x 23 − x 24 x 13 + x 23 + x 43 − x 31 − x 32 − x 34 x 24 + x 34 − x 41 − x 42 − x 43 x ij z 1 z 2 z 3 z 4 y ij s.t. = 0, ≤ u ij, ∀ (i, j) ∈ E, ≥ 0, ∀ (i, j) ∈ E. 37 The Primal Formulation Corresponding Dual variables

38 The Dual of Max-Flow: the Min-Cut Problem min ∑ (i,j) ∈ A u ij y ij s.t. z 4 − z 1 = 1, −z 2 + z 1 + y 12 ≥ 0, −z 3 + z 1 + y 13 ≥ 0,... −z 4 + z 2 + y 24 ≥ 0, −z 4 + z 3 + y 34 ≥ 0, y,z ≥ 0. x 41 x 12 x 13 … x 24 x 34 1 if i is on the sink side 1, if z i =0 and z j =1 z i = and y ij = 0 if i is on the source side 0 otherwise Corresponding Primal variables

Yinyu Ye, Stanford, MS&E211 Lecture Notes #8 39 The Min-Cut Solution: Min-Cut Value=8 y 24 = 1 1 Sin k Sourc e Z 4 =1 Z 3 =1 Z 2 =0 z 1 =0 y 23 = 1 y 13 = 1 2 All other y ij =0

THANK YOU!!!