1. Taken from Kevin Wayne’s slides (Princeton University) COSC 3101A - Design and Analysis of Algorithms 13 Maximum Flow

2. 2 Contents Contents. n Maximum flow problem. n Minimum cut problem. n Max-flow min-cut theorem. n Augmenting path algorithm. n Capacity-scaling. n Shortest augmenting path.

3. 3 n Network reliability. n Security of statistical data. n Distributed computing. n Egalitarian stable matching. n Distributed computing. n Many many more... Maximum Flow and Minimum Cut Max flow and min cut. n Two very rich algorithmic problems. n Cornerstone problem in combinatorial optimization. n Beautiful mathematical duality. Nontrivial applications / reductions. n Network connectivity. n Bipartite matching. n Data mining. n Open-pit mining. n Airline scheduling. n Image processing. n Project selection. n Baseball elimination.

4. 4 Max flow network: G = (V, E, s, t, u). n (V, E) = directed graph, no parallel arcs. n Two distinguished nodes: s = source, t = sink. n u(e) = capacity of arc e. Max Flow Network Capacity s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4

5. 5 An s-t flow is a function f: E   that satisfies: n For each e  E: 0  f(e)  u(e) (capacity) n For each v  V – {s, t}: (conservation) Flows s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 4 0 0 0 0 0 0 0 4 4 0 4 0 0 0 Capacity Flow

6. 6 An s-t flow is a function f: E   that satisfies: n For each e  E: 0  f(e)  u(e) (capacity) n For each v  V – {s, t}: (conservation) MAX FLOW: find s-t flow that maximizes net flow out of the source. Flows s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 4 0 0 0 0 0 0 0 4 4 0 4 0 Value = 4 0 0 Capacity Flow

7. 7 An s-t flow is a function f: E   that satisfies: n For each e  E: 0  f(e)  u(e) (capacity) n For each v  V – {s, t}: (conservation) MAX FLOW: find s-t flow that maximizes net flow out of the source. Flows s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 10 6 6 11 1 10 3 8 8 0 4 0 Value = 24 0 0 Capacity Flow

8. 8 An s-t flow is a function f: E   that satisfies: n For each e  E: 0  f(e)  u(e) (capacity) n For each v  V – {s, t}: (conservation) MAX FLOW: find s-t flow that maximizes net flow out of the source. Flows s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 10 9 9 14 4 10 4 8 9 1 0 0 Value = 28 0 0 Capacity Flow

9. 9 Networks communication Network telephone exchanges, computers, satellites NodesArcs cables, fiber optics, microwave relays Flow voice, video, packets circuits gates, registers, processors wirescurrent mechanicaljointsrods, beams, springsheat, energy hydraulic reservoirs, pumping stations, lakes pipelinesfluid, oil financialstocks, currencytransactionsmoney transportation airports, rail yards, street intersections highways, railbeds, airway routes freight, vehicles, passengers chemicalsitesbondsenergy

10. 10 An s-t cut is a node partition (S, T) such that s  S, t  T. n The capacity of an s-t cut (S, T) is: Min s-t cut: find an s-t cut of minimum capacity. Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 30

11. 11 An s-t cut is a node partition (S, T) such that s  S, t  T. n The capacity of an s-t cut (S, T) is: Min s-t cut: find an s-t cut of minimum capacity. Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 62

12. 12 An s-t cut is a node partition (S, T) such that s  S, t  T. n The capacity of an s-t cut (S, T) is: Min s-t cut: find an s-t cut of minimum capacity. Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 28

13. 13 L1. Let f be a flow, and let (S, T) be a cut. Then, the net flow sent across the cut is equal to the amount reaching t. Flows and Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Value = 24 10 6 6 0 4 8 8 0 4 0 0 0

14. 14 10 6 6 0 4 8 8 0 4 0 0 0 L1. Let f be a flow, and let (S, T) be a cut. Then, the net flow sent across the cut is equal to the amount reaching t. Flows and Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Value = 24

15. 15 10 6 6 0 4 8 8 0 4 0 0 L1. Let f be a flow, and let (S, T) be a cut. Then, the net flow sent across the cut is equal to the amount reaching t. Flows and Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Value = 24 0

16. 16 Let f be a flow, and let (S, T) be a cut. Then, Proof by induction on |S|. n Base case: S = { s }. n Inductive hypothesis: assume true for |S| < k. – consider cut (S, T) with |S| = k – S = S'  { v } for some v  s, t, |S' | = k-1  cap(S', T') = | f |. – adding v to S' increase cut capacity by Flows and Cuts s v t After s v t S' Before S

17. 17 L2. Let f be a flow, and let (S, T) be a cut. Then, | f |  cap(S, T). Proof. Corollary. Let f be a flow, and let (S, T) be a cut. If |f| = cap(S, T), then f is a max flow and (S, T) is a min cut. Flows and Cuts s t S T 7 6 8 4

18. 18 Max Flow and Min Cut Corollary. Let f be a flow, and let (S, T) be a cut. If |f| = cap(S, T), then f is a max flow and (S, T) is a min cut. s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 10 9 9 15 4 10 4 8 9 1 0 0 Flow value = 28 0 0 Cut capacity = 28

19. 19 Max-Flow Min-Cut Theorem MAX-FLOW MIN-CUT THEOREM (Ford-Fulkerson, 1956): In any network, the value of the max flow is equal to the value of the min cut. n "Good characterization." n Proof IOU. s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 10 9 9 15 4 10 4 8 9 1 0 0 Flow value = 28 0 0 Cut capacity = 28

20. 20 Towards an Algorithm Find an s-t path where each arc has u(e) > f(e) and "augment" flow along the path. s 4 2 5 3t 4 0 0 0 0 0 0 4 0 4 4 Flow value = 0 10 13 10

21. 21 Towards an Algorithm Find an s-t path where each arc has u(e) > f(e) and "augment" flow along the path. n Repeat until you get stuck. s 4 2 5 3t 4 0 0 0 0 0 0 4 0 4 4 Flow value = 10 10 13 10 X X X

22. 22 Towards an Algorithm Find an s-t path where each arc has u(e) > f(e) and "augment" flow along the path. n Repeat until you get stuck. n Greedy algorithm fails. s 4 2 5 3t 10 13 10 4 0 0 0 4 0 4 4 Flow value = 10 s 4 2 5 3t 10 13 10 5 4 4 6 4 5 4 5 5 Flow value = 14

23. 23 Residual Arcs Original graph G = (V, E). n Flow f(e). n Arc e = (v, w)  E. Residual graph: G f = (V, E f ). n Residual arcs e = (v, w) and e R = (w, v). n "Undo" flow sent. vw 17 6 Capacity Flow vw 11 Residual capacity 6

24. 24 Residual Graph and Augmenting Paths Residual graph: G f = (V, E f ). n E f = {e : f(e) 0}. s 4 2 5 3t 10 13 10 4 0 0 0 4 0 4 4 s 4 2 5 3t 4 4 4 4 3 G GfGf

25. 25 Augmenting Path Augmenting path = path in residual graph. s 4 2 5 3t 10 13 10 4 0 0 0 4 0 4 4 s 4 2 5 3t 4 4 4 4 3 G GfGf 4 4 6 4 4 X X X X X

26. 26 Augmenting Path Augmenting path = path in residual graph. n Max flow  no augmenting paths ??? s 4 2 5 3t 10 13 10 4 4 4 6 4 4 4 4 4 s 4 2 5 3t 6 4 4 4 4 7 Flow value = 14 G GfGf

27. 27 Max-Flow Min-Cut Theorem Augmenting path theorem (Ford-Fulkerson, 1956): A flow f is a max flow if and only if there are no augmenting paths. MAX-FLOW MIN-CUT THEOREM (Ford-Fulkerson, 1956): the value of the max flow is equal to the value of the min cut. We prove both simultaneously by showing the TFAE: (i)f is a max flow. (ii)There is no augmenting path relative to f. (iii)There exists a cut (S, T) such that |f| = cap(S, T).

28. 28 Proof of Max-Flow Min-Cut Theorem We prove both simultaneously by showing the TFAE: (i)f is a max flow. (ii)There is no augmenting path relative to f. (iii)There exists a cut (S, T) such that |f| = cap(S, T). (i)  (ii) n We show contrapositive. n Let f be a flow. If there exists an augmenting path, then we can improve f by sending flow along path. (iii)  (i) n Next slide. (iii)  (i) n This was the Corollary to Lemma 2.

29. 29 Proof of Max-Flow Min-Cut Theorem We prove both simultaneously by showing the TFAE: (i)f is a max flow. (ii)There is no augmenting path relative to f. (iii)There exists a cut (S, T) such that |f| = cap(S, T). (ii)  (iii) n Let f be a flow with no augmenting paths. n Let S be set of vertices reachable from s in residual graph. – clearly s  S, and t  S by definition of f s t Original Network S T

30. 30 Augmenting Path Algorithm b  bottleneck(P) FOREACH e  P IF (e  E) // forward arc f(e)  f(e) + b ELSE // backwards arc f(e R )  f(e) - b RETURN f Augment (f, P) FOREACH e  E f(e)  0 G f  residual graph WHILE (there exists augmenting path P) f  augment(f, P) update G f RETURN f FordFulkerson (V, E, s, t)

31. 31 Running Time Assumption: all capacities are integers between 0 and U. Invariant: every flow value f(e) and every residual capacities u f (e) remains an integer throughout the algorithm. Theorem: the algorithm terminates in at most | f * |  nU iterations. Corollary: if U = 1, then algorithm runs in O(mn) time. Integrality theorem: if all arc capacities are integers, then there exists a max flow f for which every flow value f(e) is an integer. Note: algorithm may not terminate on some pathological instances (with irrational capacities). Moreover, flow value may not even converge to correct answer.

32. 32 100 0 0 s 4 2 t 1 0 0 0 Choosing Good Augmenting Paths Use care when selecting augmenting paths. 100

33. 33 100 0 0 s 4 2 t 1 0 0 0 Choosing Good Augmenting Paths Use care when selecting augmenting paths. 100 1 1 1 X X X

34. 34 100 1 0 s 4 2 t 1 1 0 Choosing Good Augmenting Paths Use care when selecting augmenting paths. 100 1

35. 35 100 1 0 s 4 2 t 1 1 1 0 Choosing Good Augmenting Paths Use care when selecting augmenting paths. 100 1 0 1 X X X

36. 36 s 4 2 t 1 100 0 1 11 1 Choosing Good Augmenting Paths Use care when selecting augmenting paths. 200 iterations possible.

37. 37 Choosing Good Augmenting Paths Use care when selecting augmenting paths. n Some choices lead to exponential algorithms. n Clever choices lead to polynomial algorithms. Goal: choose augmenting paths so that: n Can find augmenting paths efficiently. n Few iterations. Edmonds-Karp (1972): choose augmenting path with n Max bottleneck capacity.(fat path) n Sufficiently large capacity.(capacity-scaling) n Fewest number of arcs.(shortest path)

38. 38 Capacity Scaling Intuition: choosing path with highest bottleneck capacity increases flow by max possible amount. n Don't worry about finding exact highest bottleneck path. n Maintain scaling parameter . n Let G f (  ) be the subgraph of the residual graph consisting of only arcs with capacity at least . 110 s 4 2 t 1 170 102 122 GfGf 110 s 4 2 t 170 102 122 G f (100)

39. 39 Capacity Scaling Intuition: choosing path with highest bottleneck capacity increases flow by max possible amount. n Don't worry about finding exact highest bottleneck path. n Maintain scaling parameter . n Let G f (  ) be the subgraph of the residual graph consisting of only arcs with capacity at least . FOREACH e  E, f(e)  0   smallest power of 2 greater than or equal to U WHILE (   1) G f (  )   -residual graph WHILE (there exists augmenting path P in G f (  )) f  augment(f, P) update G f (  )    / 2 RETURN f ScalingMaxFlow(V, E, s, t)

40. 40 Capacity Scaling: Analysis L1. If all arc capacities are integers, then throughout the algorithm, all flow and residual capacity values remain integers. n Thus,  = 1  G f (  ) = G f, so upon termination f is a max flow. L2. The outer while loop repeats 1 +  log 2 U  times. n Initially U   < 2U, and  decreases by a factor of 2 each iteration. L3. Let f be the flow at the end of a  -scaling phase. Then value of the maximum flow is at most | f | + m . L4. There are at most 2m augmentations per scaling phase. n Let f be the flow at the end of the previous scaling phase. n L3  |f*|  | f | + m (2  ). n Each augmentation in a  -phase increases | f | by at least . Theorem. The algorithm runs in O(m 2 log (2U) ) time.

41. 41 Capacity Scaling: Analysis L3. Let f be the flow at the end of a  -scaling phase. Then value of the maximum flow is at most |f| + m . n We show that at the end of a  -phase, there exists a cut (S, T) such that cap(S, T)  | f | + m . n Choose S to be the set of nodes reachable from s in G f (  ). – clearly s  S, and t  S by definition of S s t Original Network S T

42. 42 Choosing Good Augmenting Paths Use care when selecting augmenting paths. n Some choices lead to exponential algorithms. n Clever choices lead to polynomial algorithms. Goal: choose augmenting paths so that: n Can find augmenting paths efficiently. n Few iterations. Edmonds-Karp (1972): choose augmenting path with n Max bottleneck capacity.(fat path) n Sufficiently large capacity.(capacity-scaling) n Fewest number of arcs.(shortest path)

43. 43 Shortest Augmenting Path Intuition: choosing path via breadth first search. n Easy to implement. – may implement by coincidence! n Finds augmenting path with fewest number of arcs. FOREACH e  E f(e)  0 G f  residual graph WHILE (there exists augmenting path) find such a path P by BFS f  augment(f, P) update G f RETURN f ShortestAugmentingPath(V, E, s, t)

44. 44 Shortest Augmenting Path: Overview of Analysis L1. Throughout the algorithm, the length of the shortest path never decreases. n Proof ahead. L2. After at most m shortest path augmentations, the length of the shortest augmenting path strictly increases. n Proof ahead. Theorem. The shortest augmenting path algorithm runs in O(m 2 n) time. n O(m+n) time to find shortest augmenting path via BFS. n O(m) augmentations for paths of exactly k arcs. n If there is an augmenting path, there is a simple one.  1  k < n  O(mn) augmentations.

45. 45 Shortest Augmenting Path: Analysis Level graph of (V, E, s). n For each vertex v, define (v) to be the length (number of arcs) of shortest path from s to v. n L G = (V, E G ) is subgraph of G that contains only those arcs (v,w)  E with (w) = (v) + 1. s 2 3 5 6t s 2 3 5 6t = 0 = 1 = 2 = 3 LG: LG: G:

46. 46 Shortest Augmenting Path: Analysis Level graph of (V, E, s). n For each vertex v, define (v) to be the length (number of arcs) of shortest path from s to v. n L = (V, F) is subgraph of G that contains only those arcs (v,w)  E with (w) = (v) + 1. n Compute in O(m+n) time using BFS, deleting back and side arcs. n P is a shortest s-v path in G if and only if it is an s-v path L. s 2 3 5 6t = 0 = 1 = 2 = 3 L:

47. 47 Shortest Augmenting Path: Analysis L1. Throughout the algorithm, the length of the shortest path never decreases. n Let f and f' be flow before and after a shortest path augmentation. n Let L and L' be level graphs of G f and G f ' n Only back arcs added to G f ' – path with back arc has length greater than previous length s 2 3 5 6t = 0 = 1 = 2 = 3 L s 2 3 5 6t L'

48. 48 Shortest Augmenting Path: Analysis L2. After at most m shortest path augmentations, the length of the shortest augmenting path strictly increases. n At least one arc (the bottleneck arc) is deleted from L after each augmentation. n No new arcs added to L until length of shortest path strictly increases. s 2 3 5 6t = 0 = 1 = 2 = 3 L s 2 3 5 6t L'

49. 49 Shortest Augmenting Path: Review of Analysis L1. Throughout the algorithm, the length of the shortest path never decreases. L2. After at most m shortest path augmentations, the length of the shortest augmenting path strictly increases. Theorem. The shortest augmenting path algorithm runs in O(m 2 n) time. n O(m+n) time to find shortest augmenting path via BFS. n O(m) augmentations for paths of exactly k arcs. n O(mn) augmentations. Note:  (mn) augmentations necessary on some networks. n Try to decrease time per augmentation instead. n Dynamic trees  O(mn log n) Sleator-Tarjan, 1983 n Simple idea  O(mn 2 )

50. 50 Shortest Augmenting Path: Improved Version Two types of augmentations. n Normal augmentation: length of shortest path doesn't change. n Special augmentation: length of shortest path strictly increases. L3. Group of normal augmentations takes O(mn) time. n Explicitly maintain level graph - it changes by at most 2n arcs after each normal augmentation. n Start at s, advance along an arc in L G until reach t or get stuck. – if reach t, augment and delete at least one arc – if get stuck, delete node s 2 3 5 6t = 0 = 1 = 2 = 3 LG LG

51. 51 Shortest Augmenting Path: Improved Version s 2 3 5 6t LG LG s 2 3 5 6t LG LG s 25 6t LG LG augment delete 3

52. 52 Shortest Augmenting Path: Improved Version s 25 6t LG LG s 25 6t LG LG STOP: length of shortest path must have strictly increased augment

53. 53 Shortest Augmenting Path: Improved Version ARRAY pred[v  V] L G  level graph of G f v  s, pred[v]  nil REPEAT WHILE (there exists (v,w)  L G ) pred[w]  v, v  w IF (v = t) P  path defined by pred[] f  augment(f, P) update L G v  s, pred[v]  nil delete v from L G UNTIL (v = s) RETURN f AdvanceRetreat(V, E, f, s, t) advance retreat augment

54. 54 Shortest Augmenting Path: Improved Version Two types of augmentations. n Normal augmentation: length of shortest path doesn't change. n Special augmentation: length of shortest path strictly increases. L3. Group of normal augmentations takes O(mn) time. n Explicitly maintain level graph - it changes by at most 2n arcs after each normal augmentation. n Start at s, advance along an arc in L G until reach t or get stuck. – if reach t, augment and delete at least one arc – if get stuck, delete node – at most n advance steps before one of above events Theorem. Algorithm runs in O(mn 2 ) time. n O(mn) time between special augmentations. n At most n special augmentations.

55. 55 Choosing Good Augmenting Paths: Summary First 4 rules assume arc capacities are between 0 and U. MethodAugmentations Augmenting pathnU Max capacitym log U Capacity scalingm log U Shortest pathmn Running time mnU m log U (m + n log n) m 2 log U m2nm2n Improved shortest pathmn mn 2 Improved capacity scalingm log Umn log U

56. 56 History Dantzig Discoverer Simplex MethodBig-Oh mn 2 U1951 Year Ford, FulkersonAugmenting pathmnU1955 Edmonds-KarpShortest pathm2nm2n1970 DinitzShortest path mn 2 1970 Edmonds-Karp, DinitzCapacity scalingm 2 log U1972 Dinitz-GabowCapacity scalingmn log U1973 KarzanovPreflow-push n3n3 1974 Sleator-TarjanDynamic treesmn log n1983 Goldberg-TarjanFIFO preflow-pushmn log (n 2 / m)1986... Goldberg-RaoLength function m 3/2 log (n 2 / m) log U mn 2/3 log (n 2 / m) log U 1997