1. MINIMUM COST FLOWS: NETWORK SIMPLEX ALGORITHM A talk by: Lior Teller 1

2. A short reminder (it’s been three weeks..) 2

3. Minimum Cost Flows - Definition 3

4. Introduction Cycle Canceling Successive Shortest Paths Primal-Dual Out of Kilter Polynomial Algorithms – Capacity Scaling Today – The Simplex Method 4

5. Introduction - The Simplex Method A powerful method for: solving linear programming solving constrained optimization problems Minimum cost flow problems define a special class of linear programs Network flow problems have considerable special structure When implementation does not exploit underlying network structure – not a competitive solution procedure for solving minimum cost flow problems. Interpreting the concepts of the simplex method appropriately as network operations, we can produce an algorithm that is very efficient Our goal: developing a network-based implementation of the simplex method 5

6. xWe won’t describe the simplex method for general linear programming problems and then show how to adapt the method for minimum cost flow problems. We will develop the network simplex method directly in the context of network flow problems as a particular type of augmenting cycle algorithm. No need for any background in linear programming Building more directly on concepts we have developed already 6

7. Algorithm Outline The central concept - spanning tree solutions, obtained by fixing the flow of every arc not in a spanning tree either at zero or at the arc's flow capacity. From this we retrieve uniquely the flow on all the arcs in the spanning tree Minimum cost flow problem has at least one optimal spanning tree solution We move from one solution to another to find an optimal spanning tree solution At each step introducing one new non-tree arc into the spanning tree in place of one tree arc. This method is known as the network simplex algorithm: Spanning trees correspond to the so-called basic feasible solutions of linear programming The movement from one spanning tree solution to another corresponds to a so-called pivot operation of the general simplex method 7

8. Cycle free and spanning tree solutions For any feasible solution X: Arc (i, j) is a free arc if 0 < x ij < u ij Arc (i, j) is a restricted arc if x ij = 0 or x ij = u ij Note that we can both increase and decrease flow on a free arc. In a restricted arc we can only increase or decrease the flow. 8

9. Cycle Free Solution: A solution X is a cycle free solution if the network contains no cycle composed only of free arcs. Note: in a cycle free solution, we can augment flow on any augmenting cycle in only a single direction (some arc in any cycle will restrict us from either increasing or decreasing that arc's flow). Spanning Tree Solution A feasible solution X and an associated spanning tree of the network is a spanning tree solution if every non tree arc is a restricted arc. Note: in a spanning tree solution, the tree arcs can be free or restricted 9

10. Next we establish a fundamental result of network flows: Minimum cost flow problems always have optimal cycle free and spanning tree solutions. The network simplex algorithm will exploit this result by restricting its search for an optimal solution to only spanning tree solutions First we will see some required background for this (a small reminder) 10

11. First assume that all arcs are un capacitated The network shown in the figure contains positive flow around a cycle. We define the orientation of the cycle as the same as that of arc (4, 5). Let us augment θ units of flow along the cycle in the direction of its orientation. Increases flow on forward arcs by θ units and decreases the flow on backward arcs by θ units. The per unit incremental cost for this flow change is the sum of the costs of forward arcs minus the sum of the costs of backward arcs: Per unit change in cost Δ = 2 + 1 + 3 - 4 - 3 = - 1. 11 Reminder: negative cycles

12. Augmenting flow in the cycle decreases the cost We set θ as large as possible while keeping all arc flows non negative  We must satisfy the inequalities: 3 - θ ≥ 0 and 4 - θ ≥ 0  We set θ = 3  Some arc in the cycle has a flow at value zero  The objective function value is strictly less than the initial solution 12

13. We can extend this observation in several ways: a positive per unit change in cost (Δ) means we set θ as small as possible If Δ = 0, we are indifferent to all solutions in the interval If we impose upper bounds on the flow the range of flow that preserves feasibility an interval 13

14. (Cycle Free Property). If the objective function of a minimum cost flow problem is bounded from below over the feasible region, the problem always has an optimal cycle free solution. Proof: (With respect to a given flow x): Arc (i, j) is a free arc if 0 < x ij < u ij or restricted arc if x ij = 0 or x ij = u ij Solution x has the cycle free property - contains no cycles entirely consisting of free arcs. If a solution x contains a cycle (i.e. it is not cycle free) calculate the cycle cost If the cycle cost is negative we send θ flow around the cycle in the direction of traversal If it is positive send θ flow around the cycle in the opposite direction In either of these cases we make θ as large as possible  Cycle must be eliminating If the cycle cost is 0 we can remove the cycle by increasing the flow in either direction Applying this argument repeatedly removes all cycles from the solution. 14

15. (Spanning Tree Property) If the objective function of a minimum cost flow problem is bounded from below over the feasible region, the problem always has an optimal spanning tree solution. Proof: (This is an extension of the cycle free property): Let S be the set of free arcs If the free arcs connect all the nodes of G, then the solution is connected and acyclic (from the cycle free property)  thus forming a spanning tree Otherwise, we add to S some restricted arcs until S connects all the nodes. The set S now forms a spanning tree of G, and the feasible solution x form which S is derived is referred to as a spanning tree solution. 15

16. A spanning tree solution partitions the arc set A into three subsets: T - the arcs in the spanning tree L - the non-tree arcs whose flow is zero U - the non-tree arcs whose flow is the arc’s flow capacities We refer to the triplet (T, L, U) as a spanning tree structure.  we can associate a spanning tree structure with a spanning tree solution 16

17. 17

18. A Spanning tree can be: Non-degenerate: every arc in a spanning tree solution is a free arc Degenerate ( מנוון ): otherwise A spanning tree structure as feasible if its associated spanning tree solution is a feasible solution of the minimum cost flow problem. A spanning tree structure as optimal if its associated spanning tree solution is an optimal solution of the minimum cost flow problem. 18

19. 19 The optimality condition

20. 20

21. 21

22. Maintaining a spanning tree structure Algorithm moves from one feasible spanning to another (pivot operation) Stops when the spanning tree structure satisfies the optimality condition At each iteration, the algorithm adds one arc to the spanning tree in place of one of its current arcs. The entering arc is a non-tree arc violating its optimality condition. The algorithm adds this arc to the spanning tree, creating a negative cycle It sends the maximum possible flow in this cycle Drops an arc whose flow has reached its lower or upper bound We need a data structure to support this.. 22

23. Spanning tree data structure Predecessor index: Each node i has a unique path from the root. The index pred(i) stores the predecessor of i on this path. Depth index: The index depth(i) stores the number of arcs on the path from the root to i. Thread index: defines a traversal of a tree. set to be the node in the depth-first search encountered just after the node itself. Provide a convenient means for visiting all descendants of a node. 23

24. Computing node potentials 24

25. Computing flows 25

26. Working up from the leaves it is possible to calculate all arc flows (won’t go into details.. Need some intuition?) For the capacitated version of the problem, we add this initialization: (won’t go into details.. Need some intuition?) 26

27. Network Simple Algorithm The network simplex algorithm maintains a feasible spanning tree structure at each iteration and successively transforms it into an improved spanning tree structure until it becomes optimal. 27

28. Obtaining an Initial Spanning Tree Structure Solve the maximum flow problem  feasible solution Construct an initial spanning tree structure by converting this solution into a spanning tree solution 28

29. Optimality Testing and the Entering Arc To determine weather a spanning tree structure is optimal we check: If the spanning tree structure satisfies the conditions it is optimal  we terminate Else, the algorithm selects a non-tree violating arc to enter the tree: 29

30. Pivot Rules For any eligible arc (i, j) we refer to Ic ij I as its violation. The network simplex algorithm can select any eligible arc to enter the tree and still would terminate finitely (with some provisions for dealing with degeneracy) Pivot rules for selecting entering arcs: Dantzig's pivot rule First eligible arc pivot rule Candidate list pivot rule 30

31. Dantzig's pivot rule At each iteration - select an arc with the maximum violation to enter the tree. Motivation - the arc with the maximum violation causes the maximum decrease in the objective function per unit change in the value of flow on the selected arc  maximum decrease per pivot The algorithm performs fewer iterations than other choices for the pivot rule. However, The algorithm must consider every non tree  very time consuming.  the algorithm is not attractive. 31

32. First eligible arc pivot rule Scan the arc list sequentially and select the first eligible arc to enter the tree In a popular version of this rule, we examine the arc list in a wraparound fashion  Quickly identifies the entering arc the algorithm generally performs more iterations than it would with other pivot rules because each pivot operation produces a relatively small decrease in the objective function value 32

33. Candidate list pivot rule Selects the entering arc using a two-phase procedure consisting of major iterations and minor iterations: major iteration: construct a candidate list of eligible arcs. o Examine arcs emanating from a node and add eligible arcs to the candidate list. o We repeat this process for next nodes until the list has reached max size or we have examined all the nodes. o Next major iteration begins with the node of the previous major iteration and examines nodes in wraparound minor iterations: in each iteration selects an eligible arc from the candidate list with the maximum violation o Scans arcs in candidate list and selects maximum violation arc to enter the tree o Removes arcs that are no longer eligible from candidate list (due to changes in the node potentials) o Once the candidate list becomes empty or a reached limit on the number of minor iterations per major iteration  we rebuild the candidate list by performing another major iteration 33

34. Candidate list pivot rule (continued) By setting the: maximum allowable size of the candidate list number of minor iterations per major iteration we can obtain numerous different pivot rules. Example: -Dantzig's pivot rule: number of minor iterations per major iteration = 1 -First eligible arc pivot rule: maximum allowable size of the candidate list = 1 34

35. Leaving Arc 35

36. 36

37. Identifying the pivot cycle 37

38. Updating the Tree When leaving arc (p, q) is determined for a given entering arc (k, I), we updates the tree structure If the leaving arc is the same as the entering arc, (when δ = δ pq =u pq ), the tree does not change. the arc (k, I) merely moves from the set L to the set U, or vice versa. Else, the algorithm must perform more extensive changes. The arc (p, q) becomes a non-tree arc at its lower or upper bound, depending on whether (in the updated flow) X pq = 0 or X pq = u pq Adding arc (k, I) to the current spanning tree and deleting arc (p, q) creates a new spanning tree  the node potentials also change 38

39. Updating node potentials 39

40. Termination 40

41. Strongly feasible spanning trees A poor choice of entering and leaving arcs leads to cycling (i.e., an infinite repetitive sequence of degenerate pivots) Degeneracy is both a theoretical and practical issue  We will maintaining a special type of spanning tree, a strongly feasible spanning tree We will show the network simplex algorithm terminates finitely 41

42. Terminology and definitions The tree arcs are either: upward pointing (toward the root) downward pointing (away from the root) Strongly feasible spanning tree: 1.A spanning tree T is strongly feasible if every tree arc with zero flow is upward pointing and every tree arc whose flow equals its capacity is downward pointing 2.A spanning tree T is strongly feasible if we can send a positive amount of flow from any node to the root along the tree path without violating any flow bound 42

43. 43

44. Note that a non-degenerate spanning tree is always strongly feasible A degenerate spanning tree might or might not be strongly feasible Algorithm creates a degenerate spanning tree from a non-degenerate spanning tree whenever two or more arcs are qualified as leaving arcs and we drop only one of these  Therefore, the algorithm needs to select the leaving arc carefully so that the next spanning tree is strongly feasible. 44

45. Leaving Arc Rule: select the leaving arc as the last blocking arc encountered in traversing the pivot cycle W along its orientation starting at the apex w. We show that the leaving arc rule guarantees that in the next spanning tree every node in the cycle W can send a positive amount of flow to the root node. 45

46. Property: Each node in the segment W 2 can send a positive amount of flow to the root in the next spanning tree Arc (p, q) is the last blocking arc in W No arc in W 2 is blocking Every node in this segment can send a positive amount of flow to the root via node w (the apex) along the orientation of W 2 46

47. Property: Each node in the segment W 1 can send a positive amount of flow to the root in the next spanning tree We prove this observation by considering two cases: 1.If the previous pivot was a non-degenerate pivot, the pivot augmented a positive amount of flow δ along the arcs in W 1. After the augmentation, every node in W 1 can send a positive amount of flow back to the root opposite to the orientation of W 1 via the apex node w (each node can send at least δ units to the apex and then at least some of this flow to the root since the previous spanning tree was strongly feasible) 2.If the previous pivot was a degenerate pivot, W 1 must be contained in the segment of W between node w and node k because the property of strong feasibility implies that every node on the path from node I to node w can send a positive amount of flow to the root before the pivot, and thus no arc on this path can be a blocking arc in a degenerate pivot. Now observe that before the pivot, every node in W 1 could send a positive amount of flow to the root, and since the pivot does not change flow values, every node in W 1 must be able to send a positive amount of flow to the root after the pivot as well. 47

48. We next show that in the next spanning tree, nodes not belonging to the cycle W can also send a positive amount of flow to the root In the previous spanning tree (before the augmentation), every node j could send a positive amount of flow to the root and if the tree path from node j does not pass through the cycle W, the same path is available to carry a positive amount of flow in the next spanning tree If the tree path from node j does pass through the cycle W, the segment of this tree path to the cycle W is available to carry a positive amount of flow in the next spanning tree and once a positive amount of flow reaches the cycle W, then, as shown earlier, we can send it (or some of it) to the root node This completes the proof that the next spanning tree is strongly feasible. 48

49. We now establish the correctness of the network simplex algorithm: We have previously shown that each non-degenerate pivot strictly decreases the objective function value The number of non-degenerate pivots is finite The algorithm can, however, also perform degenerate pivots We will show that the number of successive degenerate pivots between any two non-degenerate pivots is finitely bounded. 49

50. 50

51. 51