1 Two-Phase Simplex Method file Simplex3_AMII_05b_gr Rev. 1.4 by M. Miccio on December 17, 2014 from a presentation at the Fuqua School of Business MIT and James Orlin © 2003

2 MIT and James Orlin © Today’s Lecture The simplex algorithm with constraints of type = OR ≥ The big M method The "artificial objective" w

3 Example ORLIN with contraints of type = z = -3x 1 + 2x 2 + 4x 3 + x 4 max ! -3x 1 + 3x 2 + 2x 3 + 5x 4 = 6 equality -4x 1 + 2x 2 + x 3 + 3x 4 = 2equality x j  0 non-negativity

4 MIT and James Orlin © Simplex Algorithm: getting started To start the simplex algorithm, we need a canonical form, which corresponds to a basic feasible solution. So, how do we get started? x1x ==== x2x2 x4x4 x3x z = Example ORLIN

5 MIT and James Orlin © A naïve suggestion: just choose the variables and then pivot to get into canonical form. x1x ==== x2x2 x4x4 x3x z = Suppose we try to make x 3 and x 4 the basic variables The tableau after bringing it to Jordan-canonical form: So, guessing may not create a basic feasible solution! There may not even be a feasible solution! This technique sometimes works, but ….

6 MIT and James Orlin © A suggestion that sounds dumb, but works: let’s create an artificial basis. x1x ==== x2x2 x4x4 x3x z = x6x6 x5x Let’s just make up some “artificial” variables x 5 and x 6 and use them to get started. Obvious difficulty: we are now solving a different problem. There is no guarantee that this will help us solve our original problem.

7 Example ORLIN with contraints of type = z = -3x 1 + 2x 2 + 4x 3 + x 4 min ! -3x 1 + 3x 2 + 2x 3 + 5x 4 + x 5 = 6 equality -4x 1 + 2x 2 + x 3 + 3x 4 + x 6 = 2equality x j  0 non-negativity Artificial variables x 5 x 6

8 MIT and James Orlin © Trying to make artificial bases work New problem: original model plus artificial variables. The artificial variables help us get started. What we want: optimizing the new model will produce an optimum solution to the original model. We will use the Simplex algorithm on a new problem P’, which is closely connected to original problem P. Potential Danger: the artificial variables will be positive when optimizing the new model. This would be bad. So, we can add artificial variables to our model, if we can guarantee that the variables take on a value of 0 in the optimum solution.

9 MIT and James Orlin © Today’s Lecture The simplex algorithm with contraints of type = OR ≥ The big M method The "artificial objective" w

10 MIT and James Orlin © So, how can we modify x 5 and x 6 so that they won’t be in an optimal solution? x1x ==== x2x2 x4x4 x3x z = x6x6 x5x Give them a large cost. If the cost is big enough, then x 5 and x 6 will not be positive in an optimum solution Two issues: (1)the problem is no longer in canonical form (i.e., c j =0 in the basis) (2)will an optimum for this model also be optimum for the original? Example ORLIN

11 MIT and James Orlin © (1) Getting into canonical form x1x ==== x2x2 x4x4 x3x z = x6x6 x5x Add 50 times constraint 1 and 50 times constraint 2 to the the row of cost coefficients Remaining issue: Will an optimum for this model also be optimum for the original?

12 MIT and James Orlin © pivots later We obtain an optimal basic feasible solution to the new problem. x1x ==== x2x2 x4x4 x3x z = x6x6 x5x Eliminating x 5 and x 6 does not impair the optimal solution to the original problem.

13 MIT and James Orlin © The Big M method The cost coefficient of the artificial variables should be –M in a maximization problem for some large value of M. (We used M = 50) In such a way none of these variables should be positive in an optimum solution Difficulties –How big should M be? –Large values of M can increase the problem of numerical round-off errors (also known as numerical instability)

14 MIT and James Orlin © Today’s Lecture The simplex algorithm with contraints of type = OR ≥ The big M method The Phase 1 method with the "artificial objective" w

15 MIT and James Orlin © An alternative approach: The Phase 1 method Recall: Any basic feasible solution would get the simplex method started We add artificial variables, and then focus entirely on obtaining a basic feasible solution with the artificial variables We add an "artificial objective" w ("Forma di Inammissibilità" ), which is the sum of all the artificial variables The modified linear program with this "artificial objective" w is called the Phase I problem We can then start the simplex algorithm with the first basic feasible solution we have found

16 Simplex with constraints ,  Example 2D.3 z = 6x 1 + 5x 2 max! 1) 2x 1 +5x 2  20 2) 5x 1 + x 2  5 3) 3x x 2  33  i, x i  0 System of Eqs. in a Canonic Form 1) 2x 1 +5x 2 + x 3 = 20 2) 5x 1 + x 2 -x 4 +x 6 = 5 3) 3x x 2 -x 5 +x 7 = 33 z) 6x 1 + 5x 2

17 Simplex with constraints ,  Example 2D.3  Adds a row for the artificial variables and the artificial objective (-w)  The new cost coefficients are zero for the decision variables, whereas they are 1 for the artificial variables Example 2D.3 >> simplex2p(type, c, A, rel, b) Expected tableau A = (-z) (-w) LINEAR PROGRAMMING WITH MATLAB course by Edward Neuman Department of Mathematics Southern Illinois University at Carbondale

18 SIMPLEX with MatLab® simplex2p(type, c, A, rel, b)  Adds an artificial variable for each constraint, even when NOT needed (constraint of type  ) to generate the initial Jordan canonical form  Reports in the righmost column the value with the opposite sign of the objective function (-z) and the artificial objective (-w) LINEAR PROGRAMMING WITH MATLAB course by Edward Neuman Department of Mathematics Southern Illinois University at Carbondale Example 2D.3 >> simplex2p(type, c, A, rel, b) Initial tableau A = (-z) (-w) Actual initial tableau  the row of the artificial variables is substituted in order to be a function of decision variables  the artificial objective (-w) turns equal to the opposite of the sum of resources

19 Example ORLIN ( again ! ) with contraints of type = z = -3x 1 + 2x 2 + 4x 3 + x 4 min ! -3x 1 + 3x 2 + 2x 3 + 5x 4 + x 5 = 6 equality -4x 1 + 2x 2 + x 3 + 3x 4 + x 6 = 2equality x j  0 non-negativity

20 Example ORLIN with contraints of type = >> simplex2p(type, c, A, rel, b) Initial tableau A = (-z) (-w)

21 MIT and James Orlin © Observation 1: if all we want is a basic feasible solution, then we can select any objective function. x1x ==== x2x2 x4x4 x3x w-w = 0 2 FACT: Once we find a basic feasible solution, we can reconsider the original cost coefficients. 6 2 It’s easy to bring cost coefficients into canonical form. ????? We will choose an objective function soon.

22 MIT and James Orlin © Next: add in the artificial variables, creating a basic feasible solution to the new problem x1x ==== x2x2 x4x4 x3x w-w = 0 2 Now: choose an objective such that x 5 and x 6 are guaranteed to be 0 if we optimize the objective. 6 2 Minimize x 5 + x 6. Or maximize w = -x 5 – x 6. ????? 0 01 x6x6 x5x5 1 ??

23 MIT and James Orlin © The phase 1 problem: almost in canonical form x1x ==== x2x2 x4x4 x3x w-w = x6x6 x5x5 1 To get into canonical form, add constraints 1 and 2 to the objective.

24 MIT and James Orlin © The phase 1 problem: now in canonical form x1x ==== x2x2 x4x4 x3x w-w = x6x6 x5x We are now in a form to start the simplex algorithm

25 MIT and James Orlin © Two pivots later, we have an optimal solution to the phase 1 problem. So, we have identified a basic feasible solution with basic variables x 1 and x 2. This leads to a bfs for our original problem. We now need to return to our original problem, and continue to find the optimum x1x ==== x2x2 x4x4 x3x w-w = 0 2x5x5 x6x6 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/

26 MIT and James Orlin © Recovering a bfs for the original problem x1x ==== x2x2 x4x4 x3x w-w = 0 2x5x5 x6x6 BV -w x6x6 x5x At the end of phase 1, eliminate the “artificial variables” x 5 and x /6 1/3-1/21 015/611/62/3-1/

27 MIT and James Orlin © Recovering a bfs for the original problem x1x ==== x2x2 x4x4 x3x w = 0 2x5x5 x6x6 BV -w x6x6 x5x At the end of phase 1, eliminate the “artificial variables” x 5 and x /6 1/3-1/21 015/611/62/3-1/ Reintroduce the original objective z

28 MIT and James Orlin © Recovering a bfs for the original problem x1x ==== x2x2 x4x4 x3x w = 0 2x5x5 x6x6 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/ Then bring into canonical form. We can now start phase z 0017/6-13/6-3 At the end of phase 1, eliminate the “artificial variables” x 5 and x 6. Reintroduce the original objective

29 MIT and James Orlin © Phase 2. And one pivot later It works very well. Phase 1 seems like a lot of work. It can do a lot of pivots, and the only purpose is to find a basic feasible solution, so that we can start “phase 2” It’s what is done in practice We have now solved the original problem. x1x ==== x2x2 x4x4 x3x z =

30 MIT and James Orlin © Phase 1 method. Potential Difficulties If the original problem is degenerate, it is possible that an artificial is in the basis at the end of phase 1.  Solution: pivot out the artificial variable (exit variable) If the original problem has a redundant constraint, then it is possible that an artificial variable is in the basis at the end of phase 1, and cannot be pivoted out.  Solution: eliminate (or ignore) the redundant constraint

31 MIT and James Orlin © Summary To get started with the simplex method, add an artificial basis, but ensure that these artificial variables don’t occur in an optimal solution. Big M method: put a large cost on each of the artificial variables Phase 1 method. Minimize the sum of the artificial variables. At the end of phase 1, we will have a basic feasible solution to the original problem. Use this as a starting point for Phase 2, which solves the original problem.

32 MIT and James Orlin © The following are extra slides

33 MIT and James Orlin © Towards: writing an LP in general form. This notation is useful for the text AMP. ==== 2 6 x1x1 xsxs x4x4 x3x3 -z = 0 Use  a ij to denote the constraint matrix coefficients. Use  c j to denote the cost coefficients Use  b i to denote the RHS coefficients x4x4 x3x3 c1c1 c2c2 b1b1 b2b2  a 11  a 12  a 21  a 22 The bar − indicates that it is the coefficient after some pivots -z0-z0 c3c3 c4c4  a 13  a 14  a 23  a 24

34 MIT and James Orlin © Towards: writing an LP in general form ==== 2 6 x1x1 x2x2 x4x4 x3x3 -z = 0 Let r denote the index of the row with the leaving variable. Let s denote the index of the entering variable x4x4 x3x3 c1c1 c2c2 b1b1 b2b2  a 11  a 12  a 21  a 22 cscs  a 1s  a 2s brbr  a rs  a r1 We pivot on coefficient  a rs -z0-z0 Usually, we write the basic variables as unit vectors

35 MIT and James Orlin © Towards: writing an LP in general form ==== 2 6 x1x1 x2x2 x4x4 x3x3 -z = 0 (2) the number of variables (columns) is n. (1) the number of equality constraints (rows) is m x4x4 x3x3 c1c1 c2c2 b1b1 b2b2  a 11  a 12  a 21  a 22 cscs  a 1s  a 2s brbr  a rs  a r1 To do next: rewrite the LP so that: -z0-z0

36 MIT and James Orlin © Optimality Conditions (maximization) ==== 2 6 x2x2 x4x4 x3x z = -8 This basic feasible solution is optimal! What are the optimality conditions, expressed in terms of  c ? x1x1

37 MIT and James Orlin © Optimality Conditions (maximization) ==== 2 6 x2x2 x4x4 x3x z = -8 This basic feasible solution is optimal! What are the optimality conditions, expressed in terms of  c ? x1x1 c1c1 c2c2 -z0-z0

38 MIT and James Orlin © x1x Pivoting and the min ratio rule ==== 2 6 x2x2 x4x4 x3x z = 0 1 x 1 = 0 x 2 =  x 3 =  x 4 =  z = 2  x 3 = 0 when  = 6/3 x 4 = 0 when  = 2/2. 63x3x3 x4x4 BV -z 22  is set to the min(6/3, 2/2) = min (  b 1 /  a 1s,  b 2 /  a 2s ). Pivot in variable x s, where  c s >

39 MIT and James Orlin © x1x Pivoting and the min ratio rule ==== 2 6 x2x2 x4x4 x3x z = 0 1 x 1 = 0 x 2 =  x 3 =  x 4 =  z = 2  x 3 = 0 when  = 6/3 x 4 = 0 when  = 2/2. 63x3x3 x4x4 BV -z 22  is set to the min(6/3, 2/2) = min (  b 1 /  a 1s,  b 2 /  a 2s ). Express the min ratio rule using general coefficients Pivot in variable x s, where  c s > b1b1 b2b2  a 1s  a 2s The constraint with a changed basic variable is constraint r, where r = argmin (  b 1 /  a 1s,  b 2 /  a 2s ) = 2. Min ratio rule.

40 MIT and James Orlin © x1x Pivoting to obtain a better solution ==== 2 6 x2x2 x4x4 x3x z = 0 1 x 1 = 0 x 2 =  x 3 = 3 x 4 = 0 z = 2 63x3x3 x4x4 BV -z 22 Pivot in variable x s, where  c s > 0. Pivot out the basic variable for constraint r according to the min ratio rule / /23 1

MIT and James Orlin ©

42 MIT and James Orlin © z = Creating a Phase 1 Problem x1x ==== x2x2 x4x4 x3x x5x5 x6x6 BV Eliminate the objective function, for now 6 2

43 MIT and James Orlin © z = Creating a Phase 1 Problem x1x ==== x2x2 x4x4 x3x x5x5 x6x6 BV Eliminate the objective function, for now 6 2 Add the artificial variables

44 MIT and James Orlin © z = Creating a Phase 1 Problem x1x ==== x2x2 x4x4 x3x x5x5 x6x6 BV Eliminate the objective function, for now 6 2 x6x6 x5x Add the artificial variables Minimize the sum of the artificials

45 MIT and James Orlin © z = Creating a Phase 1 Problem x1x ==== x2x2 x4x4 x3x x5x5 x6x6 BV Eliminate the objective function, for now 6 2 -w = 0 x6x6 x5x Add the artificial variables Minimize the sum of the artificials Bring into canonical form

46 MIT and James Orlin © z = Creating a Phase 1 Problem x1x ==== x2x2 x4x4 x3x x5x5 x6x6 BV Eliminate the objective function, for now 6 2 -w = 0 x6x6 x5x Add the artificial variables Minimize the sum of the artificials Bring into canonical form

47 MIT and James Orlin © Creating a bfs from the phase 1 solution x1x ==== x2x2 x4x4 x3x w = 0 2x1x1 x2x2 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/ If w = 0, eliminate artificial variables (or keep the columns but forbid them from pivoting in). At end, if w > 0, report that there is no feasible solution.

48 MIT and James Orlin © Creating a bfs from the phase 1 solution x1x ==== x2x2 x4x4 x3x w = 0 2x1x1 x2x2 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/ If w = 0, eliminate artificial variables (or keep the columns but forbid them from pivoting in). Reintroduce the original objective At end, if w > 0, report that there is no feasible solution.

49 MIT and James Orlin © Creating a bfs from the phase 1 solution x1x ==== x2x2 x4x4 x3x w = 0 2x1x1 x2x2 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/ Then bring into canonical form z If w = 0, eliminate artificial variables (or keep the columns but forbid them from pivoting in). Reintroduce the original objective At end, if w > 0, report that there is no feasible solution.

50 MIT and James Orlin © Creating a bfs from the phase 1 solution x1x ==== x2x2 x4x4 x3x w = 0 2x1x1 x2x2 BV -w x6x6 x5x /6 1/3-1/21 015/611/62/3-1/ Then bring into canonical form. This begins Phase z 0017/6-13/6-3 If w = 0, eliminate artificial variables (or keep the columns but forbid them from pivoting in). Reintroduce the original objective At end, if w > 0, report that there is no feasible solution.

MIT and James Orlin ©

52 MIT and James Orlin © Phase 1: obtaining an initial bfs We know how to obtain an optimal bfs if we are given an initial bfs. To create an initial bfs for problem P, we will use the simplex algorithm on problem P’, which is closely connected to P.

53 MIT and James Orlin © How does one obtain an initial bfs? x1x ==== x2x2 x4x4 x3x z = 0 2 FACT: Obtaining an initial basic feasible solution is (theoretically) as difficult as obtaining an optimal one 6 2 IDEA: Use the simplex algorithm to obtain an initial bfs.

54 MIT and James Orlin © Reducing to a Previously Solved Problem Juan’s Problem Find a feasible solution to -3 x x 2 +2 x x 4 = 6 -4 x x 2 +1 x x 4 = 2 x j  0 for j = 1, 2, 3, 4 Maria’s Problem Minimize x 5 + x 6 subject to: -3 x x 2 +2 x x 4 + x 5 = 6 -4 x x 2 +1 x x 4 + x 6 = 2 x j  0 for j = 1, 2, 3, 4, 5, 6

55 MIT and James Orlin © Maria’s Problem x1x ==== x2x2 x4x4 x3x3 -w 0 0 2x5x5 x6x6 BV = 0-w x6x6 x5x

56 MIT and James Orlin © Maria’s Problem x1x ==== x2x2 x4x4 x3x3 -w 0 0 2x5x5 x6x6 BV = 0-w x6x6 x5x Here is an optimal solution to Maria’s Problem. Recall: we want to minimize w = x 5 + x 6 No solution has a cost less than 0.

57 MIT and James Orlin © Juan’s Problem x1x ==== x2x2 x4x4 x3x3 -w 0 0 2x5x5 x6x6 BV = 0-w x6x6 x5x Conclusion: Juan’s Problem and Maria’s Problem are equivalent. Not so obvious: Solving Maria’s Problem using simplex will yield a bfs for Juan. To get an feasible solution to Juan’s problem, drop x 5 and x 6.

58 MIT and James Orlin © Maria’s Problem in Tableau Form x1x ==== x2x2 x4x4 x3x3 -w 0 0 2x5x5 x6x6 BV This problem is not yet in canonical form. But it is close = 0-w x6x6 x5x What transformation do we need to do?

59 MIT and James Orlin © Maria’s Problem in Canonical Form x1x ==== x2x2 x4x4 x3x w = 0 2x5x5 x6x6 BV -w x6x6 x5x Then run the simplex algorithm on Maria’s problem It will find a feasible solution (if one exists) for Juan’s problem, and it will terminate with a bfs. Add constraints 1 and 2 to the objective.

60 MIT and James Orlin © The optimal basis for Maria’s Problem x1x ==== x2x2 x4x4 x3x w = 0 2x5x5 x6x6 BV -w x6x6 x5x At the end, one can obtain a bfs for the original problem by dropping x 5 and x /6 1/3-1/21 015/611/62/3-1/ The optimal basic variables are x 1 and x 2.