2. An-Najah N. University Faculty of Engineering and Information Technology Department of Management Information systems Operations Research and Applications MIS:10676213 Prepared by Mr.Maher Abubaker Fall 2015/2016 Resources Daniel J. Epstein Department of Industrial and Systems Engineering University of Southern California Andrew and Erna Viterbi School of Engineering http://www.eecs.qmul.ac.uk/~eniale/teaching/ise330/index.html Operations Research: An Introduction, 9/EHamdy A. Taha, University of ArkansasISBN-10: 013255593X ISBN-13: 9780132555937©2011 Prentice Hall Cloth, 832 ppPublished 08/29/2010 http://www.pearsonhighered.com/educator/product/Operations-Research-An- Introduction/9780132555937.page#downlaoddiv ™ INFORMS – www.informs.org ™ ORMS - www.lionhrtpub.com/ORMS.shtml ™ Science of Better - www.scienceofbetter.org

3. Chapter 3 Simplex Method and Sensitivity Analysis

4. Simplex Method

5.  Start from the origin (0,0)  While Solution is not Optimal  Move to the next corner point  Loop

6. Simplex Method  Corner Point Feasible Solutions CPF

7. Simplex Method  Find the Best ( Optimal ) Solution

8. LP Terminology Feasible solution - solution in which all the constraints are satisfied. Optimal solution - feasible solution having the most favorable objective function value Boundary equation - the resulting equation after dropping the on a constraint Corner point solution - a feasible solution that does not lie on any line segment connecting two other feasible solutions (extreme point). The intersection of two or more boundary equations.

9. Properties of Corner Point Solutions If there is exactly one optimal solution, then it must be a corner point solution. If there are multiple solutions then at least two must be adjacent corner point solutions. There are only a finite (although often large) number of corner point solutions. If a corner point solution is better than all of its adjacent corner point solutions, then it is better than all other feasible corner points. That is - it is optimal!

10. Solve the following LPP Using Simplex Method x 1 >= 0; x 2 >= 0 Max z = 6x 1 + 4x 2 Subj. to: x 1 + x 2 <= 12 x 1 -2x 2 <= 6 x 2 <= 8 Simplex Method Example

11. Convert Inequalities to Equations Max z = 6x 1 + 4x 2 Subj. to: x 1 + x 2 + S1 = 12 x 1 -2x 2 + S2 = 6 x 2 + S3 = 8 x 1 >= 0; x 2 >= 0 S 1 >= 0, S 2 >=0, S 3 >= 0

12. Convert Inequalities to Equations Max z = 6x 1 + 4x 2 Subj. to: x 1 + x 2 + S 1 = 12 x 1 -2x 2 + S 2 = 6 x 2 + S 3 = 8 x 1 >= 0; x 2 >= 0 S 1 >= 0, S 2 >=0, S 3 >= 0 slack variables must be non-negative

13. Basic Solutions Augmented solution - solution to the problem after slack variables have been added. Basic solution - augmented corner point solution obtained by setting n variables to zero and solving for the remaining. Basic feasible solution (BFS) - feasible basic solution. Basic variables - variables that were solved for in the basic solution. Non-basic variables - variables set equal to zero in the basic solution.

14. Well, just how many basic solutions are there?!!! Let n = the number of variables m = the number of constraints After adding slack variables, there are n + m variables. Since a basic solution requires setting n variables to zero and solving uniquely for the remaining variables, there are (n+m)! / n!m! ways to select the n variables to set equal to zero or equivalently the m basic variables. If n = 20 and m = 10 (a trivially small problem), then 30!/(20! 10!) = 30,045,015. (not all are feasible)

15. Basic Solutions Again! Augmented solution - solution to the problem after slack variables have been added. Basic solution - augmented corner point solution obtained by setting n variables to zero and solving for the remaining. Basic feasible solution (BFS) - feasible basic solution. Basic variables - variables that were solved for in the basic solution. Non-basic variables - variables set equal to zero in the basic solution.

16. Outline of the Simplex Algorithm Start at a basic feasible solution (BFS) – (often the origin) Move to a better basic feasible solution – the objective function improves Stop when the basic feasible solution is better than all adjacent basic feasible solutions. – Solution is optimal

17. Now - Presenting the Simplex Algorithm! Be the first in your neighborhood to master this exciting and wonderful numerical, recursive process for solving linear programs. Two students discussing the high points of the simplex algorithm.

18. Simplex Method For Maximization and <= Constraints LPP Step1 Original Form : the Mathematical Model of the LPP Step2 Augmented Form : the Mathematical Model with equations and Slack Variables Step3 Standard Form : Prepare the model with objective function to Simlex tableau form Step 4 Tableau Form: The initial Simplex tableau with the initial basic solution

19. Example Max z = 4x 1 + x 2 Subj. to: x 1 + x 2 <= 6 x 1 <= 5 x 2 <=4 x 1 >= 0; x 2 >= 0 Convert the Model to Standard Form (SF) By Changing <= Constraints into equations by adding Slack Variables.

20. Convert Inequalities to Equations Max z = 4x 1 + x 2 Subj. to: x 1 + x 2 + s1 = 6 x 1 + s2 = 5 x 2 + s3 = 4 x 1 >= 0; x 2 >= 0 s1 >= 0, s2 >=0, s3 >= 0 slack variables must be non-negative

21. The Simplex Tableau-1 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS Standard Form ( SF ) Max z - 4x 1 - x 2 = 0 Subj. to: x 1 + x 2 + s 1 = 6 x 1 - 2x 2 + s 2 = 5 x 2 + s 3 = 4

22. The Simplex Tableau-2 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 Max z - 4x 1 - x 2 = 0 Subj. to: x 1 + x 2 + s 1 = 6 x 1 - 2x 2 + s 2 = 5 x 2 + s 3 = 4

23. The Simplex Tableau-3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s101110061s10111006 Max z - 4x 1 - x 2 = 0 Subj. to: x 1 + x 2 + s 1 = 6 x 1 - 2x 2 + s 2 = 5 x 2 + s 3 = 4

24. The Simplex Tableau-4 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s101110061s10111006 2s201001052s20100105 Max z - 4x 1 - x 2 = 0 Subj. to: x 1 + x 2 + s 1 = 6 x 1 - 2x 2 + s 2 = 5 x 2 + s 3 = 4

25. The Simplex Tableau-5 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s101110061s10111006 2s201001052s20100105 3s300100143s30010014 Max z - 4x 1 - x 2 = 0 Subj. to: x 1 + x 2 + s 1 = 6 x 1 - 2x 2 + s 2 = 5 x 2 + s 3 = 4

26. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 The Simplex Algorithm-1 Step 1: Select a new variable to enter the basis.

27. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 The Simplex Algorithm-2 Step 1: Select a new variable to enter the basis. Entering Var Pick the non-basic variable having the greatest negative value. z = 4x 1 + x 2 Basic Solution Values for the Basic Solution

28. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 2a: Select a basic variable to leave the basis.

29. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 2b: Select a basic variable to leave the basis. Pick the basic variable having the smallest ratio of the RHS divided by the corresponding positive coefficient from the incoming variable. M.R.T 6/1 5/1

30. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 2c: Select a basic variable to leave the basis. Pick the basic variable having the smallest ratio of the RHS divided by the corresponding positive coefficient from the incoming variable. Ratio 6/1 5/1 pivot point

31. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3a: Use Gauss-Jordan row operations to find the new basic solution. Gauss-Jordan row operations 1.Pivot row  Replace the leaving variable in the Basic column with the entering variable.  New pivot row = Current pivot row ÷ Pivot element 2.All other rows, including Z  New row = ( Current row ) – ( Its pivot coefficient) X ( New pivot row )

32. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3a: Use Gauss-Jordan row operations to find the new basic solution.

33. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3b: Use row operations to find the new basic solution. 0100105

34. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3c: Use row operations to find the new basic solution. 01001050100105 0010014

35. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3d: Use row operations to find the new basic solution. 01001050100105 00100140010014 001101

36. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1s 1 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z1-4-10000 1s 1 0111006 2s 2 0100105 3s 3 0010014 Step 3e: Use row operations to find the new basic solution. 01001050100105 0010014 4/1 0011-101 1/1 1 0 0 4 0 20 entering Leaving

37. Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z 1x 2 2x 1 3s 3 Basic Eq Varzx 1 x 2 s 1 s 2 s 3 RHS 0z10-104020 1s 1 0011-101 2x 1 0100105 3s 3 0010014 Step 3e: Use row operations to find the new basic solution. 0100105 001113 0011 01 10013021 entering Leaving

38. From the tableau we can test the optimality by looking to the Z-row. The Solution become optimal when all coff. in the Z-row are non-negative

39. From the final tableau we have the final optimal solution such as ( x1,x2,s1,s2,s3) = (5,1,0,0,3) X2 = 1 that means according to the X2 <=4 constraint I have 4 unit from resource ( S3 ), used just 1 unit and the remaining is slack = 3 This is clear from the sol. X2 =1 and S3 = 3

40. The Simplex Algorithm A Second Example

41. The problem Max z = 2 x 1 + x 2 + 3 x 3 Subj to: x 1 + x 2 + x 3  120 2 x 1 + 3 x 2 + 3 x 3  150 4 x 1 + 2 x 3  90 x 1, x 2, x 3  0

42. The augmented problem Max z = 2 x 1 + x 2 + 3 x 3 Subj to: x 1 + x 2 + x 3 + s 1 = 120 2 x 1 + 3 x 2 + 3 x 3 + s 2 = 150 4 x 1 + 2 x 3 + s 3 = 90 x 1, x 2, x 3, s 1, s 2, s 3  0

43. The Standard Form Max z - 2 x 1 - x 2 - 3 x 3 = 0 Subj to: x 1 + x 2 + x 3 + s 1 = 120 2 x 1 + 3 x 2 + 3 x 3 + s 2 = 150 4 x 1 + 2 x 3 + s 3 = 90 x 1, x 2, x 3, s 1, s 2, s 3  0

44. The Initial Tableau Basic Eq Varzx 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1-2-1-300 0 0 1s 1 0111100 120 2s 2 0233010 150 3s 3 0402001 90

45. Begin First Iteration Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 -2 -1 -3 0 0 0 0 1s 1 0 1 1 1 1 0 0 120 2s 2 0 2 3 3 0 1 0 150 3s 3 0 4 0 2 0 0 1 90 entering variable ratio 120/1 150/3 90/2

46. Start new solution – pivot row Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 -2 -1 -3 0 0 0 0 1s 1 0 1 1 1 1 0 0 120 2s 2 0 2 3 3 0 1 0 150 3x 3 0 2 0 1 0 0 1/2 45

47. New solution – Eq. 2 Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 -2 -1 -3 0 0 0 0 1s 1 0 1 1 1 1 0 0 120 2s 2 0 -4 3 0 0 1 -3/2 15 3x 3 0 2 0 1 0 0 1/2 45

48. New solution – Eq. 1 Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 -2 -1 -3 0 0 0 0 1s 1 0 -1 1 0 1 0 -1/2 75 2s 2 0 -4 3 0 0 1 -3/2 15 3x 3 0 2 0 1 0 0 1/2 45

49. New solution – Eq. 0 Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 4 -1 0 0 0 3/2 135 1s 1 0 -1 1 0 1 0 -1/2 75 2s 2 0 -4 3 0 0 1 -3/2 15 3x 3 0 2 0 1 0 0 1/2 45

50. New iteration Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 4 -1 0 0 0 3/2 135 1s 1 0 -1 1 0 1 0 -1/2 75 2s 2 0 -4 3 0 0 1 -3/2 15 3x 3 0 2 0 1 0 0 1/2 45 entering variable ratio 75/1 15/3

51. Pivot row operation Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 4 -1 0 0 0 3/2 135 1s 1 0 -1 1 0 1 0 -1/2 75 2x 2 0 -4/3 1 0 0 1/3 -1/2 5 3x 3 0 2 0 1 0 0 1/2 45

52. Next solution Basic Eq Varz x 1 x 2 x 3 s 1 s 2 s 3 RHS 0z1 8/3 0 0 0 1/3 1 140 1s 1 0 1/3 0 0 1 -1/3 0 70 2x 2 0 -4/3 1 0 0 1/3 -1/2 5 3x 3 0 2 0 1 0 0 1/2 45 Optimal! Solution (x1=0,x2=5,x3=45,s1=70,s2=0,s3= 0, z=140)

53. After the Simplex What can go wrong? Equality and >= Constraints ( Big-M method ) Minimization rather than maximization Unbounded solutions Alternate solutions No feasible solution Big-M Method

54. LP With E quality and/or >= Constraints Big-M method The only serious problem introduced by the other forms for functional constraints( =, >= forms ) lies in identifying an Initial BF solution. The standard approach that is used for all these cases is the artificial-variable technique. This technique constructs a more convenient artificial problem by introducing a dummy variable ( called an artificial variable ) into each constraint that needs one.

55. LP With Equality or >= Constraints Big-M method Equality constraints: a i1 X 1 + a i2 X 2 + …. + a in X n = b i actually is equivalent to a pair of inequality constraints : ai1 X1 + ai2 X2 + …. + a in X n >= bi ai1 X1 + ai2 X2 + …. + a in X n <= bi But rather than making this substitution and thereby increasing the number of constraints, it is more convenient to use the Artificial-variable technique.

56. Example Max z = 3x 1 + 5x 2 subject to: x 1 <=4 2x 2 <= 12 3x 1 + 2x 2 = 18 x 1, x 2 >=0 Max z - 3x 1 - 5x 2 = 0 subject to: x 1 + s 1 = 4 2x 2 + s 2 = 12 3x 1 + 2x 2 = 18 x 1, x 2 >=0 Augmented Form

57. Example continue 1.Apply the artificial Variable Technique by introducing a nonnegative artificial variable 3x 1 + 2x 2 + R1 = 18 2. Assign an overwhelming penalty to have R1 > 0 by changing the objective function to Z = 3x1 + 5x2 – M R1 Where M represent a huge positive number. ( this method of forcing R1 to be R1 = 0 in the optimal solution is called the Big M method. )

58. Example continue Now find the optimal solution for the real problem by applying the simplex method to the artificial problem, starting with the following final initial BF solution: Non basic variables : x1 = 0, x2 = 0 Basic Variables : S1 = 4, S2 = 12, R1 = 18. The Artificial problem : Max Z = 3x1 + 5x2 – M R1 subject to: x1 <= 4 2x2 <= 12 3x1 + 2x2 + R1 = 18

59. Example continue Now Max Z = 3x1 + 5x2 – M R1 Max Z – 3x1 – 5x2 + M R1 = 0 3x1 + 2x2 + R1 = 18 R1 = 18 – 3x1 – 2x2 Substitute R1 in Z we have Z – 3x1 – 5x2 + M ( 18 – 3x1 – 2x2 ) = 0 Z – 3x1 – 5x2 + 18M – 3M x1- 2M x2 = 0 Z – ( 3M + 3 ) x1 – ( 2M + 5 ) x2 = -18M

60. The Initial Tableau Basic Eq Varzx 1 x 2 s 1 s 2 R1 RHS 0z1 -3M -3 -2M – 5 0 0 0 -18M 1s 1 01 0 1 0 0 4 2s 2 00 2 0 1 0 12 3R103 2 0 0 1 18 Max Z – ( 3M + 3 ) x1 – ( 2M + 5 ) x2 = -18M subject to: x 1 + s 1 = 4 2x 2 + s 2 = 12 3x 1 + 2x 2 + R1 = 18 x 1, x 2 >=0

61. LP Minimization – The Problem Min z = 12x 1 + 5x 2 + 10x 3 subject to: x 1 - x 3 >= 30 x 1 + 2x 2 >= 20 x 2 + 4x 3 = 40 x 1, x 2, x 3 >=0

62. LP Minimization Min z = 12x 1 + 5x 2 + 10x 3 subject to: x 1 - x 3 = 30 x 1 + 2x 2 = 20 x 2 + 4x 3 = 40 x 1, x 2, x 3 >=0 - S1 -s2 Subtract surplus variables x 4, x 5 >= 0 +R1 +R2 +R3 + MR1 + MR2 + MR3 x 4, x 5, x 6, x 7, x 8 >= 0 Add artificial variables Augmented Problem Animated

63. Basis VarZ x 1 x 2 x 3 s1 s2 R1 R2 R3 RHS z1 -12 -5 -10 0 0 - M -M -M0 R10 1 0 -1 -1 0 1 0 0 30 R2 0 1 2 0 0 -1 0 1 0 20 R3 0 0 1 4 0 0 0 0 1 40 Z new 1 -12 +2M -5+3M -10+3M -M -M 0 0 0 90M Animated

64. Step 1: ( the Optimality test for Minimization Problems ) Optimal when all values in Z-Row are non-positive Entering Variable: Select the non-basic variable having the largest positive coefficient in the objective function row to enter the basis.. Leaving variable : the same as before Animated

65. Unbounded solutions Alternate solutions No feasible solution Degeneracy Special Cases in LPP

66. Unbounded Solutions Max 2x 1 + 4x 2 Subj. to: x 1 + x 2 >= 10 x 1 – x 2 <= 20 x 1, x 2 >= 0 102030 20 10 5 x feasible region Animated

67. Basis VarZ x 1 x 2 x 3 x 4 x 5 RHS z1 -2 -4 00 - M 0 x 5 0 1 1 -1 0 1 10 x 4 0 1 -1 0 1 0 20 z new 1 -2+M –4+M –M 0 0 10M More of Unbounded Solutions x 3 enters basis -x 3 + x 5 = 10 Unbounded Solution: no variable qualifies to be the leaving basic variable! Animated

68. Degeneracy Example max z = 3 x1 + 9 x2 Sto: X1 + 4 x2 <= 8 X1 + 2 x2 <= 4 X1,x2 >= 0 Cycling or circling

69. Alternate Solutions Max 3x 1 + x 2 Subj. to: x 1 + x 2 <= 20 3x 1 + x 2 <= 30 x 1, x 2 >= 0 102030 30201053020105 Animated

70. Basis Var Z x 1 x 2 s 1 S 2 RHS z 1 -3 -1 0 0 0 s 1 0 1 1 1 0 20 S 2 0 3 1 0 1 30 More of Alternate Solutions x 1 enters basis Animated s 2 leaves

71. Basis VarZ x 1 x 2 S 1 S 2 RHS z1 0 0 0 1 30 x 3 0 0 2/3 1 -1/3 10 x 1 0 1 1/3 0 1/3 10 More of Alternate Solutions optimal; however x 2 can enter basis Animated S 1 leaves

72. Basis VarZ x 1 x 2 x 3 x 4 RHS z1 0 0 01 30 x 2 0 0 1 3/2 -1/2 15 x 1 0 1 0 -1/2 1/2 5 More of Alternate Solutions Animated

73. No Feasible Solution Min 2x 1 + 4x 2 Subj. to: x 1 + x 2 <= 10 x 1 + x 2 >= 20 x 1, x 2 >= 0 102030 20 10 5 Animated Min 2x 1 + 4x 2 Subj. to: x 1 + x 2 +s1 = 10 x 1 + x 2 - S1 + R1 = 20 x 1, x 2 >= 0

74. Basis VarZ x 1 x 2 s1 S1 R1 RHS z1 -2 -4 0 0 - M -M s10 1 1 1 0 0 10 R1 0 1 1 0 -1 1 20 More of No Feasible Solution z new 1 -2+M –4+M 0 -M 0 20M x 1 enters basis Animated S 1 leaves

75. Basis VarZ x 1 x 2 x 3 x 4 x 5 RHS z1 0 -2 2-M -M 0 20 -10M x 1 0 1 1 1 0 0 10 R1 0 0 0 -1 -1 1 10 More of No Feasible Solution No feasible solution: If unable to drive all the artificial variables from the basis, by the time the optimality conditions are encountered, then there is no feasible solution. Animated

76. Summary If all values in the pivotal column are zeros or negative then the solution is unbounded If all RHS negative then the solution is infeasible.

77. IF a slack variable take a positive value, this mean that there is a resource that not used totally. Unit Worth ( Shadow Price ) of a resource is measured in terms of the units of the objective and it measures the improvement of the objective, as a result of increasing the resource by 1 unit.( It is the shadow price of the resource or the dual price ). Any resource that is not fully consumed is called Abundant resource and its shadow price =0. Any resource that is fully consumed by the activity is called Scarce resource. Unit worth or shadow price is determined using the coefficient of the slack variables in the final Z-Row.