1. 3.3 Implementation (1) naive implementation (2) revised simplex method
(3) full tableau implementation
(1) Naive implementation :
Given basis 𝐵. Compute 𝑝 ′ = 𝑐 𝐵 ′ 𝐵 −1 ( solve 𝑝 ′ 𝐵= 𝑐 𝐵 ′)
Choose 𝑗 such that 𝑐 𝑗 = 𝑐 𝑗 − 𝑐 𝐵 ′ 𝐵 −1 𝐴 𝑗 = 𝑐 𝑗 −𝑝′ 𝐴 𝑗 <0, 𝑗∈𝑁.
𝑥 𝐵 𝑥 𝑁 = 𝐵 −1 𝑏 0 +𝜃 𝑑 𝐵 𝑑 𝑁 = 𝐵 −1 𝑏 0 +𝜃 − 𝐵 −1 𝐴 𝑗 𝑒 𝑗
Let 𝑢= 𝐵 −1 𝐴 𝑗 ( solve 𝐵𝑢= 𝐴 𝑗 )
Determine 𝜃 ∗ {= min 𝑖∈𝐵: 𝑢 𝑖 >0 ( 𝐵 −1 𝑏 𝑖 / 𝑢 𝑖 )}
Let 𝐵(𝑙) be the index of the leaving basic variable.
Replace 𝐴 𝐵(𝑙) by 𝐴 𝑗 in the basis and update basis indices.
Update solution 𝑥.
Naive implementation itself is frequently called the revised simplex method compared to the full tableau implementation.
Linear Programming 2017

2. (2) Revised simplex method :
Naive implementation needs to find 𝑝 ′ = 𝑐 𝐵 ′ 𝐵 −1 and 𝑢= 𝐵 −1 𝐴 𝑗 ( or solve 𝑝 ′ 𝐵= 𝑐 𝐵 ′, 𝐵𝑢= 𝐴 𝑗 ) in each iteration.
Update 𝐵 −1 efficiently so that computational burden is reduced. (compute 𝑐 𝐵 ′ 𝐵 −1 and 𝐵 −1 𝐴 𝑗 easily. Similar idea can be used to update 𝐵 efficiently and find 𝑝,𝑢 vectors easily.)
𝐵= 𝐴 𝐵(1) ,…, 𝐴 𝐵(𝑚) → 𝐵 = 𝐴 𝐵(1) ,…, 𝐴 𝐵(𝑙−1) , 𝐴 𝑗 , 𝐴 𝐵(𝑙+1) ,…, 𝐴 𝐵(𝑚)
𝐵 −1 𝐵 = 𝑒 1 ,…, 𝑒 𝑙−1 ,𝑢, 𝑒 𝑙+1 ,…, 𝑒 𝑚 , 𝑢= 𝐵 −1 𝐴 𝑗
Premultiply elementary row operation matrices 𝑄 𝑘 𝑄 𝑘−1 … 𝑄 1 ≡𝑄 to 𝐵 −1 𝐵 so that 𝑄 𝐵 −1 𝐵 =𝑄 𝑒 1 ,…, 𝑒 𝑙−1 ,𝑢, 𝑒 𝑙+1 ,…, 𝑒 𝑚 =𝐼
 𝑄 𝐵 −1 = 𝐵 −1
Hence applying the same row operations ( to convert 𝑢 to 𝑒 𝑙 ) to 𝐵 −1 results in 𝐵 −1 . ( see example in the text)
Linear Programming 2017

3. (3) Full tableau implementation : 𝐴𝑥=𝑏  𝐵 −1 𝐴𝑥= 𝐵 −1 𝑏
𝐴𝑥=𝑏  𝐵 −1 𝐴𝑥= 𝐵 −1 𝑏
maintain 𝐵 −1 𝑏: 𝐵 −1 𝐴 1 ,…, 𝐵 −1 𝐴 𝑛
can read current b.f.s. 𝑥 from 𝐵 −1 𝑏 and 𝑑 𝐵 =− 𝐵 −1 𝐴 𝑗 .
Update to 𝐵 −1 𝑏:𝐴 . We know 𝐵 −1 =𝑄 𝐵 −1 .
Hence 𝐵 −1 𝑏:𝐴 =𝑄 𝐵 −1 𝑏:𝐴 =𝑄 𝐵 −1 𝑏: 𝐵 −1 𝐴 .
So apply row operations, which convert 𝑢= 𝐵 −1 𝐴 𝑗 to 𝑒 𝑙 , to the matrix 𝐵 −1 𝑏: 𝐵 −1 𝐴 .
(To find exiting column 𝐴 𝐵(𝑙) and step size 𝜃 ∗ , compare 𝑥 𝐵(𝑖) 𝑢 𝑖 for 𝑢 𝑖 >0, 𝑖=1,…,𝑚)
(pivot column, pivot row, pivot element)
Also maintain and update information about reduced costs and objective value.
Linear Programming 2017

4. Currently 0-th row is 0:𝑐′ −𝑔′ 𝑏:𝐴 , where 𝑔 ′ = 𝑐 𝐵 ′ 𝐵 −1 .
To update the 0-th row, add 𝑘× (pivot row) to 0-th row for some scalar 𝑘 so that the coefficient for the entering variable 𝑥 𝑗 in 0-th row becomes 0.
Currently 0-th row is 0:𝑐′ −𝑔′ 𝑏:𝐴 , where 𝑔 ′ = 𝑐 𝐵 ′ 𝐵 −1 .
Let column 𝑗 be the pivot column and row 𝑙 be the pivot row.
Pivot row 𝑙 of 𝐵 −1 𝑏: 𝐵 −1 𝐴 is ℎ′ 𝑏:𝐴 where ℎ′ is 𝑙−𝑡ℎ row of 𝐵 −1 .
Hence, after adding, new 0-th row still represented as
0:𝑐′ −𝑝′ 𝑏:𝐴 for some 𝑝, while 𝑐 𝐵 (𝑙) −𝑝′ 𝐴 𝐵 𝑙 = 𝑐 𝑗 −𝑝′ 𝐴 𝑗 =0.
Linear Programming 2017

5. Now for 𝑥 𝑖 basic, 𝑖≠𝑙, have 𝑐 𝑖 remains at 0.
(continued)
Now for 𝑥 𝑖 basic, 𝑖≠𝑙, have 𝑐 𝑖 remains at 0.
( 𝑐 𝑖 = 0 before the pivot. Have 𝐵 −1 𝐴 𝐵(𝑖) = 𝑒 𝑖 , 𝑖≠𝑙.
Hence 𝐵 −1 𝐴 𝐵(𝑖) 𝑙 =0, 𝑖≠𝑙 )
 𝑐 𝐵 (𝑖) −𝑝′ 𝐴 𝐵 𝑖 =0, for all 𝑖∈ 𝐵
 𝑐 𝐵 ′− 𝑝 ′ 𝐵 =0
 𝑝′= 𝑐 𝐵 ′ 𝐵 −1
 new 0-th row is 0:𝑐′ − 𝑐 𝐵 ′ 𝐵 −1 𝑏:𝐴 as desired.
Linear Programming 2017

6. See text example 3.5 for more iterations.
𝑥 4 =
𝑥 5 =
𝑥 6 =
𝑥 4 =
𝑥 1 =
𝑥 6 =
See text example 3.5 for more iterations.
Linear Programming 2017

7. (1) Tableau form also can be derived as the following :
(Remarks)
(1) Tableau form also can be derived as the following :
Given min 𝑐 ′ 𝑥, 𝐴𝑥=𝑏, 𝑥≥0,
let 𝐴= 𝐵:𝑁 , 𝑥= 𝑥 𝐵 , 𝑥 𝑁 , 𝑐= 𝑐 𝐵 , 𝑐 𝑁 , where 𝐵 is the current basis.
Also let 𝑧 denote the value of the objective function, i. e. 𝑧= 𝑐 ′ 𝑥.
Since all feasible solutions must satisfy 𝐴𝑥=𝑏, they must satisfy 𝐵:𝑁 𝑥 𝐵 𝑥 𝑁 =𝑏  𝐵 𝑥 𝐵 +𝑁 𝑥 𝑁 =𝑏
 𝐵 𝑥 𝐵 =𝑏−𝑁 𝑥 𝑁  𝑥 𝐵 = 𝐵 −1 𝑏− 𝐵 −1 𝑁 𝑥 𝑁
𝐼 𝑥 𝐵 + 𝐵 −1 𝑁 𝑥 𝑁 = 𝐵 −1 𝑏
or in matrix form, 𝐼: 𝐵 −1 𝑁 𝑥 𝐵 𝑥 𝑁 = 𝐵 −1 𝑏
Linear Programming 2017

8. 𝑧= 𝑐 ′ 𝑥= 𝑐 𝐵 ′ 𝑥 𝐵 + 𝑐 𝑁 ′ 𝑥 𝑁 = 𝑐 𝐵 ′ 𝐵 −1 𝑏− 𝐵 −1 𝑁 𝑥 𝑁 + 𝑐 𝑁 ′ 𝑥 𝑁
(continued)
Since all feasible solutions must satisfy these equations, we can plug in the expression for 𝑥 𝐵 into the objective function to obtain
𝑧= 𝑐 ′ 𝑥= 𝑐 𝐵 ′ 𝑥 𝐵 + 𝑐 𝑁 ′ 𝑥 𝑁 = 𝑐 𝐵 ′ 𝐵 −1 𝑏− 𝐵 −1 𝑁 𝑥 𝑁 + 𝑐 𝑁 ′ 𝑥 𝑁
= 𝑐 𝐵 ′ 𝐵 −1 𝑏+0′ 𝑥 𝐵 + 𝑐 𝑁 ′− 𝑐 𝐵 ′ 𝐵 −1 𝑁 𝑥 𝑁
= 𝑐 𝐵 ′ 𝐵 −1 𝑏+ 𝑐 𝑁 ′−𝑝′𝑁 𝑥 𝑁 , where 𝑝 ′ = 𝑐 𝐵 ′ 𝐵 −1 or 𝑝 ′ 𝐵= 𝑐 𝐵 ′.
Hence obtain the tableau with respect to the current basis B
𝑧− 𝑐 𝐵 ′ 𝐵 −1 𝑏= 0 ′ 𝑥 𝐵 + 𝑐 𝑁 ′− 𝑐 𝐵 ′ 𝐵 −1 𝑁 𝑥 𝑁
𝐵 −1 𝑏= 𝐼 𝑥 𝐵 + 𝐵 −1 𝑁 𝑥 𝑁 = 𝐵 −1 𝐴 𝑥
Linear Programming 2017

9. (continued) (2) Tableau also can be obtained using the following logic
(continued) (2) Tableau also can be obtained using the following logic. Note that elementary row operations on equations do not change the set of feasible solutions, but the representation is changed. Suppose current basis 𝐵 is known. From −𝑧+ 𝑐 𝐵 ′ 𝑥 𝐵 + 𝑐 𝑁 ′ 𝑥 𝑁 =0 𝐵 𝑥 𝐵 +𝑁 𝑥 𝑁 =𝑏 We compute multiplier vector 𝑝 for the constraints by solving 𝑝′𝐵= 𝑐 𝐵 ′ 𝑝′= 𝑐 𝐵 ′ 𝐵 −1 . Then we take linear combination of constraints using weight vector −𝑝 and add it to objective row, resulting in −𝑧+0′ 𝑥 𝐵 + 𝑐 𝑁 ′− 𝑐 𝐵 ′ 𝐵 −1 𝑁 𝑐 𝑁 ′− 𝑐 𝐵 ′ 𝐵 −1 𝑁 𝑥 𝑁 =− 𝑐 𝐵 ′ 𝐵 −1 𝑏, or 𝑧− 𝑐 𝐵 ′ 𝐵 −1 𝑏= 0 ′ 𝑥 𝐵 + 𝑐 𝑁 ′− 𝑐 𝐵 ′ 𝐵 −1 𝑁 𝑥 𝑁 . Also, we multiply 𝐵 −1 on both sides of constraints  𝐼 𝑥 𝐵 + 𝐵 −1 𝑁 𝑥 𝑁 = 𝐵 −1 𝑏. Here, the 𝑖−𝑡ℎ row of 𝐵 −1 is the weight vector we use for weighted sum of constraints to obtain updated 𝑖−𝑡ℎ constraint.
Linear Programming 2017

10. (continued) (3) Updating the tableau using elementary row operations is tantamount to updating the system of linear equations using elementary row operations so that the current basic feasible solution for updated basis 𝐵 can be easily identified. Hence the submatrix of the updated 𝐴 matrix corresponding to the new basis 𝐵 constitutes an identity matrix. However, the set of feasible solutions does not change when we apply elementary row operations to the system of equations.
Linear Programming 2017

11. Practical Performance Enhancements
In commercial LP solvers, 𝐵 −1 is not updated in the revised simplex method. Instead we update the representation of 𝐵 as 𝐵=𝐿𝑈, where 𝐿 is lower triangular and 𝑈 is upper triangular matrices (with some row permutation allowed, called 𝐿𝑈 decomposition, triangular factorization).
We solve the system (with proper updates), 𝑝′𝐿𝑈= 𝑐 𝐵 ′, 𝐿𝑈𝑢= 𝐴 𝑗 , each system takes 𝑂 𝑚 2 to solve and numerically more stable than using 𝐵 −1 . Moreover, less fill-in occurs in 𝐿𝑈 decomposition than in 𝐵 −1 , which is important when we solve large sparse problems.
Linear Programming 2017

12. 3.4 Anticycling 1. Lexicographic pivoting rule
Def : 𝑢∈ 𝑅 𝑛 is said to be lexicographically larger than 𝑣∈ 𝑅 𝑛 if 𝑢≠𝑣 and the first nonzero component of 𝑢−𝑣 is positive ( 𝑢 > 𝑣 )
Lexicographic pivoting rule
(1) choose entering 𝑥 𝑗 with 𝑐 𝑗 <0. Compute updated column 𝑢= 𝐵 −1 𝐴 𝑗 .
(2) For each 𝑖 with 𝑢 𝑖 >0, divide 𝑖−𝑡ℎ row of the tableau by 𝑢 𝑖 and choose lexicographically smallest row.
If row 𝑙 is smallest, 𝑥 𝐵(𝑙) leaves basis.
Linear Programming 2017

13. Suppose pivot column is the third one ( 𝑗=3)
Ex 3.7:
Suppose pivot column is the third one ( 𝑗=3)
ratio = 1/3 for 1st and 3rd row
third row is pivot row,
and 𝑥 𝐵(3) exits the basis.
Linear Programming 2017

14. (a) every row except 0-th remains lexicographically positive.
Thm 3.4 : Suppose the rows in the current simplex tableau is lexicographically positive except 0-th row and lexicographic rule is used, then
(a) every row except 0-th remains lexicographically positive.
(b) 0-th row strictly increases lexicographically.
(c) simplex method terminates finitely.
Linear Programming 2017

15. Then (𝑙−𝑡ℎ row) / 𝑢 𝑙 < (𝑖−𝑡ℎ row) / 𝑢 𝑖 , 𝑖≠𝑙, 𝑢 𝑖 >0
Pf)
(a) Suppose 𝑥 𝑗 enters, 𝑥 𝑙 leaves ( 𝑢 𝑙 >0, pivot row is l-th row. )
Then (𝑙−𝑡ℎ row) / 𝑢 𝑙 < (𝑖−𝑡ℎ row) / 𝑢 𝑖 , 𝑖≠𝑙, 𝑢 𝑖 >0
(𝑙−𝑡ℎ row)  (𝑙−𝑡ℎ row) / 𝑢 𝑙 ( lexicographically positive )
For 𝑖−𝑡ℎ row, 𝑖≠𝑙
(1) 𝑢 𝑖 <0 ; add pos. num.  (𝑙−𝑡ℎ row) to 𝑖−𝑡ℎ row  lex. pos.
(2) 𝑢 𝑖 >0 ; (new 𝑖−𝑡ℎ row) = (old 𝑖−𝑡ℎ row)− 𝑢 𝑖 𝑢 𝑙 ×(old 𝑙−𝑡ℎ row)
 𝑢 𝑖 old 𝑖−𝑡ℎ row 𝑢 𝑖 − old 𝑙−𝑡ℎ row 𝑢 𝑙 is lexicographically positive
 (new 𝑖−𝑡ℎ row) is lexicographically positive
(3) 𝑢 𝑖 =0 ; remains unchanged
(b) 𝑐 𝑗 <0  we add positive multiple of 𝑙−𝑡ℎ row to 0−𝑡ℎ row
(c) 0−𝑡ℎ row determined by current basis
 no basis repeated since 0-th row increases lexicographically
 finite termination. 
Linear Programming 2017

16. (2) Idea of lexicographic rule is related to the perturbation method.
Remarks :
(1) To have initial lexicographically positive rows, permute the columns (variables) so that the basic variables come first in the current tableau
(2) Idea of lexicographic rule is related to the perturbation method.
If no degenerate solution  objective value strictly decreases, hence no cycling ( decrease of objective function value is 𝜃 ∗ 𝑐 𝑗 ).
Hence add small positive 𝜀 𝑖 to 𝑥 𝐵(𝑖) , 𝑖=1,…,𝑚 to obtain
𝑥 𝐵 𝑖 = 𝐵 −1 𝑏 𝑖 + 𝜀 𝑖 , where 0< 𝜀 𝑚 ≪ 𝜀 𝑚−1 ≪…≪ 𝜀 2 ≪ 𝜀 1 ≪1.
Linear Programming 2017

17. (continued)
It can be shown that no degenerate solution appears in subsequent iterations ( think 𝜀 𝑖 ′𝑠 as symbols), hence cycling is avoided. Lexicographic rule is an implementation of the perturbation method without using 𝜀 𝑖 ′𝑠 explicitly.
Note that the coefficient matrix of 𝜀 𝑖 ′𝑠 and basic variables are all identity matrices. Hence the simplex iterations (elementary row operations) result in the same coefficient matrices.
Linear Programming 2017

18. 2. Bland’s rule ( smallest subscript rule )
(1) Choose smallest 𝑗 among the nonbasic variables having 𝑐 𝑗 <0 and let the column 𝐴 𝑗 enter the basis
(2) Out of all basic variables 𝑥 𝑖 ′𝑠 that are tied in the minimum ratio test for choosing an exiting variable, choose the one with the smallest value of index 𝑖.
Pf) see p.37-38, Vasek Chvatal, Linear Programming, Freeman, Note that the proof is given for the maximization problem.
Note that the lexicographic rule and the smallest subscript rule can be started and stopped at any time during the simplex iterations.
Linear Programming 2017

19. 3.5 Finding an initial b.f.s.
Given (P): min 𝑐 ′ 𝑥, s.t. 𝐴𝑥=𝑏, 𝑥≥ (assume 𝑏≥0 )
Introduce artificial variables and solve
(P-I) min 𝑦 1 + 𝑦 2 +…+ 𝑦 𝑚 , s.t. 𝐴𝑥+𝐼𝑦=𝑏, 𝑥≥0, 𝑦≥0
Initial b.f.s. : 𝑥=0, 𝑦=𝑏
If optimal value >0  (P) is infeasible
(If (P) feasible  (P-I) has a solution with 𝑦=0)
If optimal value =0  all 𝑦 𝑖 =0, so current optimal solution 𝑥 gives a feasible solution to (P). Drop 𝑦 𝑖 variables and use the original objective function.
However, we need a b.f.s. to use simplex. Have trouble if some artificial variables remain basic in the optimal basis.
Linear Programming 2017

20. Driving artificial variables out of the basis (in tableau form)
Pivot element
Linear Programming 2017

21. So bring 𝑥 𝑗 into the basis by pivoting (solution not changed)
Suppose 𝑥 𝐵(1) ,…, 𝑥 𝐵(𝑘) , 𝑘<𝑚 are basic variables which are from original variables.
Suppose artificial variable 𝑦 𝑖 is in the 𝑙−𝑡ℎ position of the basis (𝑙−𝑡ℎ component of the column for 𝑦 𝑖 in the optimal tableau is 1 and all other components are 0.) and 𝑙−𝑡ℎ component of 𝐵 −1 𝐴 𝑗 is nonzero for some nonbasic original variable 𝑥 𝑗 .
Then 𝐵 −1 𝐴 𝐵(1) ,…, 𝐵 −1 𝐴 𝐵(𝑘) = 𝑒 1 ,…, 𝑒 𝑘 and 𝐵 −1 𝐴 𝑗 are linearly independent  𝐴 𝐵(1) ,…, 𝐴 𝐵 𝑘 , 𝐴 𝑗 linearly independent
So bring 𝑥 𝑗 into the basis by pivoting (solution not changed)
If not exist 𝑥 𝑗 with 𝐵 −1 𝐴 𝑗 𝑙 ≠0  𝑔′𝐴=0′ (𝑔′ is 𝑙−𝑡ℎ row of 𝐵 −1 )
So rows of 𝐴 linearly dependent.
Also have 𝑔 ′ 𝑏=0 since 𝐴𝑥=𝑏 feasible.
Hence 𝑔 ′ 𝐴𝑥= 𝑔 ′ 𝑏 ′ 𝑥=0 redundant eq. and it is 𝑙−𝑡ℎ row of tableau  can eliminate it.
Linear Programming 2017

22. Remarks
Note that although we may eliminate the 𝑙−𝑡ℎ row of the current tableau, it may not imply that the 𝑙−𝑡ℎ row of the initial tableau is redundant.
To see this, suppose that 𝑖−𝑡ℎ artificial variable (with corresponding column 𝑒 𝑖 in the initial tableau) is in the 𝑙−𝑡ℎ position of the current basis matrix (hence 𝐵 −1 𝑒 𝑖 = 𝑒 𝑙 in the current tableau).
Let 𝑔′ be the 𝑙−𝑡ℎ row of 𝐵 −1 , then from 𝑔′ 𝑒 𝑖 =1, we know that 𝑖−𝑡ℎ component of 𝑔 is 1.
Then, from 𝑔 ′ 𝐴=0′ and 𝑔 ′ 𝑏=0, 𝑖−𝑡ℎ row of 𝑏:𝐴 can be expressed as a linear combination of the other rows, hence 𝑖−𝑡ℎ row in the original tableau is redundant. (What effect can be observed if there are additional artificial basic variable 𝑦 𝑘 ?)
Linear Programming 2017

23. Sometimes, we may want to retain the redundant rows when we solve the problem because we do not want to change the problem data so that we can perform sensitivity analysis later, i.e. change the data a little bit and solve the problem again.
Then the artificial variables corresponding to the redundant equations should remain in the basis (we should not drop the variable). It will not leave the basis in subsequent iterations since the corresponding row has all 0 coefficients.
If we do not drive the artificial variables out of the basis and perform the simplex iterations using the current b.f.s., it may happen that the value of the basic artificial variables become positive, hence gives an infeasible solution to the original problem.
To avoid this, modification on the simplex method is needed or we may use the bounded variable simplex method by setting the upper bounds of the remaining artificial variables to 0. ( lower bounds are 0 )
Linear Programming 2017

24. Two-phase simplex method
See text p.116 for two-phase simplex method
Big-M method
Use the objective function, min 𝑗=1 𝑛 𝑐 𝑗 𝑥 𝑗 +𝑀 𝑖=1 𝑚 𝑦 𝑖 , where 𝑀 is a large number.
see text sec. 3.6 for definition of 𝑘-dimensional simplex and the interpretation of simplex method by column geometry.
Linear Programming 2017

25. 3.7 Computational efficiency of the simplex method
See text sec. 1.6 Algorithms and operation counts.
Count the number of operations of an algorithm. Polynomial time vs. Exponential time. Size of the numbers involved may need to be considered (Chapter 8, 9).
Each iteration of the simplex method takes polynomial time of m, n and length of encoding of data.
But number of iterations is exponential in the worst case.
Empirically, number of iterations is 𝑂(𝑚) and 𝑂 log 𝑛 .
For pivoting rules, there exist counter examples on which simplex takes exponential number of iterations, hence simplex algorithm is not a polynomial time algorithm.
( Still, there exists a possibility that some other pivoting rules may provide polynomial running time. Though it may be very difficult to prove.)
Linear Programming 2017