1. Design and Analysis of Algorithms
Yoram Moses
Lecture 7
April 22, 2010

2. Linear Programming

3. Standard Form Maximization All constraints are ≤ inequalities
All variables are non-negative
Objective function
Constraints
Non-negativity constraints

4. Slack Form Maximization n+m variables and m constraints
n “regular variables” and m “slack variables”
All variables are non-negative
All constraints, except for non-negativity, are equalities
Slack variables appear only on LHS of equality constraints
Each equality constraint has a single unique slack variable

5. Conversion to Slack Form: Example
Standard LP
Slack LP

6. Basic vs. Non-basic Variables
Slack variables  basic variables
Regular variables  non-basic variables
B = set of basic variables, N = set of non-basic variables
Ex: B = { 4,5,6 }, N = { 1,2,3 }
|B| = m, |N| = n
For a solution x, xB = basic part of x, xN = non-basic part of x

7. Short Matrix Form |B| = m, |N| = n
Constraints can be indexed by members of B
For a solution x,
xB = basic part of x (i.e., xi for all i  B)
xN = non-basic part of x (i.e., xj for all j  N)

8. Basic Solutions Every slack form is associated with a basic solution:
All non-basics are set to 0 (i.e., xN = 0)
All basics are set to corresponding free coefficients (i.e., xB = b)
Ex: x1 = 0, x2 = 0, x3 = 0, x4 = 30, x5 = 24, x6 = 36

9. Tight Constraints & Constraint Violation
Tight constraint: One in which the basic variable is forced to 0 in the basic solution
The corresponding free coefficient is 0.
Violated constraint: One in which the basic variable is forced to be negative in the basic solution.
the corresponding free coefficient is negative.

10. Basic Feasible Solutions
Basic Feasible Solution (BFS): A basic solution, which is feasible
Easy fact: A basic solution is feasible if and only if all free coefficients are non-negative.
Lemma: Every BFS corresponds to a vertex of the feasible region polytope.

11. Simplex Algorithm: Overview
Works in iterations
At each iteration: transform one slack form P into an equivalent slack form P’
Objective value of basic solution of P’ is always at least as good as that of P
Stop when reaching a local optimum

12. Moving to an Equivalent Slack Form
How to increase the objective value of the BFS?
Increase from 0 the value of some non-basic variable, whose coefficient in the objective function is positive.
By how much?
As much as possible without violating any of the constraints.
objective: 0
Can increase x1 by at most 9.
Objective increases to 27.

13. Switching Basic with Non-basic
Suppose that we increase a non-basic variable xi until some constraint j becomes tight
xj, the basic variable of the constraint j, becomes 0
We can thus switch between xi and xj
xi will become the basic variable of constraint j
xj will become a non-basic
When increasing x1 to 9, x6 becomes 0.
We switch between x1 and x6.

14. Switching Basic with Non-basic
We write x1 as a function of other non-basics and x6:

15. Switching Basic with Non-basic
We rewrite the objective function:

16. Switching Basic with Non-basic
We rewrite all the constraints as well, and obtain the following equivalent linear program:

17. Switching Basic with Non-basic
The new LP is equivalent to the previous LP
We just rewrote x1 in terms of other variables
Basic solution: x1 = 9, x2 = 0, x3 = 0, x4 = 21, x5 = 6, x6 = 0
New objective value: 27

18. Example continued objective: 27 objective: 27.75 Choose x3
First constraint to become tight is constraint 3.
objective: 27.75

19. Example continued objective: 27.75 objective: 28 Choose x2
First constraint to become tight is constraint 2.
objective: 28

20. Example continued objective: 28
No more non-basics whose coefficient in the objective function is positive
We stop and output basic solution as the optimal solution
Solution: x1 = 8, x2 = 4, x3 = 0, x4 = 18, x5 = 0, x6 = 0. Value: 28.

21. Back to the Standard Form
objective: 28
Solution: x1 = 8, x2 = 4, x3 = 0. Value: 28.

22. Pivoting Pivot: a single iteration of the simplex algorithm
Choose a non-basic variable xi whose coefficient in the objective function is > 0
xi is called the “entering variable”
If more than one exists, choose one according to some pivoting rule
Find the first constraint j that will be violated when we increase the value of xi from 0
Make xi the basic variable of constraint j, and make xj a non-basic variable
xj is called the “leaving variable”
Write xi as a function of xj and the other non-basics
Rewrite the objective function and the constraints

23. Pivoting: Geometric Intuition
Lemma: Pivoting corresponds to moving from one vertex of the feasible region to a neighbor vertex, whose objective value is at least as good.

24. Unbounded Programs
Sometimes it is possible to increase the value of the entering value unboundedly, without violating any constraint
In this case the optimal solution of the LP is unbounded
Pivot will return “unbounded”

25. The Simplex Algorithm find an initial BFS
while there is a non-basic variable whose coefficient in the objective function is > 0
run pivot
if pivot returns “unbounded” return “unbounded”
return BFS of current slack form as the optimal solution
Geometric view: Repeatedly move from a vertex of the feasible region to a better neighbor vertex, until a local maximum is reached.
Initial BFS is found by solving an auxiliary linear program (read section in book)

26. Simplex Analysis: Correctness
If LP is infeasible, S’x will fail to find an initial BFS
If LP is unbounded, Pivot will return “unbounded”
If LP has a bounded optimal solution, it has one at a vertex
Simplex will reach a local maximum vertex
Local maximum vertex must be a global maximum
Hence, Simplex will output an optimal solution

27. Simplex Analysis: Running Time
We have not specified the two “pivoting rules”
For choosing the entering variable
For choosing the leaving variable
Degeneracy: objective value of BFS does not improve in an invocation of Pivot
Unwise pivoting rules may lead to infinite loops (i.e., everlasting degeneracy)

28. Degeneracy objective: 0 entering: x1 leaving: x4

29. Pivoting Rules
Bland’s rule: choose entering/leaving variable with smallest index.
Lemma: If Simplex uses Bland’s rule, it never cycles.
Conclusion: Simplex has at most iterations.
Theorem [Kalai]: There is a randomized pivoting rule, with which Simplex runs for a sub-exponential number of iterations in expectation.
Open problem: Is there a pivoting rule with which Simplex runs in polynomial time?

30. End of Lecture 8