Design and Analysis of Algorithms Yoram Moses Lecture 7 April 22, 2010 http://www.ee.technion.ac.il/courses/046002

Linear Programming

Standard Form Maximization All constraints are ≤ inequalities All variables are non-negative Objective function Constraints Non-negativity constraints

Slack Form Maximization n+m variables and m constraints n “regular variables” and m “slack variables” All variables are non-negative All constraints, except for non-negativity, are equalities Slack variables appear only on LHS of equality constraints Each equality constraint has a single unique slack variable

Conversion to Slack Form: Example Standard LP Slack LP

Basic vs. Non-basic Variables Slack variables  basic variables Regular variables  non-basic variables B = set of basic variables, N = set of non-basic variables Ex: B = { 4,5,6 }, N = { 1,2,3 } |B| = m, |N| = n For a solution x, xB = basic part of x, xN = non-basic part of x

Short Matrix Form |B| = m, |N| = n Constraints can be indexed by members of B For a solution x, xB = basic part of x (i.e., xi for all i  B) xN = non-basic part of x (i.e., xj for all j  N)

Basic Solutions Every slack form is associated with a basic solution: All non-basics are set to 0 (i.e., xN = 0) All basics are set to corresponding free coefficients (i.e., xB = b) Ex: x1 = 0, x2 = 0, x3 = 0, x4 = 30, x5 = 24, x6 = 36

Tight Constraints & Constraint Violation Tight constraint: One in which the basic variable is forced to 0 in the basic solution The corresponding free coefficient is 0. Violated constraint: One in which the basic variable is forced to be negative in the basic solution. the corresponding free coefficient is negative.

Basic Feasible Solutions Basic Feasible Solution (BFS): A basic solution, which is feasible Easy fact: A basic solution is feasible if and only if all free coefficients are non-negative. Lemma: Every BFS corresponds to a vertex of the feasible region polytope.

Simplex Algorithm: Overview Works in iterations At each iteration: transform one slack form P into an equivalent slack form P’ Objective value of basic solution of P’ is always at least as good as that of P Stop when reaching a local optimum

Moving to an Equivalent Slack Form How to increase the objective value of the BFS? Increase from 0 the value of some non-basic variable, whose coefficient in the objective function is positive. By how much? As much as possible without violating any of the constraints. objective: 0 Can increase x1 by at most 9. Objective increases to 27.

Switching Basic with Non-basic Suppose that we increase a non-basic variable xi until some constraint j becomes tight xj, the basic variable of the constraint j, becomes 0 We can thus switch between xi and xj xi will become the basic variable of constraint j xj will become a non-basic When increasing x1 to 9, x6 becomes 0. We switch between x1 and x6.

Switching Basic with Non-basic We write x1 as a function of other non-basics and x6:

Switching Basic with Non-basic We rewrite the objective function:

Switching Basic with Non-basic We rewrite all the constraints as well, and obtain the following equivalent linear program:

Switching Basic with Non-basic The new LP is equivalent to the previous LP We just rewrote x1 in terms of other variables Basic solution: x1 = 9, x2 = 0, x3 = 0, x4 = 21, x5 = 6, x6 = 0 New objective value: 27

Example continued objective: 27 objective: 27.75 Choose x3 First constraint to become tight is constraint 3. objective: 27.75

Example continued objective: 27.75 objective: 28 Choose x2 First constraint to become tight is constraint 2. objective: 28

Example continued objective: 28 No more non-basics whose coefficient in the objective function is positive We stop and output basic solution as the optimal solution Solution: x1 = 8, x2 = 4, x3 = 0, x4 = 18, x5 = 0, x6 = 0. Value: 28.

Back to the Standard Form objective: 28 Solution: x1 = 8, x2 = 4, x3 = 0. Value: 28.

Pivoting Pivot: a single iteration of the simplex algorithm Choose a non-basic variable xi whose coefficient in the objective function is > 0 xi is called the “entering variable” If more than one exists, choose one according to some pivoting rule Find the first constraint j that will be violated when we increase the value of xi from 0 Make xi the basic variable of constraint j, and make xj a non-basic variable xj is called the “leaving variable” Write xi as a function of xj and the other non-basics Rewrite the objective function and the constraints

Pivoting: Geometric Intuition Lemma: Pivoting corresponds to moving from one vertex of the feasible region to a neighbor vertex, whose objective value is at least as good.

Unbounded Programs Sometimes it is possible to increase the value of the entering value unboundedly, without violating any constraint In this case the optimal solution of the LP is unbounded Pivot will return “unbounded”

The Simplex Algorithm find an initial BFS while there is a non-basic variable whose coefficient in the objective function is > 0 run pivot if pivot returns “unbounded” return “unbounded” return BFS of current slack form as the optimal solution Geometric view: Repeatedly move from a vertex of the feasible region to a better neighbor vertex, until a local maximum is reached. Initial BFS is found by solving an auxiliary linear program (read section in book)

Simplex Analysis: Correctness If LP is infeasible, S’x will fail to find an initial BFS If LP is unbounded, Pivot will return “unbounded” If LP has a bounded optimal solution, it has one at a vertex Simplex will reach a local maximum vertex Local maximum vertex must be a global maximum Hence, Simplex will output an optimal solution

Simplex Analysis: Running Time We have not specified the two “pivoting rules” For choosing the entering variable For choosing the leaving variable Degeneracy: objective value of BFS does not improve in an invocation of Pivot Unwise pivoting rules may lead to infinite loops (i.e., everlasting degeneracy)

Degeneracy objective: 0 entering: x1 leaving: x4

Pivoting Rules Bland’s rule: choose entering/leaving variable with smallest index. Lemma: If Simplex uses Bland’s rule, it never cycles. Conclusion: Simplex has at most iterations. Theorem [Kalai]: There is a randomized pivoting rule, with which Simplex runs for a sub-exponential number of iterations in expectation. Open problem: Is there a pivoting rule with which Simplex runs in polynomial time?

End of Lecture 8