Uri Zwick – Tel Aviv Univ. Randomized pivoting rules for the simplex algorithm Upper bounds TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA MDS summer school “The Combinatorics of Linear and Semidefinite Programming” August 14-16, 2012

Maximize a linear objective function subject to a set of linear equalities and inequalities Linear Programming Find the highest point in a polytope

Move up, along an edge to a neighboring vertex, until reaching the top The Simplex Algorithm [Dantzig (1947)]

Upper bound Theorem for polytopes [Klee (1964)] [McMullen (1971)] Number of vertices of a d-dimensional n-faceted polytope is at most:

Largest improvement Largest slope Dantzig’s rule – Largest modified cost Bland’s rule – avoids cycling Lexicographic rule – also avoids cycling Deterministic pivoting rules All known to require an exponential number of steps, in the worst-case Klee-Minty (1972) Jeroslow (1973), Avis-Chvátal (1978), Goldfarb-Sit (1979), …, Amenta-Ziegler (1996)

Algorithms for Linear Programming Ellipsoid (Khachiyan 1979) Interior-point (Karmakar 1984) Smoothed analysis based (ST’04, KS’06) Other (DV’08, …) Simplex (Dantzig 1947) No polynomial versions known Polynomial, but not strongly polynomial Not “combinatorial”

Is there a polynomial pivoting rule? Is the diameter polynomial?

Hirsch conjecture (1957): The diameter of a d-dimensional, n-faceted polytope is at most n−d Refuted Santos (2010)! Diameter is still believed to be polynomial Quasi-polynomial upper bound [Kalai-Kleitman (1992)]

Random-Edge Choose a random improving edge Randomized pivoting rules Random-Facet is sub-exponential! Random-Facet To be explained shortly ☺ [Kalai (1992)] [Matoušek-Sharir-Welzl (1996)] Are Random-Edge and Random-Facet polynomial ???

Random-Facet algorithm(s) Upper bound on diameter [Kalai-Kleitman (1992)] Random-Facet primal version [Kalai (1992)] Seidel’s randomized LP algorithm [Seidel (1991)] Random-Facet dual version [Matoušek Sharir-Welzl (1992)]

Orient the polytope such that the target vertex is the highest Upper bound on diameter [Kalai-Kleitman (1992)] Active facet a facet containing a vertex (strictly) higher than the current vertex Bound the monotone diameter

Upper bound on diameter [Kalai-Kleitman (1992)]

Exercise: Verify and complete the details. Upper bound on diameter [Kalai-Kleitman (1992)]

“Wishful” Random Facet  Find a vertex of a random active facet. (How?)  Solve recursively within that facet.  If top not reached, step out of the facet and recurse.

Primal Random-Facet [Kalai (1992)]  Choose a random facet containing the current vertex.  Solve recursively within that facet.  If top not reached, do a pivoting step out of the facet and recurse.

(Toy) Exercise: Primal Random-Facet in 2D

How do we solve such a recurrence? One option: generating functions [MSW (1996)] Once a right bound is known, it is not too hard to verify it by induction Seidel’s challenge: Can it be taught?

Solving the recurrence

Exercise: Verify and justify (or find a bug…)

The RANDOM FACET algorithm Analysis

Primal Random-Facet Non-recursive version  Choose a random permutation of the facets f 1,f 2,…,f d containing the current vertex v.  Find the first facet f i that is beneficial to leave and move to a new vertex v’ contained in a new facet f’ i.  Choose a new random ordering of f 1,f 2,…,f i-1,f’ i. Keep the ordering of f i+1,…,f d. Repeat. Exercise: Really?

(Randomized) Bland’s rule  Choose a (random) permutation of all facets f 1,f 2,…,f n.  Find the first facet f i containing the current vertex v that is beneficial to leave and move to a new vertex v’. Open problem: Subexponential ???

Linear Programming Duality d variables n inequalities n variables n inequalities d equalities n-d free variables n inequalities

Linear Programming Duality d variables n inequalities n-d free variables n inequalities

Clarkson’s Algorithm [Clarkson (1988)] No need to consider the case n>9d 2

Seidel’s algorithm [Seidel (1991)]  Choose a random inequality and ignore it.  Solve recursively without this inequality.  If the vertex obtained satisfies the ignored inequality, we are done.  Otherwise, replace the ignored inequality by an equality and solve recursively.

Dual Simplex Algorithms

Dual Pivot Step Exercise: What is the complexity of a dual pivoting step?

Dual Random Facet [Matoušek-Sharir-Welzl (1992)]  Choose a random facet not containing the current dual vertex.  Solve recursively without this facet, starting from the current dual vertex.  If the vertex obtained is on the right side of the ignored facet, i.e., is a vertex, we are done.  Otherwise, do a dual pivoting step to a dual vertex on the ignored facet and recurse.

Hidden dimension Dual Random Facet - Analysis [Matoušek-Sharir-Welzl (1992)]

Primal vs. Dual Exercise: Coincidence?

“It is remarkable to see how different paths have led to rather similar results so close in time.” [Kalai (1992)]

Abstract objective functions (AOFs) Every face should have a unique sink Acyclic Unique Sink Orientations (AUSOs)

AUSOs of n-cubes The diameter is exactly n Stickney, Watson (1978) Morris (2001) Szabó, Welzl (2001) Gärtner (2002) USOs and AUSOs No diameter issue! 2n facets 2 n vertices

Klee-Minty cubes (1972) Taken from a paper by Gärtner-Henk-Ziegler

Random-Facet-1 “[Kalai (1992)]”  Try to find vertices on cn active facets.  If top not reached, choose a random facet, go back to the vertex found on it, and solve recursively within it.  If top not reached, do a pivoting step out of the facet.  Try again to find vertices on cn active facets.  If top not reached, try again, from any starting vertex.

Analysis of Random-Facet-1 “[Kalai (1992)]”

RANDOM FACET on the hypercube [Ludwig (1995)] [Gärtner (2002)] All correct ! Would never be switched ! There is a hidden order of the indices under which the sink returned by the first recursive call correctly fixes the first i bits