UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2006 Lecture 9 Wednesday, 11/15/06 Linear Programming

Overview ä Motivation & Basics ä Standard & Slack Forms ä Formulating Problems as Linear Programs ä Simplex Algorithm ä Example ä High-Level Algorithm ä Correctness ä Roadmap ä Key Concepts ä Initial Basic Feasible Solution ä Duality ä Literature Case Study

Motivation & Basics

Motivation: A Political Problem source: textbook Cormen et al. Goal: Win election by winning majority of votes in each region. Subgoal: Win majority of votes in each region while minimizing advertising cost. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads

Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural source: textbook Cormen et al.

General Linear Programs real numbers variables Linear function Linear inequalities Linear constraints source: textbook Cormen et al.

Overview of Linear Programming Convex feasible region Objective function Objective value source: textbook Cormen et al.

Standard & Slack Forms

Standard Form objective function constraints source: textbook Cormen et al.

Standard Form (compact) mxn matrix m-dimensional vector n-dimensional vectors source: textbook Cormen et al. Can specify linear program in standard form by (A,b,c).

Converting to Standard Form source: textbook Cormen et al.

Converting to Standard Form (continued) source: textbook Cormen et al. Negate coefficients Transforming minimization to maximization

Converting to Standard Form (continued) source: textbook Cormen et al. If x j has no non-negativity constraint, replace each occurrence of x j with x j ’ – x j ”. Giving each variable a non-negativity constraint New non-negativity constraints

Converting to Standard Form (continued) source: textbook Cormen et al. Transforming equality constraints to inequality constraints

Converting to Standard Form (continued) source: textbook Cormen et al. Changing sense of an inequality constraint Rationale:

Converting Linear Programs into Slack Form source: textbook Cormen et al. for algorithmic ease, transform all constraints except non-negativity ones into equalities for inequality constraint: define slack slack variable instead of s basic variables non-basic variables

Converting Linear Programs into Slack Form (continued) source: textbook Cormen et al. objective function

Converting Linear Programs into Slack Form (continued) source: textbook Cormen et al. Compact Form: (N, B, A, b, c, v) set of indices of non-basic variables set of indices of basic variables Slack Form Example Compact Form negative of slack form coefficients

Formulating Problems as Linear Programs

Shortest Paths source: textbook Cormen et al. Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not?

Maximum Flow source: textbook Cormen et al.

Minimum Cost Flow source: textbook Cormen et al.

Multicommodity Flow source: textbook Cormen et al. should be s i

Simplex Method

Solving a Linear Program source: textbook Cormen et al. ä Simplex algorithm ä Geometric interpretation ä Visit vertices on the boundary of the simplex representing the convex feasible region ä Transforms set of inequalities using process similar to Gaussian elimination ä Run-time ä not polynomial in worst-case ä often very fast in practice ä Ellipsoid method ä Run-time ä polynomial ä slow in practice ä Interior-Point methods ä Run-time ä polynomial ä for large inputs, performance can be competitive with simplex method ä Moves through interior of feasible region

Simplex Algorithm: Example Basic Solution source: textbook Cormen et al. Standard Form Slack Form Basic Solution: set each nonbasic variable to 0. Basic Solution:

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x 1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x 6 of that constraint, swap role with x 1. Rewrite other equations with x 6 on RHS. PIVOT leaving variable entering variable new objective value

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Next Iteration: select x 3 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Next Iteration: select x 2 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

Simplex Algorithm: Pivoting source: textbook Cormen et al. leaving variable entering variable Rewrite the equation that has x l on LHS to have x e on LHS Update remaining equations by substituting RHS of new equation for each occurrence of x e. Do the same for objective function. Update sets of nonbasic, basic variables.

Simplex Algorithm: Pseudocode source: textbook Cormen et al. to be defined later (detects infeasibility) initial basic solution optimal solution detects unboundedness

Correctness: Roadmap (Key Pieces) Theorem 29.13: Fundamental Theorem of Linear Programming For LP model in standard form, either: 1. exists optimal solution with finite objective function value & SIMPLEX returns one, or 2. infeasible & SIMPLEX returns INFEASIBLE, or 3. Unbounded & SIMPLEX returns UNBOUNDED Lemma 29.2: Basic solution feasible -> if SIMPLEX finds solution it is feasible; if reports unbounded, then model is unbounded Lemma 29.3: Algebraic lemma Theorem 29.10: LP duality: SIMPLEX primal result is optimal & dual is optimal Lemma 29.5: Iteration bound, cycling Lemma 29.4: Slack form uniqueness Lemma 29.6: Tie-breaking Lemma 29.8: Weak LP duality Corollary 29.9: Conditions for which feasible solutions for primal, dual programs are optimal Lemma 29.1: Pivot results Lemma 29.12: Infeasibility detection Lemma 29.11: L aux Lemma 29.7: Basic solution feasible -> SIMPLEX either reports unbounded or finds feasible solution in iterations

Proof of Correctness Key Concepts Initial Basic Feasible Solution

Finding an Initial Solution source: textbook Cormen et al. An LP model whose initial basic solution is not feasible

Finding an Initial Solution (continued) source: textbook Cormen et al. Auxiliary LP model L aux :

Finding an Initial Solution (continued) source: textbook Cormen et al.

Finding an Initial Solution (continued) source: textbook Cormen et al. Original LP model L aux L aux in slack form

Finding an Initial Solution (continued) source: textbook Cormen et al. PIVOT

Finding an Initial Solution (continued) source: textbook Cormen et al.

Duality Proof of Correctness Key Concepts

Linear Programming Duality source: textbook Cormen et al. max becomes min RHS coefficients swap places with objective function coefficients sense changes x variables go away y variables appear

Duality Example source: textbook Cormen et al.

Weak Linear Programming Duality source: textbook Cormen et al. Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.

Weak Linear Programming Duality (continued) source: textbook Cormen et al.

Finding a Dual Solution source: textbook Cormen et al. Finding a dual solution whose value is equal to that of an optimal primal solution…

Optimality source: textbook Cormen et al.

Literature Case Study