1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2006 Lecture 9 Wednesday, 11/15/06 Linear Programming

2. Overview ä Motivation & Basics ä Standard & Slack Forms ä Formulating Problems as Linear Programs ä Simplex Algorithm ä Example ä High-Level Algorithm ä Correctness ä Roadmap ä Key Concepts ä Initial Basic Feasible Solution ä Duality ä Literature Case Study

3. Motivation & Basics

4. Motivation: A Political Problem source: 91.503 textbook Cormen et al. Goal: Win election by winning majority of votes in each region. Subgoal: Win majority of votes in each region while minimizing advertising cost. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads

5. Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural source: 91.503 textbook Cormen et al.

6. General Linear Programs real numbers variables Linear function Linear inequalities Linear constraints source: 91.503 textbook Cormen et al.

7. Overview of Linear Programming Convex feasible region Objective function Objective value source: 91.503 textbook Cormen et al.

8. Standard & Slack Forms

9. Standard Form objective function constraints source: 91.503 textbook Cormen et al.

10. Standard Form (compact) mxn matrix m-dimensional vector n-dimensional vectors source: 91.503 textbook Cormen et al. Can specify linear program in standard form by (A,b,c).

11. Converting to Standard Form source: 91.503 textbook Cormen et al.

12. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Negate coefficients Transforming minimization to maximization

13. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. If x j has no non-negativity constraint, replace each occurrence of x j with x j ’ – x j ”. Giving each variable a non-negativity constraint New non-negativity constraints

14. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Transforming equality constraints to inequality constraints

15. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Changing sense of an inequality constraint Rationale:

16. Converting Linear Programs into Slack Form source: 91.503 textbook Cormen et al. for algorithmic ease, transform all constraints except non-negativity ones into equalities for inequality constraint: define slack slack variable instead of s basic variables non-basic variables

17. Converting Linear Programs into Slack Form (continued) source: 91.503 textbook Cormen et al. objective function

18. Converting Linear Programs into Slack Form (continued) source: 91.503 textbook Cormen et al. Compact Form: (N, B, A, b, c, v) set of indices of non-basic variables set of indices of basic variables Slack Form Example Compact Form negative of slack form coefficients

19. Formulating Problems as Linear Programs

20. Shortest Paths source: 91.503 textbook Cormen et al. Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not?

21. Maximum Flow source: 91.503 textbook Cormen et al.

22. Minimum Cost Flow source: 91.503 textbook Cormen et al.

23. Multicommodity Flow source: 91.503 textbook Cormen et al. should be s i

24. Simplex Method

25. Solving a Linear Program source: 91.503 textbook Cormen et al. ä Simplex algorithm ä Geometric interpretation ä Visit vertices on the boundary of the simplex representing the convex feasible region ä Transforms set of inequalities using process similar to Gaussian elimination ä Run-time ä not polynomial in worst-case ä often very fast in practice ä Ellipsoid method ä Run-time ä polynomial ä slow in practice ä Interior-Point methods ä Run-time ä polynomial ä for large inputs, performance can be competitive with simplex method ä Moves through interior of feasible region

26. Simplex Algorithm: Example Basic Solution source: 91.503 textbook Cormen et al. Standard Form Slack Form Basic Solution: set each nonbasic variable to 0. Basic Solution:

27. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x 1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x 6 of that constraint, swap role with x 1. Rewrite other equations with x 6 on RHS. PIVOT leaving variable entering variable new objective value

28. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Next Iteration: select x 3 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

29. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Next Iteration: select x 2 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

30. Simplex Algorithm: Pivoting source: 91.503 textbook Cormen et al. leaving variable entering variable Rewrite the equation that has x l on LHS to have x e on LHS Update remaining equations by substituting RHS of new equation for each occurrence of x e. Do the same for objective function. Update sets of nonbasic, basic variables.

31. Simplex Algorithm: Pseudocode source: 91.503 textbook Cormen et al. to be defined later (detects infeasibility) initial basic solution optimal solution detects unboundedness

32. Correctness: Roadmap (Key Pieces) Theorem 29.13: Fundamental Theorem of Linear Programming For LP model in standard form, either: 1. exists optimal solution with finite objective function value & SIMPLEX returns one, or 2. infeasible & SIMPLEX returns INFEASIBLE, or 3. Unbounded & SIMPLEX returns UNBOUNDED Lemma 29.2: Basic solution feasible -> if SIMPLEX finds solution it is feasible; if reports unbounded, then model is unbounded Lemma 29.3: Algebraic lemma Theorem 29.10: LP duality: SIMPLEX primal result is optimal & dual is optimal Lemma 29.5: Iteration bound, cycling Lemma 29.4: Slack form uniqueness Lemma 29.6: Tie-breaking Lemma 29.8: Weak LP duality Corollary 29.9: Conditions for which feasible solutions for primal, dual programs are optimal Lemma 29.1: Pivot results Lemma 29.12: Infeasibility detection Lemma 29.11: L aux Lemma 29.7: Basic solution feasible -> SIMPLEX either reports unbounded or finds feasible solution in iterations

33. Proof of Correctness Key Concepts Initial Basic Feasible Solution

34. Finding an Initial Solution source: 91.503 textbook Cormen et al. An LP model whose initial basic solution is not feasible

35. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. Auxiliary LP model L aux :

36. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al.

37. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. Original LP model L aux L aux in slack form

38. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. PIVOT

39. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al.

40. Duality Proof of Correctness Key Concepts

41. Linear Programming Duality source: 91.503 textbook Cormen et al. max becomes min RHS coefficients swap places with objective function coefficients sense changes x variables go away y variables appear

42. Duality Example source: 91.503 textbook Cormen et al.

43. Weak Linear Programming Duality source: 91.503 textbook Cormen et al. Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.

44. Weak Linear Programming Duality (continued) source: 91.503 textbook Cormen et al.

45. Finding a Dual Solution source: 91.503 textbook Cormen et al. Finding a dual solution whose value is equal to that of an optimal primal solution…

46. Optimality source: 91.503 textbook Cormen et al.

47. Literature Case Study