1 Linear Programming Jose Rolim University of Geneva

L.P.Jose Rolim2 What is Linear Programming?  Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem.  Method: State the problem and define variables whose values will be determined. Develop a linear programming model:  Write the problem as an optimization formula (a linear expression to be minimized or maximized)  Write a set of linear constraints An available LP solver (computer program) gives the values of variables.

L.P.Jose Rolim3 Types of LP  LP – all variables are real.  ILP – all variables are integers.  MILP – some variables are integers, others are real.

L.P.Jose Rolim4 A single variable problem  Consider variable x  Problem: find the maximum value of x subject to constraint, 0 ≤ x ≤ 15.  Solution: x = 15.

L.P.Jose Rolim5 Single Variable Problem (Cont.)  Consider more complex constraints:  Maximize x, subject to following constraints x ≥ 0(1) 5x ≤ 75(2) 6x ≤ 30(3) x ≤ 10(4) x (1) (2) (3) (4) All constraints satisfied Solution, x = 5

L.P.Jose Rolim6 A Two-Variable Problem  Manufacture of x 1 chairs and x 2 tables:  Maximize profit, P = 45x x 2 dollars  Subject to resource constraints: 400 boards of wood,5x x 2 ≤ 400(1) 450 man-hours of labor,10x x 2 ≤ 450(2) x 1 ≥ 0(3) x 2 ≥ 0 (4)

L.P.Jose Rolim7 Solution: Two-Variable Problem Chairs, x 1 Tables, x 2 (1) (2) (24, 14) Profit increasing decresing P = 2200 P = 0 Best solution: 24 chairs, 14 tables Profit = 45× ×14 = 2200 dollars

L.P.Jose Rolim8 Change Chair Profit, $64/Unit  Manufacture of x 1 chairs and x 2 tables:  Maximize profit, P = 64x x 2 dollars  Subject to resource constraints: 400 boards of wood,5x x 2 ≤ 400(1) 450 man-hours of labor,10x x 2 ≤ 450(2) x 1 ≥ 0(3) x 2 ≥ 0 (4)

L.P.Jose Rolim9 Solution: $64 Profit/Chair Chairs, x 1 Tables, x 2 (1) (2) Profit increasing decresing P = 2880 P = 0 Best solution: 45 chairs, 0 tables Profit = 64× ×0 = 2880 dollars (24, 14)

L.P.Jose Rolim10 Motivation: A Political Problem Goal: Win election by winning majority of votes in each region. Subgoal: Win majority of votes in each region while minimizing advertising cost. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads

L.P.Jose Rolim11 Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural

L.P.Jose Rolim12 General Linear Programs real numbers variables Linear function Linear inequalities Linear constraints

L.P.Jose Rolim13 Overview of Linear Programming Convex feasible region Objective function Objective value

L.P.Jose Rolim14 Standard Form objective function constraints.

L.P.Jose Rolim15 Standard Form (compact) mxn matrix m-dimensional vector n-dimensional vectors Can specify linear program in standard form by (A,b,c).

L.P.Jose Rolim16 Converting to Standard Form

L.P.Jose Rolim17 Converting to Standard Form (continued) Negate coefficients Transforming minimization to maximization

L.P.Jose Rolim18 Converting to Standard Form (continued) If x j has no non-negativity constraint, replace each occurrence of x j with x j ’ – x j ”. Giving each variable a non-negativity constraint New non-negativity constraints

L.P.Jose Rolim19 Converting to Standard Form (continued) Transforming equality constraints to inequality constraints

L.P.Jose Rolim20 Converting to Standard Form (continued). Changing sense of an inequality constraint Rationale:

L.P.Jose Rolim21 Converting Linear Programs into Slack Form for algorithmic ease, transform all constraints except non-negativity ones into equalities for inequality constraint: define slack slack variable instead of s basic variables non-basic variables

L.P.Jose Rolim22 Converting Linear Programs into Slack Form (continued) objective function

L.P.Jose Rolim23 Converting Linear Programs into Slack Form (continued) Compact Form: (N, B, A, b, c, v) set of indices of non-basic variables set of indices of basic variables Slack Form Example Compact Form negative of slack form coefficients

L.P.Jose Rolim24 Shortest Paths. Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not?

L.P.Jose Rolim25 Maximum Flow.

L.P.Jose Rolim26 Minimum Cost Flow.

L.P.Jose Rolim27 Multicommodity Flow. should be s i

L.P.Jose Rolim28 Solving a Linear Program  Simplex algorithm  Geometric interpretation Visit vertices on the boundary of the simplex representing the convex feasible region  Transforms set of inequalities using process similar to Gaussian elimination  Run-time not polynomial in worst-case often very fast in practice  Ellipsoid method  Run-time polynomial slow in practice  Interior-Point methods  Run-time polynomial for large inputs, performance can be competitive with simplex method  Moves through interior of feasible region

L.P.Jose Rolim29 Simplex Algorithm: Example Basic Solution Standard Form Slack Form Basic Solution: set each nonbasic variable to 0. Basic Solution:

L.P.Jose Rolim30 Simplex Algorithm: Example Reformulating the LP Model Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x 1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x 6 of that constraint, swap role with x 1. Rewrite other equations with x 6 on RHS. PIVOT leaving variable entering variable new objective value

L.P.Jose Rolim31 Simplex Algorithm: Example Reformulating the LP Model Next Iteration: select x 3 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

L.P.Jose Rolim32 Simplex Algorithm: Example Reformulating the LP Model. Next Iteration: select x 2 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

L.P.Jose Rolim33 Simplex Algorithm: Pivoting leaving variable entering variable Rewrite the equation that has x l on LHS to have x e on LHS Update remaining equations by substituting RHS of new equation for each occurrence of x e. Do the same for objective function. Update sets of nonbasic, basic variables.

L.P.Jose Rolim34 Simplex Algorithm: Pseudocode source: textbook Cormen et al. to be defined later (detects infeasibility) initial basic solution optimal solution detects unboundedness

L.P.Jose Rolim35 Finding an Initial Solution source: textbook Cormen et al. An LP model whose initial basic solution is not feasible

L.P.Jose Rolim36 Finding an Initial Solution (continued) Auxiliary LP model L aux :

L.P.Jose Rolim37 Finding an Initial Solution (continued).

L.P.Jose Rolim38 Finding an Initial Solution (continued) Original LP model L aux L aux in slack form

L.P.Jose Rolim39 Finding an Initial Solution (continued) PIVOT

L.P.Jose Rolim40 Finding an Initial Solution (continued)

L.P.Jose Rolim41 Linear Programming Duality max becomes min RHS coefficients swap places with objective function coefficients sense changes x variables go away y variables appear

L.P.Jose Rolim42 Duality Example

L.P.Jose Rolim43 Weak Linear Programming Duality Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.

L.P.Jose Rolim44 Weak Linear Programming Duality (continued)

L.P.Jose Rolim45 Finding a Dual Solution Finding a dual solution whose value is equal to that of an optimal primal solution…

L.P.Jose Rolim46 Optimality.