IEOR 4004 Final Review part II. April 30, 2014

Integer Programming Standard problems max cx Ax = b x ≥ 0 x integer pure IP = all variables integer mixed IP = some variables integer Standard problems Knapsack Fixed-charge Problem (Facility location) Cutting Stock Problem Set cover Travelling Salesman Problem Job/Machine Scheduling Problems

IP formulations fixed charge (penalty) for a failing a constraint alternative constraints (logical or) if-then constraints piece-wise linear objective constraint constraint add penalty for failing the constraint f ≤ 0 f ≤ My +py y ∈ {0,1} to the objective f ≤ 0 f ≤ My y ∈ {0,1} g ≤ 0 g ≤ M(1-y) f ≤ My if f > 0 , then g ≤ 0 y ∈ {0,1} g ≤ M(1-y)

optimal solution to the LP relaxation gives an upper bound Branch-and-Bound max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13 x1,x2,x3 ≥ 0 and integer max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13 x1,x2,x3 ≥ 0 drop the integrality Linear Programming (LP) relaxation Integer Program solve the LP relaxation optimal solution to the LP relaxation gives an upper bound branch on fractional variables (if exist) how do we (re)-solve the subproblems quickly ? Upper-bounded Dual Simplex special subproblem (Knapsack, Transportation Problem) fractional solution = solution to the LP relaxation

Upper-bounded Dual Simplex Dually feasible solution Optimal ? 1. check upper bounds 2. check lower bounds 1. substitute xi = B – xi’ where 0 ≤ xi ≤ B 2. pivot (dual ratio test) xi out, xj in the basis given a dually feasible basic solution (all coeffs in z negative) check if all basic variables satisfy their upper bounds if some xi fails, if xi should be ≤ B but is > B, then substitute xi = B – xi’ check if all basic variables satisfy their lower bounds if some xi < 0, then use dual ratio test to find xj such that the ratio b/a is smallest possible where: a is the coefficient of xj in the equation for xi (-b) is the coefficient of xj in the equation for z a is positive pivot xi out and xj into the basis x1 = -2 + 2x2 – 4x3 + 3x4 x5 = 1 + x2 + 2x3 – 2x4 z = 5 – 3x2 – 5x3 – 4x4 x1 leaves, ratio test: x2: 3 / 2 = 1.5 x4: 4 / 3 = 1.333 x4 enters

Branching tree optimal fractional solution max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤ 20 x1,x2 ≥ 0 & integer optimal fractional solution not an integer solution branch LP relaxation x1 = 1 x2 = 2.4 z = 18 x2 ≤ 2.4 = 2 x2 ≥ 2.4 = 3 x2 ≤ 2 x2 ≥ 3 x1 = 1.25 x2 = 2 z = 17.5 x1 = 0.25 x2 = 3 z = 16.5 optimal solution x1 ≤ 1 x1 ≥ 2 x1 ≤ 0 x1 ≥ 1 x1 = 1 x2 = 2 z = 16 x1 = 2 x2 = 0.8 z = 16 x1 = 0 x2 = 3.2 z = 16 infeasible candidate solution don’t need to branch (no solution here is better than z=16 and we already know a candidate solution of value 16)

not an integer solution Example Initial solution x1 = 2.6 x2 = 0 x3 = 0 z = 31.2 max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 + x4 = 13 x1,x2,x3,x4 ≥ 0 max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13 x1,x2,x3 ≥ 0 x1 ≤ 2 x1 ≥ 3 x1 = 2 x2 = 1.5 x3 = 0 z = 30 infeasible x1 = 2.6 – 0.4x2 – 0.6x3 – 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x2 ≤ 1 x2 ≥ 2 Subproblem: x1 ≥ 3 dictionary is not feasible anymore x1 = 2 x2 = 1 x3 = 1/3 z = 29⅔ x1 = 1.8 x2 = 2 x3 = 0 z = 29.6 Repeat How to quickly recover an optimum (without starting over) ? Upper-bounded Dual Simplex method x1 ≥ 3 substitute x1 = 3 + x5 x1 ≤ 2 substitute x1 = 2 – x1’ check upper bounds: OK check lower bounds: -0.4 = x5 ≥ 0 x5 leaves, ratio test: x2: no constraint x3: no constraint x4: no constraint check upper bounds: OK check lower bounds: -0.6 = x1’ ≥ 0 x1’ leaves, ratio test: x2: 0.8/0.4 = 2 x3: 2.2/0.6 = 3⅔ x4: 2.4/0.2 = 12 x3 ≥ 1 optimal solution recovered x3 = 0 x1 ≤ 1 x1 = 2 x1’ = 0 x1 = 2–x1’ = 2 x2 = 1.5 x3 = 0 z = 30 x2 = 1.5 + 2.5x1’ – 1.5x3 – 0.5x4 z = 30 – 2x1’ – x3 – 2x4 x1’ = -0.6 + 0.4x2 + 0.6x3 + 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x5 = -0.4 – 0.4x2 – 0.6x3 – 0.2x4 z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4 x1 = 2 x2 = 1 x3 = 0 z = 28 x1 = 2 x2 = 0 x3 = 1 z = 29 x1 = 1 x2 = 4 x3 = 0 z = 28 infeasible not an integer solution branch on x2 ≤ 1 and x2 ≥ 2 infeasible LP x2 enters candidate solution optimal candidate solution

Knapsack 2.4 2 1⅔ max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13 value/ weight 2.4 2 1⅔ Knapsack Initial solution max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13 x1,x2,x3 ≥ 0 and integer x1 = 2.6 x2 = 0 x3 = 0 z = 31.2 optimal greedy strategy (for LP relaxation) pick items by value/weight, highest first take as many items of the same type as possible x1 ≤ 2 x1 ≥ 3 x1 = 2 x2 = 1.5 x3 = 0 z = 30 infeasible x2 ≥ 2 substitute x2 = 2 + x5 x1 ≥ 3 substitute x1 = 3 + x4 x2 ≤ 1 pick 2 units of item #1 then 1 unit of item #2 and then 1/3 of item #3 x2 ≤ 1 x2 ≥ 2 max 12x1+ 4x5 + 5x3 + 8 s.t. 5x1+ 2x5 + 3x3 ≤ 9 max 12x4+ 4x2 + 5x3 + 15 s.t. 5x4+ 2x2 + 3x3 ≤ -2 x3 ≥ 1 substitute x3 = 1 + x7 x1 = 2 x2 = 1 x3 = 1/3 z = 29⅔ x1 = 1.8 x2 = 2 x3 = 0 z = 29.6 max 12x1+ 4x2 + 5x7 + 5 s.t. 5x1+ 2x2 + 3x7 ≤ 10 x3 = 0 do not pick item #3 (1 unit unused in the bag) pick 1.8 units of item #1 x1 ≤ 2 pick 2 units of item #1 and then 1.5 of item #2 (as much as possible) pick 2 units of item #1 x1 = 2 substitute x1 = 2 x1 ≤ 1 pick 1 units of item #1 then 2 of item #5 (i.e. 4 units of item #2) x3 = 0 x3 ≥ 1 x1 ≤ 1 x1 = 2 max 4x5 + 5x3 + 32 s.t. 2x5 + 3x3 ≤ -1 x1 = 2 x2 = 1 x3 = 0 z = 28 x1 = 2 x2 = 0 x3 = 1 z = 29 x1 = 1 x2 = 4 x3 = 0 z = 28 infeasible candidate solution optimal candidate solution

Cutting Planes only for pure IPs solve LP relaxation optimal fractional solution max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤ 20 x1,x2 ≥ 0 & integer 1 2 3 4 5 x1 + x2 ≤ 3 x1,x2 ≥ 0 & integer new fractional optimum only for pure IPs solve LP relaxation if not an integer optimum, make the feasible region smaller by cutting out fractional points (without removing any integer points) how to generate cuts ? Gomory cuts

Gomory cuts max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤ 20 optimal solution max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤ 20 x1 + x2 ≤ 3 x1,x2 ≥ 0 & integer max 6x1 + 5x2 s.t. 4x1 + 5x2 + x3 = 16 8x1 + 5x2 + x4 = 20 x1 + x2 +x5 = 3 x1,x2,x3,x4,x5 ≥ 0 & integer 1 2 3 4 5 optimal fractional solution x1 = 1 – x6 + 2x5 x2 = 2 + x6 – 3x5 x3 = 2 – x6 + 7x5 x4 = 2 + 3x6 – x5 z = 16 – x6 – 3x5 x1 = 1⅔ – ⅓x4 + 1⅔x5 x2 = 1⅓ + ⅓x4 – 2⅔x5 x3 = 2⅔ – ⅓x4 + 6⅔x5 z = 16⅔ – ⅓x4 – 3⅓x5 split to whole and fractional part x6 = – ⅔ + ⅓x4 + ⅓x5 z = 16⅔ – ⅓x4 – 3⅓x5 x1 = 1⅔ – ⅓x4 + 1⅔x5 1+⅔ 0+⅓ 2–⅓ negative coeffients x1 – 2x5 – 1 = ⅔ – ⅓x4 – ⅓x5 ≤ 0 < 1 Use Dual Simplex to recover the optimum cut whole part fractional part positive constant

Goal: – value of the formula for a target state (in the last stage) – save on calculation by storing the intermediate results fi(j) Dynamic Programming optimal substructure property time (months, years) budget (available resources) Break down the problem into stages each stage has possible states next stage depends only on the previous stage identify possible decisions/transitions at each stage find a formula to evaluate (by recursion) Evaluate stage by stage fi ( j ) = optimum value for the first i stages that ends in state j of stage i 1 1 1 2 2 2 e.g. cost of cheapest production plan for first i months where we end up with j items in inventory at the end of month i 3 3 3 (inventory in month i+1 only depends on production and inventory level in month i) stage 1 stage 2 stage 3

Dynamic Programming #1 #2 #3 6 1 2 3 4 5 #1 #2 #3 cash 1 2 3 4 5 6 start #1 9 16 23 30 37 44 #2 10 19 26 33 40 47 #3 49 #1. x1 > 0 thousand $ yields c1(x1)=7x1+2 thousand $ Investments: #2. x2 > 0 thousand $ yields c2(x2)=3x2+7 thousand $ #3. x3 > 0 thousand $ yields c3(x3)=4x3+5 thousand $ We can invest up to $6,000. c1(0)=c2(0)=c3(0)=0 green arrow = best choice of k decision in stage i = how much we invest into #i (all we need to know is how much money we have available for investing) fi( j ) = maximum profit we can get by investing j thousand dollars into first i investments f0( j ) = 0 profit from #i leftover cash fi( j ) = max { } ci(k) + fi-1( ) j – k k ∈ {0,1,...,j} best possible profit with the leftover cash how much we invest in #i

Traveling Salesman Problem Find a shortest route that goes through all cities and comes back where it starts Map of cities Shortest route

Traveling Salesman Problem Work backwards – which is the last vertex visited before v ? s v f(S,v) = min f(S\{x}, x) + cxv x x ∈ S (x,v) ∈ E initial condition: f(Ø,s) = 0 S\{x} S solution: f(V\{s},s) f(S,v) = length of optimal substructure property a shortest route starting at s ending at v and visiting all nodes in S brute force (trying all routes) is O(n!) time and O(1) space complexity: O(n22n) time and space

Good luck on the final exam.