Lagrangean Relaxation

Overview Decomposition based approach. Start with Easy constraints Complicating Constraints. Put the complicating constraints into the objective and delete them from the constraints. We will obtain a lower bound on the optimal solution for minimization problems. In many situations, this bound is close to the optimal solution value.

An Example: Constrained Shortest Paths Given: a network G = (N,A) cij cost for arc (i,j) tij traversal time for arc (i,j) z* = Min s. t. Complicating constraint

Example Find the shortest path from node 1 to node 6 with a transit time at most 10 $cij, tij i j $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6

Shortest Paths with Transit Time Restrictions Shortest path problems are easy. Shortest path problems with transit time restrictions are NP-hard. We say that constrained optimization problem Y is a relaxation of problem X if Y is obtained from X by eliminating one or more constraints. We will “relax” the complicating constraint, and then use a “heuristic” of penalizing too much transit time. We will then connect it to the theory of Lagrangian relaxations.

Shortest Paths with Transit Time Restrictions Step 1. (A Lagrangian relaxation approach). Penalize violation of the constraint in the objective function. z(λ) = Min Complicating constraint Note: z*(λ) ≤ z* ∀λ ≥ 0

Shortest Paths with Transit Time Restrictions Step 2. Delete the complicating constraint(s) from the problem. The resulting problem is called the Lagrangian relaxation. Complicating constraint L(λ) = Min Note: L(λ) ≤ z(λ) ≤ z* ∀λ ≥ 0

What is the effect of varying λ? $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6 Case 1: λ = 0 1 2 4 5 3 6 10 12 cij + λ tij j i P = P = 1-2-4-6 c(P) = c(P) = 3 t(P) = t(P) = 18

Question to class If λ = 0, the min cost path is found. What happens to the (real) cost of the path as λ increases from 0? What path is determined as λ gets VERY large? What happens to the (real) transit time of the path as λ increases from 0? cij + λ tij j i

Let λ = 1 Case 2: λ = 1 2 4 1 6 5 3 P = P = 1-2-5-6 c(P) = 5 c(P) = $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6 Case 2: λ = 1 2 2 4 8 11 5 1 11 6 3 12 13 4 5 3 15 P = P = 1-2-5-6 c(P) = 5 c(P) = t(P) = t(P) = 15

Let λ = 2 Case 3: λ = 2 2 4 1 6 5 3 P = P = 1-2-5-6 c(P) = 5 c(P) = $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6 Case 3: λ = 2 3 2 4 15 21 8 1 12 6 5 19 16 6 5 3 18 P = P = 1-2-5-6 c(P) = 5 c(P) = t(P) = t(P) = 15

And alternative shortest path when λ = 2 $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6 3 2 4 15 21 8 1 12 6 5 19 16 6 5 3 18 P = P = 1-3-2-5-6 c(P) = c(P) = 15 t(P) = t(P) = 10

Let λ = 5 Case 4: λ = 5 2 4 1 6 5 3 P = P = 1-3-2-4-5-6 c(P) = 24 $1,10 $1,1 $1,7 $2,3 $10,3 $12,3 $2,2 $1,2 $10,1 $5,7 1 2 4 5 3 6 Case 4: λ = 5 3 2 4 15 21 8 1 12 6 5 19 16 6 5 3 18 P = P = 1-3-2-4-5-6 c(P) = 24 c(P) = t(P) = t(P) = 8

A parametric analysis Toll modified cost Cost Transit Time Modified cost -10λ 0 ≤ λ ≤ ⅔ 3 + 18λ 3 18 3 + 8λ ⅔ ≤ λ ≤ 2 5 + 15λ 5 15 5 + 3λ 2 ≤ λ ≤ 4.5 15 + 10λ 10 4.5 ≤ λ < ∞ 24 + 8λ 24 8 24 - 2λ A lower bound on z* The best value of λ is the one that maximizes the lower bound.

Costs Modified Cost – 10λ Transit Times λ λ

Problem splitting tricks

Embedded Network Structure Application Embedded Network Structure Networks with side constraints minimum cost flows shortest paths Traveling Salesman Problem assignment problem minimum cost spanning tree Vehicle routing assignment problem variant of min cost spanning tree Network design shortest paths Two-duty operator scheduling shortest paths minimum cost flows Multi-time production planning shortest paths / DPs minimum cost flows

Problem splitting tricks

Lagrangian Relaxation What can this be used for ? Primary usage: Bounding ! Because it is a relaxation, the optimal value will bound the optimal value of the real problem ! Lagrangian heuristics, i.e. generate a “good” solution based on a solution to the relaxed problem. Problem reduction, i.e. reduce the original problem based on the solution to the relaxed problem.

Two Problems Facing a problem we need to decide: Which constraints to relax (strategic choice) How to find the best lagrangean multipliers, (tactical choice)

Which constraints to relax Which constraints to relax depends on two things: Computational effort: Number of Lagrangian multipliers Hardness of relaxed problem to solve Integrality of relaxed problem: If it is integral, we can only do as good as the straightforward LP relaxation ! (Integrality: the solution to relaxed problem is guaranteed to be integral.)

Multiplier adjustment Two different types are given: Subgradient optimisation Constraint generation method Of these, subgradient optimisation is the method of choice. This is general method which nearly always works ! Here we will only consider this Method, although more efficient (but much more complicated) adjustment methods have been suggested.

Linear Program Min: s.t.

Lagrangian Relaxation

Problem reformulation Note: Each constraint is associated with a multiplier i.

The subgradient (subgradient of multiplier i)

Subgradient Optimization

Subgradient Optimization In the subgradient search; Try to update UB in every iteration Halve the step multiplier,Π, if there is no improvement in LB in the last certain number of iterations. Stop the alpgorithm either based on the number of iterations or based on the gap between UB and LB.

Example: Set covering

Relaxed Set coverint How can we solve this problem to optimality ???

Optimization Algorithm The answer is so simple that we are reluctant calling it an optimization algorithm: Choose all x’s with negative coefficients ! What does this tell us about the strength of the relaxation ?

Rewritten: Relaxed Set covering where

Lower bound

Try it out yourself Starting with ZUB=6,=2 and i=0 for i=1,2,3, perform 3 iterations of the subgradient optimization method on the previous example problem. Please show the lower bound and multiplier values of each iteration.