# The Traveling Salesman Problem in Theory & Practice Lecture 10: The Cutting Plane and Branch-and-Bound Approaches to TSP Optimization 1 April 2014 David.

## Presentation on theme: "The Traveling Salesman Problem in Theory & Practice Lecture 10: The Cutting Plane and Branch-and-Bound Approaches to TSP Optimization 1 April 2014 David."— Presentation transcript:

The Traveling Salesman Problem in Theory & Practice Lecture 10: The Cutting Plane and Branch-and-Bound Approaches to TSP Optimization 1 April 2014 David S. Johnson dstiflerj@gmail.com http://davidsjohnson.net Seeley Mudd 523, Tuesdays and Fridays

Outline 1.Cutting Planes 2.Branch and Bound 3.Student Presentation by Arka Bhattacharya, on An O(log n/loglog n) randomized approximation algorithm for the ATSP, by Asadpour, Goemans, Madry, Gharan, and Saberi, SODA 2010.

Optimization, The Early Days [Applgate, Bixby, Chvátal, & Cook, The Traveling Salesman Problem, 2006] In the 1940s and 50s, even 10-city instances were state-of-the-art. C omputers were much slower than today, if you had access to one at all, and compiler optimization almost non-existent. 9!/2 = 181,440 was just too daunting, so even pruned exhaustive search over permutations seemed infeasible. First breakthrough was [Dantzig, Fulkerson, & Johnson, Solution of a large-scale traveling- salesman problem, Operations Research 2 (1954), 393-410], which solved a 49-city problem, generated by picking one city from each of the then 48 U.S. states, plus Washington DC, and using the roadmap distances between them. (Actually, they deleted 7 East-coast cities, solved the resulting 42-city instance, and, lucky for them, the optimal tour contained a link from Washington DC to Boston, and the shortest path between those two contained all the deleted cities.)

The Cutting Plane Approach Solve the edge-based integer programming formulation for the TSP, as follows: 1.Start by solving a weak linear programming relaxation. 2.While the LP solution is not a tour, a.Identify a valid inequality that holds for all tours but not the current solution (a cutting plane or cut for short). b.Add it to the formulation and re-solve. Dantzig, Ford, and Fulkerson started with Minimize Σ e d e x e, subject to Σ xe x e = 2 for all cities x. where d e is the length of edge e. Note: This can be solved in O(N 3 ) time by b-matching techniques. Dantzig et al. solved it using the Simplex algorithm. By Hand.

The Cutting Plane Approach, Illustrated -- Points in R N(N-1)/2 corresponding to a tour. Hyperplane perpendicular to the vector of edge lengths Optimal Tour

Optimal Tour is a point on the convex hull of all tours. Unfortunately, the LP relaxation of the TSP, especially when we omit the subtour constraints, can be a very poor approximation to the convex hull of tours. Facet To improve it, add more constraints (cuts) Note: Subtour constraints are all facets.

Replace each city by a pair of cities. Digression: Solving the Degree-2 Problem in O(N 3 ) Time Graphical TSP Instance w1w1 w4w4 w2w2 w3w3

Replace each edge by a four edges, one joining each pair of a copy of one endpoint to a copy of the other, all with the original weight. Doesnt work – can get multiple copies of the same original edge. Compute a standard weighted matching. First Idea

Doesnt work – can get multiple copies of the same original edge.

Replace each edge by a gadget like this, with all new edges having weight zero except the middle one, which gets the weight of the original edge. If you dont take the weighted link, then you must take its neighbors. Hence you cannot cover any representatives of the original edges endpoint with this gadget. If you do take the weighted link, then you must take edges covering representatives of both of the original edges endpoints. w1w1 Second Idea

Compute a standard weighted matching. The resulting matching corresponds to a set of edges in our original graph that makes every city have degree 2.

Finding Violated Constraints First Choice: Subtour-elimination (subtour) constraints. Σ e={u,v},uS,v S x e 2 for S a proper subset of the cities. How to find a violated one: Consider the graph consisting of edges e with x e > 0. Determine its connected components (takes linear time). If the whole graph is not connected, we get a violated constraint for each connected component. This is unfortunately not enough – the graph containing all the non- zero edges can be highly connected and still violate a subtour constraint. Danzig et al. found violated subtour constraints by hand. We can do it in O(N 4 ) time:

Finding Violated Subtour Constraints Let G be a graph with the set C of all cities as vertices, and an edge joining cities a and b if and only if x {a,b} > 0, that edge having weight equal to x {a,b}. Pick a city c. If there is a violated subtour constraint, c will be on one side of it and some other city c will be on the other. Thus there must be some a set S of cities containing c but not c, such that the total weight of edges linking S and C-S less than 2. But by the maxflow-mincut theorem, this means that the maximum flow from c to c, where an edges capacity equals its weight, will be less than 2. Such a maxflow problem can be solved in time O(N 3 ). To find a violated subtour constraint (if any exists) we need only solve such problems for all pairs (c,c), c S-{c}, for a total of O(N 4 ) time.

Digression: Computing the HK bound in Polynomial Time (by Linear Programming) Input data: – Number N of cities – Edge lengths d {a,b} for each pair of cities. – Selected city c 0. Variables: – x {a,b} for each pair {a,b} of cities (from the LP relaxation of the TSP). – f (a,b,c) for each ordered pair (a,b) of distinct cities and each city c c 0. Goal: Minimize Σ {a,b} d {a,b} x {a,b} Subject to the following constraints.

Constraints of the HK Linear Program Variable range constraints: 0 z 1, for all variables z. Capacity constraints: f (a,b,c) x {a,b}, for all triples (a,b,c). Flow conservation constraints: Σ b f (b,a,c) = Σ b f (a,b,c), for all c c 0 and all a {c 0,c}. MinCut constraints: Σ b f (b,c,c) 2, for all c c 0. O(N 3 ) total variables and constraints

After reaching a solution that satisfied all degree-2 and subtour constraints, there were still fractional variable values. Solid edges have x e = 1, dashed edges have x e = ½. Back to [DFJ54]: More Cutting Planes Needed Picture from [ABCC06]

Another Facet Class: Comb Inequalities H T1T1 T2T2 T3T3 T t-1 TtTt Teeth T i are disjoint, t 3 is odd, all teeth contain at least one city in H and one not in H.

H T1T1 T2T2 T3T3 T t-1 TtTt For Y the handle or a tooth, let δ x (Y) be the total value of the edge variables x e for edges with one endpoint in Y and one outside. By the subtour inequalities, we must have δ x (Y) 2 for each such Y. δ x (Y) also must be even, which is exploited to prove the following comb inequality, which holds whenever x corresponds to a tour: δ x (H) + δ x (T i ) 3t+1 t i=1

H T1T1 T2T2 T3T3 T t-1 TtTt Theorem: If x is a 0-1 vector of length n(n-1)/2 corresponding to a tour (with x[i] = 1 if and only if edge e i is in the tour) and the sets H and T i are as described, then Proof: If δ x (T i ) = 2, then the tour corresponding to x must contain a Hamilton path through the cities in T i. That path must have at least one edge between a city in H and one not in H, which will be counted in both δ x (T i ) and δ x (H). If δ x (T i ) > 2, then we must have δ x (T i ) 4. Suppose that the number of teeth with the latter property is k. Then our sum is at least 3t + k, which is at least 3t + 1 if k > 0. If k = 0, then δ x (H) t. Since t is odd, this means δ x (H) t+1. In both cases we have that our overall sum is at least 3t+1. QED δ x (H) + δ x (T i ) 3t+1. t i=1

Back to [DFJ54]: Picture from [ABCC06] H T1T1 T3T3 T2T2 Are there any violated combs? δ x (T 1 ) = 1 + ½ + ½ = 2 δ x (T 2 ) = ½ + ½ + ½ + ½ = 2 δ x (T 3 ) = 1 + ½ + ½ = 2 δ x (H) = 1 + 1 + 1 = 3 δ x (H) + δ x (T i ) = 9 < 3t+1 = 10. t i=1

New LP Solution Are there any more violated combs? Picture from [ABCC06] H T2T2 T1T1 T3T3 δ x (T 1 ) = 1 + ½ + ½ = 2 δ x (T 2 ) = ½ + ½ + ½ + ½ = 2 δ x (T 3 ) = ½ + ½ + ½ + ½ = 2 δ x (H) = 1 + 1 + ½ + ½ = 3 With this extra constraint, the LP yields an optimal tour. δ x (H) + δ x (T i ) = 9 < 3t+1 = 10. t i=1

Finding Violated Combs No general polynomial-time algorithm known. Not known to be NP-hard either. Many fast heuristics (See [ABCC06] for details and references). Some polynomial-time solvable special cases: Combs in which all teeth contain exactly two cities. – These are called blossom inequalities. – Takes time O(N 3 ). – See [Padberg & Rao, Odd minimum cuts and b-matchings, Math. of O.R. 7 (1982), 67-80]. Combs with t teeth for fixed t. – See [R. Carr, Separating clique trees and bipartition inequalities having a fixed number of handles and teeth in polynomial time, Math. of O.R. 22 (1997), 257- 265]. – Takes time O(N 2t ) which is not practical even for combs with 3 teeth (and even this assumes that your value of x already satisfies all the subtour constraints).

Generalizations of Combs I: The Clique Tree A non-empty collection of pairwise-disjoint handles H 1, H 2, …, H h. A set of at least three pairwise-disjoint teeth T 1, T 2, …, T t. These sets satisfying: – Each tooth contains at least on city that is not in any handle. – Each handle intersects an odd number of teeth, with that number being at least 3. – The graph G, with a vertex for each handle and each tooth, and an edge between to vertices if the corresponding sets intersect, is a tree. h δ x (H i ) + δ x (T j ) 2h + 3t - 1 i=1j=1 h t A violated one can be found in polynomial time for fixed h and t [Carr,1997]. Clique Tree Inequality: These inequalities all correspond to TSP facets [Gr ö tschel & Pullyblank, 1986].

Generalizations of Combs II: Stars and Paths Start with a set of common-tooth combs {H i,T 1,T 2,…,T k }, 1 i k, where – H i H i+1, 1 i < k, and – H i+1 - H i - T j = ϕ. Call a sequence {p, p+1, …, p+k} {1, 2, …, h} a handle interval for tooth T j if it is a maximal such sequence with the following property: H p T j = H p+1 T j = … = H p+k T j h H3H3 H2H2 H1H1 T1T1 T2T2 T3T3 T4T4 T5T5 {1}, {2,3} for T 1 {1,2,3} for T 5 {1}, {2,3} for T 4 {1}, {2}, {3} for T 3 {1,2}, {3} for T 2 If each city in a tooth can only be used in one comb, we will need two copies of the teeth T 1, T 2, and T 4, and three copies of T 5.

Stars and Paths, Continued Now suppose we have α i copies of handle H i and β j copies of tooth T j. We get Star Inequality if, for each tooth T j and each handle interval I for T j, α i β j. We have a Path Inequality if, for each tooth T j and each handle interval I for T j, α i = β j. In both cases, for valid tours, we have Path Inequalities are facet-inducing [Naddef & Pochet, 2001]. h i I α i δ x (H i ) + β j δ x (T j ) (t+1) α i + 2 β j i=1 j=1 ht h i=1 t j=1

More Cutting Planes Domino Parity Cuts [Letchford, 2000] Bipartition Cuts [Carr, 1997]. Local Cuts [ABCC, 2006]. … But the number of facets of the TSP polytope grows exponentially in N, so the cutting plane approach, by itself, runs out of gas quite quickly. Another idea is needed.

Assume we have an algorithm A LB that computes a lower bound on the TSP length when certain edges are fixed (forced to be in the tour, or forced not to be in the tour), and which, for some subproblems, may produce a tour as well. Start with an initial heuristic-created champion tour T UB, an upper bound UB = length(T UB ) on the optimal tour length, and a single live subproblem in which no edge is fixed. While there is a live subproblem, pick one, say subproblem P, and apply algorithm A LB to it. – If LB < UB 1.Pick an edge e that is unfixed in P and create two new subproblems as its children, one with e forced to be in the tour, and one with e forced not to be in the tour. 2.If algorithm A LB produced a tour T, and length(T) < UB 1.Set UB = length(T) and T UB = T. 2.Delete all subproblems with current LB UB, as well as their children and their ancestors that no longer have any live children. – Otherwise (LB UB), delete subproblem P and all its ancestors that no longer have live children. Halt. Our current champion is an optimal tour. Branch & Bound for the TSP

Initial LP, UB = 100, LB = 90 LB = 92LB = 93 X {a,b} = 0X {a,b} = 1 LB = 92LB = 100LB = 98LB = 97 X {a,c} = 0X {c,d} = 0X {c,d} = 1X {a,c} = 1 LB = 101LB = 100 X {e,a} = 0 X {e,a} = 1 New Opt = 97 UB = 97 Opt = 97

First Effective Implementation A branch-and-bound scheme of Held & Karp that used their iterative one-tree-based approach to approximating the LP relaxation of the TSP (d escribed last week) for computing lower bounds. [Held & Karp, The traveling-salesman problem and minimum spanning trees: Part II, Math. Programming 1 (1971), 6-25]. Beat Dantzig-Fulkerson-Johnsons record of 49 cities, set 17 years earlier, by solving two 64-city instances (one random Euclidean instance, and one that merged the 42-city reduced instance of Dantzig et al. with 22 random Euclidean cities). Unlike Dantzig et al., who solved their instance by hand, this one was solved by computer (programmed Linda Ibrahim at UC Berkeley). Subsequent work along this line pushed the record up to 67. [Camerini, Fratta, & Maffioli, On improving relaxation methods by modified gradient techniques, Math. Programming Study 3 (1975), 26-34]. But again we seem to be hitting limits and another idea is needed.

Branch & Cut Combine Branch-and-bound with Cutting Planes. More details next time.

Download ppt "The Traveling Salesman Problem in Theory & Practice Lecture 10: The Cutting Plane and Branch-and-Bound Approaches to TSP Optimization 1 April 2014 David."

Similar presentations