 # Approximation Algorithms

## Presentation on theme: "Approximation Algorithms"— Presentation transcript:

Approximation Algorithms

Approximation Algorithms
Outlines Why approximation algorithm? Approximation ratio Approximation vertex cover problem Approximation traveling salesman problem (TSP) Other interesting problems Approximation Algorithms

3 approaches for NP-complete Problems
For small inputs, an exponential algorithm is OK. Try to solve important special cases in polynomial time. Find near-optimal solutions in polynomial time (worst case or on average). An algorithm that returns near-optimal solutions is called an approximation algorithm. Approximation Algorithms

Approximation Algorithms
Approximation Ratio If, for any input of size n for a problem, the cost C of the solution produced by an algorithm is within a factor of ρ(n) of the cost C∗ of an optimal solution (where max(C/C∗ , C∗/C) ≤ ρ(n)), we say that, the algorithm has an approximation ratio of ρ(n) or the algorithm is a ρ(n)-approximation algorithm. Approximation Algorithms

On Approximation Ratio
maximization problem 0 < C ≤ C ∗ C ∗ /C gives the factor by which the cost of an optimal solution is larger than the cost of the approximate solution. minimization problem 0 < C ∗ ≤ C C/C ∗ gives the factor by which the cost of the approximate solution is larger than the cost of an optimal solution. A 1-approximation algorithm produces an optimal solution. large approximation ratio => a solution can be much worse than optimal. Approximation Algorithms

Polynomial-time approximation algorithms
may: have small constant approximation ratios. have approximation ratios that grow as functions of the input size n. achieve increasingly smaller approximation ratios by using more and more computation time. Approximation Algorithms

Approximation Algorithms
Approximation scheme An approximation algorithm that takes as input both an instance of the problem, and a value  > 0 such that for any fixed , the scheme is a (1 + )-approximation algorithm. Approximation Algorithms

Polynomial-time Approximation Scheme
An approximation scheme is a polynomial-time approximation scheme if for any fixed > 0, the scheme runs in time polynomial in the size n of its input instance. a fully polynomial-time approximation scheme if it is an approximation scheme and its running time is polynomial both in 1/ and in the size n of the input instance. Ex: running time of O((1/)2 n3). Approximation Algorithms

Approximate Vertex Cover

Approximation Algorithms
Vertex Cover A vertex cover of an undirected graph G = (V, E) is a subset V’ ⊆V such that if (u, v) is an edge of G, then either u∈V or v∈V. The size of a vertex cover is the number of vertices in it. An optimal vertex cover is a vertex cover of minimum size in an undirected graph. The vertex-cover problem is to find an optimal vertex cover in a given undirected graph. Approximation Algorithms

Example of Vertex Covers
B A B C D E C D E F F Approximation Algorithms

Approximate Vertex Cover
AVC(G) C ←  E ← E[G] while E   do Choose an arbitrary edge (u,v) from E C ← C ∪ {u, v} E=E-{(x, y)| x=u or x=v or y=u or y=v} return C Approximation Algorithms

Find Approximted Vertex Covers (1)
E ← E[G] while E   do Choose an arbitrary edge (u,v) from E C ← C ∪ {u, v} E = E - {(x, y)| x=u or x=v or y=u or y=v} return C A B C D E F Approximation Algorithms

Find Approximted Vertex Covers (2)
E ← set of edges in G while E   do Choose an arbitrary edge (u,v) from E C ← C ∪ {u, v} E = E - {(x, y)| x=u or x=v or y=u or y=v} return C b c d a e f g c d e g f b a Approximation Algorithms

Approximation ratio of AVC (1)
AVC is a polynomial-time 2-approximation algorithm. Proof From the algorithm, AVC runs in O(V+E), which is polynomial time. Let C* be a set of optimal vertex cover. Let C be the set of vertices returned by AVC. C is a vertex cover because AVC loops until all edges in the graph is removed. Next, we will show that |C|/|C*| ≤ 2. Approximation Algorithms

Approximation ratio of AVC (2)
Let A be the set of all edges chosen in the loop. Since all edges incident on the chosen nodes in C are removed, no two edges in A share an endpoint. An edge is chosen in the loop when neither of its endpoints are already in C. Thus, |C|=2|A|. Approximation Algorithms

Approximation ratio of AVC (3)
For any vertex v in C*, there is at least one edge in A with v as an endpoint (because C* is a cover). And, no two edges in A share an endpoint. Thus, |A| ≤ |C*|. Thus, |C|=2|A|≤ 2|C*|. That is, |C|/|C*| ≤ 2. Thus, AVC has approximation ratio of 2. Approximation Algorithms

Traveling Salesman Problem

Traveling salesman problem (TSP)
Given a complete undirected graph G=(V, E) that has a nonnegative integer cost c(u, v) associated with each edge (u, v)∈ E, find a hamiltonian cycle (a tour) of G with minimum cost. C(A) =  c(u, v) (u,v)∈A TSP is an NP-complete problem. Approximation Algorithms

TSP with triangle inequality
The cost function c satisfies the triangle inequality if for all vertices u, v,w ∈ V, c(u, w) ≤ c(u, v) + c(v, w) . Example of cost functions that satisfy the triangle inequality Euclidean distance in the plane. Even with the triangle inequality, TSP is NP-complete. Approximation Algorithms

Approximation TSP w triangle inequality
ATSP(G, c) Select a vertex r∈V [G] to be a “root” vertex Compute a minimum spanning tree T for G from root r using MST-PRIM(G, c, r). Let L be the list of vertices visited in a preorder tree walk of T. Return the hamiltonian cycle H that visits the vertices in the order L. Approximation Algorithms

Example: Approximate TSP (1)
From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Graph whose cost is the euclidean distance Minimum spanning tree with root a. Approximation Algorithms

Example: Approximate TSP (2)
From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Preorder traversal and preorder walk Tour obtained from preorder walk Approximation Algorithms

Example: Approximate TSP (3)
From: Cormen, Rivest, Leiserson and Stein, Introduction to Algorithms, MIT Press, 2001. Approximate tour Shortest tour Approximation Algorithms

ATSP is 2-approximation algorithm (1)
APPROX-TSP-TOUR is a 2-approximation algorithm for the traveling-salesman problem with the triangle inequality. Proof Let H* denote an optimal tour for the given set of vertices. If an edge is removed from H*, the result is a spanning tree, say Ts. Thus, c(Ts) ≤ c(H*). Let T be a minimum spanning tree. Then, c(T) ≤ c(Ts). Thus, c(T) ≤ c(H*). Approximation Algorithms

ATSP is 2-approximation algorithm (2)
Let W be a full walk of T. Then, W lists the vertices when they are first visited and also whenever they are returned to after a visit to a subtree. Since the full walk traverses every edge of T exactly twice, c(W) = 2c(T ). From c(T) ≤ c(H*) and c(W) = 2c(T ), we get c(W) ≤ 2c(H*). Approximation Algorithms

ATSP is 2-approximation algorithm (3)
However, W is not a tour, since it visits some vertices more than once. By the triangle inequality, we can delete a visit to any vertex from W and the cost does not increase. (If a vertex v is deleted from W between visits to u and w, the resulting ordering specifies going directly from u to w.) By repeatedly applying this operation, we can remove from W all but the first visit to each vertex. Approximation Algorithms

ATSP is 2-approximation algorithm (4)
Let H be the cycle corresponding to this preorder walk. H is a hamiltonian cycle, because every vertex is visited exactly once. Since H is obtained by deleting vertices from the full walk W, c(H) ≤ c(W). c(H) ≤ 2c(H*) since c(H) ≤ c(W) & c(W) ≤ 2c(H*). That is, APPROX-TSP-TOUR is a 2-approximation algorithm. Approximation Algorithms

MAX-3-CNF satisfiability
A randomized approximation algorithm

MAX-3CNF-Satisfiability
Given a 3-CNF expression , satisfiability, find an assignment of the variables that maximizes the number of clauses evaluating to 1. According to the definition of 3-CNF satisfiability, we require each clause to consist of exactly three distinct literals. We further assume that no clause contains both a variable and its negation. We now show that randomly setting each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (1)
Given an instance of MAX-3-CNF satisfiability with m clauses and variables x1, x2, , xn, the randomized algorithm that independently sets each variable to 0/1 with prob. 1/2 is a randomized 8/7-approximation algorithm. Proof Suppose that we have independently set each variable to 1 with probability 1/2 and to 0 with probability 1/2. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (2)
For i = 1, 2, , n, we define the indicator random variable Yi = I {clause i is satisfied} , so that Yi = 1 as long as at least one of the literals in the ith clause has been set to 1. Since no literal appears more than once in the same clause, and since we have assumed that no variable and its negation appear in the same clause, the settings of the three literals in each clause are independent. Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (3)
A clause is not satisfied only if all three of its literals are set to 0, and so Pr {clause i is not satisfied} = (1/ 2)3 = 1/ 8. Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8. Therefore, E [Yi ] = 7/ 8. Let Y be the number of satisfied clauses overall, so that Y = Y1 + Y2 +· · · +Ym. Then, we have m E [Y ] = E [ ∑ Yi ] i=1 Approximation Algorithms

Randomized MAX-3-CNF-SAT is 8/7-approximate (4)
E [Y ] = E [ ∑ Yi ] m i =1 = ∑ E [Yi ] (by linearity of expectation) i=1m = ∑ 7/8 i=1 = 7m/8 . Since m is an upper bound on the number of satisfied clauses, the approximation ratio is at most m/(7m/ 8) = 8/ 7. Approximation Algorithms

Approximating weighted vertex cover
Using linear programming

Minimum-weight vertex-cover problem
For any vertex cover V′ ⊆ V, we define the weight of the vertex cover w(V′) = ∑ w(v). vV ′ Given an undirected graph G = (V, E) in which each vertex vV has an associated positive weight w(v), find a vertex cover of minimum weight. We cannot apply the algorithm used for unweighted vertex cover, nor random solution. Solution: Use linear programming to compute a lower bound on the weight of the minimum-weight vertex cover. Round this solution and use it to obtain a vertex cover. Approximation Algorithms

Objective/Constraint for Linear Programming
Minimize w(V′) = ∑ w(v) x(v). vV ′ subject to x(u) + x(v) ≥ 1 for each v  V (every edge must be covered) x(v) ≤ 1 for each v  V x(v) ≥ 0 for each v V. (a vertex is either in or not in the cover) Approximation Algorithms

Approximation Algorithms
AMINVC AMINVC(G,w) C = Ø computex, an optimal solution to the linear program for each v  V do ifx(v) ≥ 1/2 then C = C  {v} return C Approximation Algorithms

The approx.-ratio of AMINVC is 2 (1)
AMINVC is 2-approximation algorithm for the minimum-weight vertex-cover problem. Proof Let C* be an optimal solution to the minimum-weight vertex-cover problem. Let z* be the value of an optimal solution to the linear program. Since an optimal vertex cover is a feasible solution to the linear program, z* ≤ w(C*). Approximation Algorithms

The approx.-ratio of AMINVC is 2 (2)
Next, we claim that by rounding the fractional values of the variablesx(v), we produce a set C that is a vertex cover and satisfies w(C) ≤ 2z*. To see that C is a vertex cover, consider any edge (u, v)  E. Because x(u) + x(v) ≥ 1, at least one ofx(u) andx(v) is at least 1/2. Then, at least one of u and v will be included in the vertex cover, and so every edge will be covered. Approximation Algorithms

The approx.-ratio of AMINVC is 2 (3)
Now we consider the weight of the cover. We have z* = ∑ w(v) ·x(v) vV ≥ ∑ w(v)·x(v) ≥ ∑ w(v)·(1/2) vV:x(v)≥1/2 vV:x(v)≥1/2 = ∑ w(v)·(1/2) = (1/2)·∑ w(v) vC vC = w(C)/2 Thus, w(C) ≤ 2z* ≤ 2w(C*) AMINVC is a 2-approximation algorithm. Approximation Algorithms

Fully polynomial-time approximate scheme
Subset Sum Fully polynomial-time approximate scheme

Exponential-time Exact Subset Sum
EXACT-SUBSET-SUM(S, t) n ← |S| L0 ← 0 for i ← 1 to n do Li ← MERGE-LISTS(Li−1, Li−1 + xi ) remove from Li every element a, a>t return the largest element in Ln O(2n) Max. size of Li is 2n. Approximation Algorithms

Approximation Algorithms
Example Let S = {1, 4, 5} and t =8. L0 = 0 L1 = 0, 1 , L2 = 0, 1, 4, 5 , L3 = 0, 1, 4, 5, 6, 9, 10 . n ← |S| L0 ← 0 for i ← 1 to n do Li ← MERGE(Li−1,Li−1+xi ) Li+1= Li – {a|a Li, a>t} return the largest element in Ln Approximation Algorithms

fully polynomial-time approximation scheme
Approximation by trimming each list Li after it is created. If two values in L are close to each other, then one of them can be removed. Let δ, 0 < δ < 1, be a trimming parameter. x approximates y with trimming parameter δ if y/(1 + δ) ≤ x ≤ y . To trim a list L by δ remove an element z from L if there is an element x in L′ which approximates y. Approximation Algorithms

Trimming a list: Example
Let δ = 0.1 and L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29. To trim L, we obtain L′ = 10, 12, 15, 20, 23, 29. 11 is represented by 10 21 and 22 are represented by 20 24 is represented by 23. Approximation Algorithms

Approximation Algorithms
APPROX-SUBSET-SUM APPROX-SUBSET-SUM(S, t, ) n = |S| L0 = 0 for i = 1 to n do Li = MERGE-LISTS(Li−1, Li−1 + xi ) Li = TRIM(Li, /2n) Li+1= Li – {a| a Li and a>t } Find the largest value, z*, in Ln return z* Approximation Algorithms