1. Approximation Algorithms

2. Approximation Algorithms
Outlines
Why approximation algorithm?
Approximation ratio
Approximation vertex cover problem
Approximation traveling salesman problem (TSP)
Other interesting problems
Approximation Algorithms

3. 3 approaches for NP-complete Problems
For small inputs, an exponential algorithm is OK.
Try to solve important special cases in polynomial time.
Find near-optimal solutions in polynomial time (worst case or on average).
An algorithm that returns near-optimal solutions is called an approximation algorithm.
Approximation Algorithms

4. Approximation Algorithms
Approximation Ratio
If, for any input of size n for a problem, the cost C of the solution produced by an algorithm is within a factor of ρ(n) of the cost C∗ of an optimal solution (where max(C/C∗ , C∗/C) ≤ ρ(n)), we say that,
the algorithm has an approximation ratio of ρ(n) or
the algorithm is a ρ(n)-approximation algorithm.
Approximation Algorithms

5. On Approximation Ratio
maximization problem
0 < C ≤ C ∗
C ∗ /C gives the factor by which the cost of an optimal solution is larger than the cost of the approximate solution.
minimization problem
0 < C ∗ ≤ C
C/C ∗ gives the factor by which the cost of the approximate solution is larger than the cost of an optimal solution.
A 1-approximation algorithm produces an optimal solution.
large approximation ratio => a solution can be much worse than optimal.
Approximation Algorithms

6. Polynomial-time approximation algorithms
may:
have small constant approximation ratios.
have approximation ratios that grow as functions of the input size n.
achieve increasingly smaller approximation ratios by using more and more computation time.
Approximation Algorithms

7. Approximation Algorithms
Approximation scheme
An approximation algorithm that takes as input both
an instance of the problem, and
a value  > 0 such that for any fixed , the scheme is a (1 + )-approximation algorithm.
Approximation Algorithms

8. Polynomial-time Approximation Scheme
An approximation scheme is
a polynomial-time approximation scheme
if for any fixed > 0, the scheme runs in time polynomial in the size n of its input instance.
a fully polynomial-time approximation scheme
if it is an approximation scheme and its running time is polynomial both in 1/ and in the size n of the input instance.
Ex: running time of O((1/)2 n3).
Approximation Algorithms

9. Approximate Vertex Cover

10. Approximation Algorithms
Vertex Cover
A vertex cover of an undirected graph G = (V, E) is a subset V’ ⊆V such that if (u, v) is an edge of G, then either u∈V or v∈V.
The size of a vertex cover is the number of vertices in it.
An optimal vertex cover is a vertex cover of minimum size in an undirected graph.
The vertex-cover problem is to find an optimal vertex cover in a given undirected graph.
Approximation Algorithms

11. Example of Vertex CoversB A B C D E C D E F F
Approximation Algorithms

12. Approximate Vertex Cover
AVC(G)
C ← 
E ← E[G]
while E  
do Choose an arbitrary edge (u,v) from E
C ← C ∪ {u, v}
E=E-{(x, y)| x=u or x=v or y=u or y=v}
return C
Approximation Algorithms

13. Find Approximted Vertex Covers (1)
E ← E[G]
while E  
do Choose an arbitrary
edge (u,v) from E
C ← C ∪ {u, v}
E = E - {(x, y)|
x=u or x=v or
y=u or y=v}
return C A B C D E F
Approximation Algorithms

14. Find Approximted Vertex Covers (2)
E ← set of edges in G
while E  
do Choose an arbitrary
edge (u,v) from E
C ← C ∪ {u, v}
E = E - {(x, y)|
x=u or x=v or
y=u or y=v}
return C b c d a e f g c d e g f b a
Approximation Algorithms

15. Approximation ratio of AVC (1)
AVC is a polynomial-time 2-approximation algorithm.
Proof
From the algorithm, AVC runs in O(V+E), which is polynomial time.
Let C* be a set of optimal vertex cover.
Let C be the set of vertices returned by AVC.
C is a vertex cover because AVC loops until all edges in the graph is removed.
Next, we will show that |C|/|C*| ≤ 2.
Approximation Algorithms

16. Approximation ratio of AVC (2)
Let A be the set of all edges chosen in the loop.
Since all edges incident on the chosen nodes in C are removed, no two edges in A share an endpoint.
An edge is chosen in the loop when neither of its endpoints are already in C.
Thus, |C|=2|A|.
Approximation Algorithms

17. Approximation ratio of AVC (3)
For any vertex v in C*, there is at least one edge in A with v as an endpoint (because C* is a cover).
And, no two edges in A share an endpoint.
Thus, |A| ≤ |C*|.
Thus, |C|=2|A|≤ 2|C*|.
That is, |C|/|C*| ≤ 2.
Thus, AVC has approximation ratio of 2.
Approximation Algorithms

18. Traveling Salesman Problem

19. Traveling salesman problem (TSP)
Given a complete undirected graph G=(V, E) that has a nonnegative integer cost c(u, v) associated with each edge (u, v)∈ E, find a hamiltonian cycle (a tour) of G with minimum cost.
C(A) =  c(u, v)
(u,v)∈A
TSP is an NP-complete problem.
Approximation Algorithms

20. TSP with triangle inequality
The cost function c satisfies the triangle inequality if for all vertices u, v,w ∈ V,
c(u, w) ≤ c(u, v) + c(v, w) .
Example of cost functions that satisfy the triangle inequality
Euclidean distance in the plane.
Even with the triangle inequality, TSP is NP-complete.
Approximation Algorithms

21. Approximation TSP w triangle inequality
ATSP(G, c)
Select a vertex r∈V [G] to be a “root” vertex
Compute a minimum spanning tree T for G from root r using MST-PRIM(G, c, r).
Let L be the list of vertices visited in a preorder tree walk of T.
Return the hamiltonian cycle H that visits the vertices in the order L.
Approximation Algorithms

22. Example: Approximate TSP (1)
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Graph whose cost is the
euclidean distance
Minimum spanning tree
with root a.
Approximation Algorithms

23. Example: Approximate TSP (2)
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Preorder traversal and
preorder walk
Tour obtained from
preorder walk
Approximation Algorithms

24. Example: Approximate TSP (3)
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Approximate tour
Shortest tour
Approximation Algorithms

25. ATSP is 2-approximation algorithm (1)
APPROX-TSP-TOUR is a 2-approximation algorithm for the traveling-salesman problem with the triangle inequality.
Proof
Let H* denote an optimal tour for the given set of vertices.
If an edge is removed from H*, the result is a spanning tree, say Ts. Thus, c(Ts) ≤ c(H*).
Let T be a minimum spanning tree.
Then, c(T) ≤ c(Ts).
Thus, c(T) ≤ c(H*).
Approximation Algorithms

26. ATSP is 2-approximation algorithm (2)
Let W be a full walk of T.
Then, W lists the vertices when they are first visited and also whenever they are returned to after a visit to a subtree.
Since the full walk traverses every edge of T exactly twice, c(W) = 2c(T ).
From c(T) ≤ c(H*) and c(W) = 2c(T ),
we get c(W) ≤ 2c(H*).
Approximation Algorithms

27. ATSP is 2-approximation algorithm (3)
However, W is not a tour, since it visits some vertices more than once.
By the triangle inequality, we can delete a visit to any vertex from W and the cost does not increase.
(If a vertex v is deleted from W between visits to u and w, the resulting ordering specifies going directly from u to w.)
By repeatedly applying this operation, we can remove from W all but the first visit to each vertex.
Approximation Algorithms

28. ATSP is 2-approximation algorithm (4)
Let H be the cycle corresponding to this preorder walk.
H is a hamiltonian cycle, because every vertex is visited exactly once.
Since H is obtained by deleting vertices from the full walk W, c(H) ≤ c(W).
c(H) ≤ 2c(H*) since c(H) ≤ c(W) & c(W) ≤ 2c(H*).
That is, APPROX-TSP-TOUR is a 2-approximation algorithm.
Approximation Algorithms

29. MAX-3-CNF satisfiability
A randomized approximation algorithm

30. MAX-3CNF-Satisfiability
Given a 3-CNF expression , satisfiability, find an assignment of the variables that maximizes the number of clauses evaluating to 1.
According to the definition of 3-CNF satisfiability, we require each clause to consist of exactly three distinct literals.
We further assume that no clause contains both a variable and its negation.
We now show that randomly setting each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm.
Approximation Algorithms

31. Randomized MAX-3-CNF-SAT is 8/7-approximate (1)
Given an instance of MAX-3-CNF satisfiability with m clauses and variables x1, x2, , xn, the randomized algorithm that independently sets each variable to 0/1 with prob. 1/2 is a randomized 8/7-approximation algorithm.
Proof
Suppose that we have independently set each variable to 1 with probability 1/2 and to 0 with probability 1/2.
Approximation Algorithms

32. Randomized MAX-3-CNF-SAT is 8/7-approximate (2)
For i = 1, 2, , n, we define the indicator random variable
Yi = I {clause i is satisfied} ,
so that Yi = 1 as long as at least one of the literals in the ith clause has been set to 1.
Since no literal appears more than once in the same clause, and since we have assumed that no variable and its negation appear in the same clause, the settings of the three literals in each clause are independent.
Approximation Algorithms

33. Randomized MAX-3-CNF-SAT is 8/7-approximate (3)
A clause is not satisfied only if all three of its literals are set to 0, and so Pr {clause i is not satisfied} = (1/ 2)3 = 1/ 8.
Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8.
Therefore, E [Yi ] = 7/ 8.
Let Y be the number of satisfied clauses overall, so that Y = Y1 + Y2 +· · · +Ym. Then, we have m
E [Y ] = E [ ∑ Yi ]
i=1
Approximation Algorithms

34. Randomized MAX-3-CNF-SAT is 8/7-approximate (4)
E [Y ] = E [ ∑ Yi ]
m i =1
= ∑ E [Yi ] (by linearity of expectation)
i=1m
= ∑ 7/8
i=1
= 7m/8 .
Since m is an upper bound on the number of satisfied clauses, the approximation ratio is at most m/(7m/ 8) = 8/ 7.
Approximation Algorithms

35. Approximating weighted vertex cover
Using linear programming

36. Minimum-weight vertex-cover problem
For any vertex cover V′ ⊆ V, we define the weight of the vertex cover
w(V′) = ∑ w(v).
vV ′
Given an undirected graph G = (V, E) in which each vertex vV has an associated positive weight w(v), find a vertex cover of minimum weight.
We cannot apply
the algorithm used for unweighted vertex cover, nor
random solution.
Solution:
Use linear programming to compute a lower bound on the weight of the minimum-weight vertex cover.
Round this solution and use it to obtain a vertex cover.
Approximation Algorithms

37. Objective/Constraint for Linear Programming
Minimize w(V′) = ∑ w(v) x(v).
vV ′
subject to
x(u) + x(v) ≥ 1 for each v  V
(every edge must be covered)
x(v) ≤ 1 for each v  V
x(v) ≥ 0 for each v V.
(a vertex is either in or not in the cover)
Approximation Algorithms

38. Approximation Algorithms
AMINVC
AMINVC(G,w)
C = Ø
computex, an optimal solution to the linear program
for each v  V
do ifx(v) ≥ 1/2
then C = C  {v}
return C
Approximation Algorithms

39. The approx.-ratio of AMINVC is 2 (1)
AMINVC is 2-approximation algorithm for the minimum-weight vertex-cover problem.
Proof
Let C* be an optimal solution to the minimum-weight vertex-cover problem.
Let z* be the value of an optimal solution to the linear program.
Since an optimal vertex cover is a feasible solution to the linear program,
z* ≤ w(C*).
Approximation Algorithms

40. The approx.-ratio of AMINVC is 2 (2)
Next, we claim that by rounding the fractional values of the variablesx(v), we produce a set C that is a vertex cover and satisfies w(C) ≤ 2z*.
To see that C is a vertex cover, consider any edge (u, v)  E.
Because x(u) + x(v) ≥ 1, at least one ofx(u) andx(v) is at least 1/2.
Then, at least one of u and v will be included in the vertex cover, and so every edge will be covered.
Approximation Algorithms

41. The approx.-ratio of AMINVC is 2 (3)
Now we consider the weight of the cover. We have
z* = ∑ w(v) ·x(v)
vV
≥ ∑ w(v)·x(v) ≥ ∑ w(v)·(1/2)
vV:x(v)≥1/2 vV:x(v)≥1/2
= ∑ w(v)·(1/2) = (1/2)·∑ w(v)
vC vC
= w(C)/2
Thus, w(C) ≤ 2z* ≤ 2w(C*)
AMINVC is a 2-approximation algorithm.
Approximation Algorithms

42. Fully polynomial-time approximate scheme
Subset Sum
Fully polynomial-time approximate scheme

43. Exponential-time Exact Subset Sum
EXACT-SUBSET-SUM(S, t)
n ← |S|
L0 ← 0
for i ← 1 to n
do Li ← MERGE-LISTS(Li−1, Li−1 + xi )
remove from Li every element a, a>t
return the largest element in Ln
O(2n)
Max. size of Li is 2n.
Approximation Algorithms

44. Approximation Algorithms
Example
Let S = {1, 4, 5} and t =8.
L0 = 0
L1 = 0, 1 ,
L2 = 0, 1, 4, 5 ,
L3 = 0, 1, 4, 5, 6, 9, 10 .
n ← |S|
L0 ← 0
for i ← 1 to n
do Li ← MERGE(Li−1,Li−1+xi )
Li+1= Li – {a|a Li, a>t}
return the largest element in Ln
Approximation Algorithms

45. fully polynomial-time approximation scheme
Approximation by trimming each list Li after it is created.
If two values in L are close to each other, then one of them can be removed.
Let δ, 0 < δ < 1, be a trimming parameter.
x approximates y with trimming parameter δ if y/(1 + δ) ≤ x ≤ y .
To trim a list L by δ
remove an element z from L if there is an element x in L′ which approximates y.
Approximation Algorithms

46. Trimming a list: Example
Let δ = 0.1 and
L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29.
To trim L, we obtain
L′ = 10, 12, 15, 20, 23, 29.
11 is represented by 10
21 and 22 are represented by 20
24 is represented by 23.
Approximation Algorithms

47. Approximation Algorithms
APPROX-SUBSET-SUM
APPROX-SUBSET-SUM(S, t, )
n = |S|
L0 = 0
for i = 1 to n
do Li = MERGE-LISTS(Li−1, Li−1 + xi )
Li = TRIM(Li, /2n)
Li+1= Li – {a| a Li and a>t }
Find the largest value, z*, in Ln
return z*
Approximation Algorithms