1. 1 Approximation algorithms Algorithms and Networks 2015/2016 Hans L. Bodlaender Johan M. M. van Rooij TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A AAA

2. 2 What to do if a problem is NP-complete?  We have already seen many options to deal with NP- complete problems.  FTP, special cases, exact exponential-time algorithms, heuristics.  In other courses: local search, ILP, constraint programming, …  Approximation algorithms are one of these options.  An approximation algorithm is a heuristic with a performance guarantee.  We consider polynomial-time approximation algorithms.  Non-optimal solutions, but with some performance guarantee compared to the optimal solution.  Also useful as a starting point for other approaches:  Local search, branch and bound.

3. 3 Different forms of approximation algorithms (outline of lecture) Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference.  |OPT – ALG| · c 2. APX: Constant-factor approximation.  Approximation ratio: ALG/OPT · c for minimisation problems.  Approximation ratio: OPT/ALG · c for maximisation problems. 3. f(n)-APX: Approximation by a factor of f(n).  f(n) depends only one the size of the input. 4. PTAS: Polynomial-time approximation scheme.  Approximation ratio 1+ ² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². 5. FPTAS: Fully polynomial-time approximation scheme.  Approximation ratio 1+ ² for any ² > 0, while the algorithm runs in time polynomial in n and 1/ ².

4. 4 Absolute constant difference  Algorithms that run in worst-case polynomial time, with:  |OPT – ALG| · c  Example: Planar graph colouring.  Algorithm that tests 1 and 2-colourability and if no solution found always outputs 4.  Algorithm has an error of at most 1.  These examples are rare.  Not so difficult to show:  TSP cannot be approximated by a constant factor unless PNP.

5. 5 Constant-factor approximation  Approximation ratio:  ALG/OPT · c for minimisation problems.  OPT/ALG · c for maximisation problems.  Ratio always bigger or equal to 1.  Class of problems with a constant factor approximation algorithm: APX.  Notions of APX-completeness also exist.  Examples of constant-factor approximation algorithms from earlier lectures:  2-approximations: MST based algorithms for metric TSP. (TSP with triangle inequality).  1.5-approximation: Christofides algorithm for metric TSP.

6. 6 Approximation for vertex cover Approximation algorithm for vertex cover: 1. Let E’ = E, C = ; 2. While E’  ; 1. Let {u,v} be any edge from E’ 2. C = C [ {u,v} 3. Remove every edge incident to u of v from E’. 3. Return C  Runs in polynomial time.  Returns a vertex cover.  How good is this vertex cover?

7. 7 2-approximation for vertex cover Theorem:  The algorithm on the previous slide is a 2-approximation. Proof:  Let A be the set of edges which endpoints we picked.  OPT ¸ |A|, because every edge in A must be covered.  ALG = 2|A| · 2OPT, hence ALG/OPT · 2.

8. 8 Approximation by a factor of f(n)  Approximation ratio of f(n).  Approximation ratio depends on the size of the input (and can be very bad) Set Cover  Given: finite set U and a family F of subsets of U.  Question: Does there exists a subset C of F of size at most k such that [ S2C S = U? Greedy algorithm:  While U  ; :  Select S 2 F that maximises |S Å U|  Add S to C and let U := U \ S

9. 9 An (ln(n)+1)-approximation algorithm for set cover Theorem:  The greedy set cover algorithm is an (ln(n)+1)-approximation. Proof:  The algorithm runs in polynomial time and returns a set cover.  Let S i be the i th set from F picked by the algorithm.  We assign cost c x to elements from x:  Let x be firstly covered by set S i while running the algorithm.  And, let X i = S i \ (S 1 [ S 2 [... [ S i-1 )  Define: c x = 1 / |X i |.  Now we have: ...

10. 10 Proof of the (ln(n)+1)-approximation algorithm for set cover  The n th Harmonic number H(n) is defined as:  From mathematics (bounding using integrals):  On the next two slides, we will prove that for any S 2 F:  From this we derive our approximation ratio:  Where C* is the optimal set cover.

11. 11 Bounding the cost of any set S  What remains is to prove that for any S 2 F:  Remind:  If x firstly covered by S i, then c x = 1 / |X i |  Where: X i = S i \ (S 1 [ S 2 [... [ S i-1 )  For any S 2 F, let u i be the number of uncovered items in S when the algorithm selects S i.  Take the smallest value k s.t. u k = 0.  With the last equality due to the selection by the algorithm.  Then...

12. 12 Bounding the cost of any set S  To prove:  Last slide:  Thus:  Remind:  Q.E.D.

13. 13 PTAS  A Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a real value >0.  For each fixed >0, the algorithm  Uses polynomial time  Is an (1+)-approximation algorithm

14. 14 FPTAS  A Fully Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a real value >0, and  For each fixed >0, the algorithm  Is an (1+)-approximation algorithm  The algorithm uses time that is polynomial in the size of X and 1/.

15. 15 Example: Knapsack  Given: n items each with a positive integer weight w(i) and integer value v(i) (1 ≤ i ≤ n), and an integer B.  Question: select items with total weight at most B such that the total value is as large as possible.

16. 16 Dynamic Programming for Knapsack  Let P be the maximum value of any item.  We can solve the problem in O(n 2 P) time with dynamic programming:  Tabulate M(i,Q): minimum total weight of a subset of items 1, …, i with total value Q for Q at most nP  M(0,0) = 0  M(0,Q) = ∞, if Q>0  M(i+1,Q) = min{ M(i,Q), M(i,Q-v(i+1)) + w(i+1) }  This algorithm is clearly correct and runs in the given time.

17. 17 Scaling for Knapsack  Take input for knapsack  Throw away all items that have weight larger than B (they are never used)  Let c be some constant  Build new input: do not change weights, but set new values v’(i) = v(i) / c   Solve scaled instance with DP optimally  Output this solution: approximates solution for original instance

18. 18 The Question is….  How do we set c, such that  Approximation ratio good enough  Polynomial time

19. 19 Approximation ratio  Consider optimal solution Y for original problem, value OPT  Value in scaled instance: at least OPT/c – n  At most n items, for each v(i)/c - v(i)/c  <1  So, DP finds a solution of value at least OPT/c –n for scaled problem  So, value of approximate solution for original instance is at least c*(OPT/c –n) = OPT - nc

20. 20 Setting c  Set c = P/(2n)  This is an FPTAS  Running time:  Largest value of an item is at most P/c = P / (P/(2n)) = 2n/.  Running time is O(n 2 * 2n/) = O(n 3 /)  Approximation: … next

21. 21 -approximation  Note that each item is a solution (we removed items with weight more than B).  So OPT ≥ P.  Algorithm gives solution of value at least: OPT – nc = OPT – n( P / (2n) ) = OPT – /2 P  OPT / (OPT – /2 P)≤ OPT / (OPT – /2 OPT) = 1/(1-/2) ≤ 1+  QED.

22. 22 2-approximation for minimum weight vertex cover  Minimum weight vertex cover:  Vertex cover where each vertex has a weight.  We look for the minimum weight vertex cover.  2-approximation for vertex cover no longer works.  We may select very heavy vertices using that algorithm.  Consider the following ILP:  It’s LP relaxation is ILP with the last constraint replaced by:  0 · x(v) · 1

23. 23 2-approximation algorithm for minimum weight vertex cover Algorithm:  Compute the optimal solution to the LP relaxation.  Output all v with x(v) ¸ ½.  Algorithm runs in polynomial time.  Linear programming can be solved in polynomial time.  Not by the simplex algorithm!!  Ellipsoid method / interior point methods.  Algorithm returns a vertex cover.  For every edge, sum of incident vertices at least 1.  Hence at least one of the vertex variables at least ½.

24. 24 Proof of 2-approximation algorithm for minimum weight vertex cover  Let z* be the solution to the LP.  Because any vertex cover is a solution to the LP we have:  Also, we can bound ALG in terms of z*:  Hence:  QED

25. 25 Conclusion Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference.  Planar graph colouring. 2. APX: Constant-factor approximation.  TSP with triangle inequality.  Vertex cover. 3. f(n)-APX: Approximation by a factor of f(n).  Set cover. 4. PTAS: Polynomial-time approximation scheme. 5. FPTAS: Fully polynomial-time approximation scheme.  Scaling for Knapsack.