1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 7 Tuesday, 4/6/10 Approximation Algorithms 1

2. Chapter 35 2

3. Motivation: Some Techniques for Treating NP-Complete Problems Approximation Algorithms Approximation Algorithms Heuristic Upper or Lower Bounds Heuristic Upper or Lower Bounds Greedy, Simulated Annealing, Genetic “Alg”, AI Greedy, Simulated Annealing, Genetic “Alg”, AI Mathematical Programming Mathematical Programming Linear Programming for part of problem Linear Programming for part of problem Integer Programming Integer Programming Quadratic, Convex Programming... Quadratic, Convex Programming... Search Space Exploration: Search Space Exploration: Gradient Descent, Local Search, Pruning, Subdivision Gradient Descent, Local Search, Pruning, Subdivision Randomization, Derandomization Randomization, Derandomization Leverage/Impose Problem Structure Leverage/Impose Problem Structure Leverage Similar Problems Leverage Similar Problems 3

4. Basic Concepts Definitions 4

5. Definitions Approximation Algorithm Approximation Algorithm produces “near-optimal” solution produces “near-optimal” solution Algorithm has approximation ratio  (n) if: Algorithm has approximation ratio  (n) if: Approximation Scheme Approximation Scheme (1+  )-approximation algorithm for fixed  (1+  )-approximation algorithm for fixed   > 0 is an input  > 0 is an input Polynomial-Time Approximation Scheme Polynomial-Time Approximation Scheme time is polynomial in n time is polynomial in n Fully Polynomial-Time Approximation Scheme Fully Polynomial-Time Approximation Scheme time is also polynomial in (1/  ) time is also polynomial in (1/  ) constant-factor decrease in  causes only constant-factor running-time increase constant-factor decrease in  causes only constant-factor running-time increase source: 91.503 textbook Cormen et al. C = cost of algorithm’s solution C* = cost of optimal solution n = number of inputs = size of instance minmax 5

6. Resources beyond textbook… “Approximation Algorithms for NP-Hard Problems”, editor Dorit Hochbaum, PWS Publishing, Co., 1997. “Approximation Algorithms for NP-Hard Problems”, editor Dorit Hochbaum, PWS Publishing, Co., 1997. “Approximation Algorithms” by Vazirani, 2001. “Approximation Algorithms” by Vazirani, 2001. “Computers and Intractability” by Garey & Johnson, 1979. “Computers and Intractability” by Garey & Johnson, 1979. UMass Lowell course taught by Prof. Jie Wang. UMass Lowell course taught by Prof. Jie Wang. 6

7. Overview VERTEX-COVER VERTEX-COVER Polynomial-time 2-approximation algorithm Polynomial-time 2-approximation algorithm TSP TSP TSP with triangle inequality TSP with triangle inequality Polynomial-time 2-approximation algorithm Polynomial-time 2-approximation algorithm TSP without triangle inequality TSP without triangle inequality Negative result on polynomial-time  (n)-approximation algorithm Negative result on polynomial-time  (n)-approximation algorithm MAX-3-CNF Satisfiability MAX-3-CNF Satisfiability Randomized  (n)-approximation algorithm Randomized  (n)-approximation algorithm SET-COVER SET-COVER polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm  (n) is a logarithmic function of set size  (n) is a logarithmic function of set size SUBSET-SUM SUBSET-SUM Fully polynomial-time approximation scheme Fully polynomial-time approximation scheme 7

8. Vertex Cover Polynomial-Time 2-Approximation Algorithm 8

9. Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2 NP-Complete (via reduction from CLIQUE in Ch. 34 of Cormen et al.) 9

10. Vertex Cover source: 91.503 textbook Cormen et al. b a c d e f ga d e f g a d g a a a c f g 10 edge (b,c) chosen in line 4 (vertex cover)

11. Vertex Cover Theorem: APPROX-VERTEX-COVER is a polynomial-time 2-approximation algorithm. polynomial-time 2-approximation algorithm. Proof: Observe: no 2 edges of A share any vertices in C due to removal of incident edges Any vertex cover must include >= 1 endpoint of each edge in A APPROX-VERTEX-COVER adds both endpoints of each edge of A transitivity Algorithm runs in time polynomial in n. 11

12. Traveling Salesman TSP with triangle inequality Polynomial-time 2-approximation algorithm TSP without triangle inequality Negative result on polynomial-time  (n)-approximation algorithm 12

13. Hamiltonian Cycle source: 91.503 textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. NP-Complete (via reduction from VERTEX-COVER in Ch. 34 of Cormen et al.) 34.2 13 Schlegl diagram of dodecahedron: 12 sides (one of Platonic solids) Hamiltonian Not Hamiltonian

14. Traveling Salesman Problem (TSP) source: 91.503 textbook Cormen et al. uv x w TSP Tour of a complete, undirected, weighted graph G=(V,E) is a Hamiltonian Cycle with a designated starting/ending vertex. TSP Decision Problem: Cost function provides edge weights. Note these are integers. Why? NP-Complete (via reduction from HAM-CYCLE in Ch. 34 of Cormen et al.) 14

15. Minimum Spanning Tree: Greedy Algorithms A B C D E F G 2 2 1 1 3 4 4 5 6 6 8 7 source: 91.503 textbook Cormen et al. for Undirected, Connected, Weighted Graph G=(V,E) Produces minimum weight tree of edges that includes every vertex. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree Spans all vertices at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue 15

16. TSP with Triangle Inequality source: 91.503 textbook Cormen et al. uv x w Triangle Inequality Cost Function Satisfies 3 3 4 5 5 final approximate tour optimal tour (not necessarily found by approximation algorithm) “full walk” = abcbhbadefegeda (removes vertex duplication) 16 Why Prim?

17. TSP with Triangle Inequality Theorem: APPROX-TSP-TOUR is a polynomial-time 2-approximation algorithm polynomial-time 2-approximation algorithm for TSP with triangle inequality. for TSP with triangle inequality. Proof: Algorithm runs in time polynomial in n = | V |. (lists vertices when they are first visited and when returned to after visiting subtree) (since deleting 1 edge from a tour creates a spanning tree) (since full walk traverses each edge of T twice) Now make W into a tour H using triangle inequality. (New inequality holds since H is formed by deleting duplicate vertices from W.) 17

18. TSP without Triangle Inequality Theorem: If, then for any constant polynomial-time approximation algorithm with ratio  for TSP without triangle inequality. Proof: (by contradiction) Suppose there is one --- call it A Showing how to use A to solve NP-complete Hamiltonian Cycle problemcontradiction! Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time): (G’=(V,E’) is complete graph on V) (assign integer cost to each edge in E’) For TSP problem (G’,c): G has Hamiltonian Cycle G does not have Hamiltonian Cycle (G’,c) contains tour of cost |V| Tour of G’ must use some edge not in E Cost of that tour of G’ Can use A on (G’,c)! G has Hamiltonian Cycle A finds tour of cost at most  |V| cost = |V| G does not have Hamiltonian Cycle A finds tour of cost >  |V| A finds tour of cost at most  (length of optimal tour of G ’) If, solving NP-complete hamiltonian cycle in polynomial time is a contradiction! 18

19. MAX-3-CNF Satisfiability 3-CNF Satisfiability Background 3-CNF Satisfiability Background Randomized Algorithms Randomized MAX-3-CNF SAT Approximation Algorithm 19

20. MAX-3-CNF Satisfiability Background on Boolean Formula Satisfiability Background on Boolean Formula Satisfiability Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: n boolean variables: x 1, x 2,..., x n n boolean variables: x 1, x 2,..., x n m boolean connectives: boolean function with 1 or 2 inputs and 1 output m boolean connectives: boolean function with 1 or 2 inputs and 1 output e.g. AND, OR, NOT, implication, iff e.g. AND, OR, NOT, implication, iff parentheses parentheses truth, satisfying assignments notions apply truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al. NP-Complete (via reduction from CIRCUIT-SAT in Ch. 34 of Cormen et al.) 20

21. MAX-3-CNF Satisfiability (continued) Background on 3-CNF-Satisfiability Background on 3-CNF-Satisfiability Instance of language SAT is a boolean formula  consisting of: Instance of language SAT is a boolean formula  consisting of: literal: variable or its negation literal: variable or its negation CNF = conjunctive normal form CNF = conjunctive normal form conjunction: AND of clauses conjunction: AND of clauses clause: OR of literal(s) clause: OR of literal(s) 3-CNF: each clause has exactly 3 distinct literals 3-CNF: each clause has exactly 3 distinct literals source: 91.503 textbook Cormen et al. NP-Complete (via reduction from SAT in Ch. 34 of Cormen et al.) MAX-3-CNF Satisfiability: optimization version of 3-CNF-SAT - Maximization: satisfy as many clauses as possible - Input Restrictions: - exactly 3 literals/clause - no clause contains both variable and its negation (this assumption can be removed) 21

22. Definition Randomized Algorithm has approximation ratio  (n) if, for expected cost C of solution produced by randomized algorithm: Randomized Algorithm has approximation ratio  (n) if, for expected cost C of solution produced by randomized algorithm: source: 91.503 textbook Cormen et al. size of input = n min max 22

23. Randomized Approximation Algorithm for MAX-3-CNF SAT source: 91.503 textbook Cormen et al. Theorem: Given an instance of MAX-3-CNF satisfiability with n variables x 1, x 2,..., x n with m clauses, the randomized algorithm that independently sets each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. Proof: 23

24. Set Cover Greedy Approximation Algorithm Greedy Approximation Algorithm polynomial-time  (n)-approximation algorithm  (n) is a logarithmic function of set size 24

25. Set Cover Problem source: 91.503 textbook Cormen et al. NP-Complete (via reduction from VERTEX- COVER as noted in Ch. 35 of Cormen et al.) 25

26. Greedy Set Covering Algorithm source: 91.503 textbook Cormen et al. Greedy: select set that covers the most uncovered elements 26

27. Set Cover Theorem: GREEDY-SET-COVER is a polynomial-time  ( n )-approximation algorithm polynomial-time  ( n )-approximation algorithm for for Proof: Algorithm runs in time polynomial in n. paid only when x is covered for the first time assume x is covered for the first time by S i (spread cost evenly across all elements covered for first time by S i ) Number of elements covered for first time by S i H (0)=0 elements already covered by first i-1 chosen sets 27 (notational caveat)

28. Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  ( n )-approximation algorithm polynomial-time  ( n )-approximation algorithm for for Proof: (continued) Cost assigned to optimal cover: Each x is in >= 1 S in C* 1 unit is charged at each stage of algorithm 28

29. Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: (continued) How does this relate to harmonic numbers?? We’ll show that: And then conclude that: which will finish the proof 29

30. Set Cover (proof continued) Proof of: For some set S: Number of elements of S remaining uncovered after S 1 …S i selected k = least index for which u k =0. Since, due to greedy nature of algorithm: since j <= u i-1 telescoping sum 30

31. Subset-Sum Exponential-Time Exact Algorithm Exponential-Time Exact Algorithm Fully Polynomial-Time Approximation Scheme 31

32. Subset-Sum Problem source: 91.503 textbook Cormen et al. positive integers DecisionProblem: NP-Complete (via reduction from 3-CNF-SAT in Ch. 34 of Cormen et al.) Optimization Problem seeks subset with largest sum <= t 32

33. Exponential-Time Exact Algorithm source: 91.503 textbook Cormen et al. MERGE-LISTS( L,L’ ) returns sorted list = merge of sorted L, L’ with duplicates removed. Identity: L i is sorted list of every element of P i <= t 33

34. Fully Polynomial-Time Approximation Scheme source: 91.503 textbook Cormen et al. Theorem: APPROX- SUBSET-SUM is a fully polynomial- time approximation scheme for subset-sum. Proof: see textbook 34 (differs from 2 nd edition)