1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 6 Tuesday, 3/23/10 NP-Completeness 1

2. Overview Graham Videotape Graham Videotape Review: Reduction to Establish Lower Bounds Review: Reduction to Establish Lower Bounds Basics Basics Practicing Some Reductions Practicing Some Reductions How to Treat NP- Hard or Complete Problems How to Treat NP- Hard or Complete Problems Simpsons Video Simpsons Video Where the Really Hard Problems Are Where the Really Hard Problems Are “Order Parameter” Critical Value “Order Parameter” Critical Value Graph Coloring Demo Graph Coloring Demo source: clip art from web 2

3. Resources Recommended:Recommended: Computers & IntractabilityComputers & Intractability by Garey & Johnsonby Garey & Johnson W.H.FreemanW.H.Freeman 19901990 ISBN 0716710455.ISBN 0716710455. For further study…For further study… Computational Complexity: A Conceptual PerspectiveComputational Complexity: A Conceptual Perspective by Oded Goldreichby Oded Goldreich 20082008 ISBN 978-0-521-88473-0ISBN 978-0-521-88473-0 3

4. Motivation 4

5. Why do we care?? When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(n k )) to solve it When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(n k )) to solve it “Intractable” problems occur frequently in practice “Intractable” problems occur frequently in practice Surface differences between tractable and intractable problems are often hard to see Surface differences between tractable and intractable problems are often hard to see e.g. MST solvable in polynomial time but Dominating Set intractable e.g. MST solvable in polynomial time but Dominating Set intractable 5

6. Dominating Set source: Garey & Johnson 6

7. Industrial Motivation Supporting Apparel Manufacturing OrderedContainment Geometric Restriction Distance-BasedSubdivision MaximumRectangle Limited Gaps MinimalEnclosure Column-Based Layout Two-Phase Layout LatticePacking Maximal Cover Containment NP-complete 7

8. Covering Research Example: Translational 2D Covering [CCCG’01, CCCG’03 conference papers] Q3Q3 Q1Q1 Q2Q2 Sample P and Q P1P1 P2P2 Translated Q Covers P P1P1 Q1Q1 Q2Q2 Q3Q3 P2P2 Translational 2D Polygon Covering Input: Input: Covering polygons Q = {Q 1, Q 2,..., Q m } Covering polygons Q = {Q 1, Q 2,..., Q m } Target polygons (or point-sets) P = {P 1, P 2,..., P n } Target polygons (or point-sets) P = {P 1, P 2,..., P n } Output: Output: Translations  = {  1,  2,...,  m } such that Translations  = {  1,  2,...,  m } such that STATIC NP-COMPLETE 8

9. Covering Problem Taxonomy covering P: finite point sets geometric covering 2D translational covering non-geometric covering P: shapes cover partition : VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location Polynomial-time algorithms for triangulation and some tilings Q: convex Q: nonconvex BOX-COVER - -Thin coverings of the plane with congruent convex shapes - -Translational covering of a convex set by a sequence of convex shapes - -Translational covering of nonconvex set with nonidentical covering shapes - -NP-hard/complete polygon problems - -polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons - -approximation algorithms for boundary, corner covers of orthogonal polygons..... covering decomposition: Q: identical... 1D interval covered by annuli 9

10. Reduction to Establish Lower Bound Tightening a Lower Bound for Maximum-Area Axis-Parallel Rectangle Problem from  (n) to  (nlgn) from  (n) to  (nlgn) 10

11. Find a Good Approximation Outer Inner A Common (sub) Problem 11

12. Given a 2D polygon that: Given a 2D polygon that: does not intersect itself does not intersect itself may have holes may have holes has n vertices has n vertices Find the Largest-Area Axis-Parallel Rectangle Find the Largest-Area Axis-Parallel Rectangle How “hard” is the problem? How “hard” is the problem? How “fast” can we find an answer? How “fast” can we find an answer? What’s the Problem? 12

13. Approach Go around lower bound brick wall by: Go around lower bound brick wall by: examining strong lower bounds for some similar problems examining strong lower bounds for some similar problems transforming a similar problem to the rectangle problem transforming a similar problem to the rectangle problem [this process is similar to what we do when we prove problems NP- complete] [this process is similar to what we do when we prove problems NP- complete] n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n 13

14. Lower Bound of  (n log n) by Transforming a ( seemingly unrelated ) Problem MAX-GAP instance: given n real numbers { x 1, x 2,... x n } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation specialized polygon x2x2x2x2 x4x4x4x4 x3x3x3x3 x1x1x1x1 Rectangle area is a solution to the MAX-GAP instance Rectangle algorithm must take at least as much time as MAX-GAP. MAX-GAP is known to be in  (n log n). Rectangle algorithm must take  (n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.] 14

15. NP-Completeness Chapter 34 15

16. Basic Concepts Polynomial Time & Problem Formalization Polynomial-Time Verification NP-Completeness & Reducibility NP-Completeness Proofs Expanding List of Hard Problems via Reduction 16

17. Polynomial Time & Problem Formalization Formalize notions of abstract and concrete decision problems Formalize notions of abstract and concrete decision problems Encoding a problem Encoding a problem Define class of polynomial-time solvable decision problems Define class of polynomial-time solvable decision problems Define complexity class Define complexity class Express relationship between decision problems and algorithms that solve them using framework of formal language theory Express relationship between decision problems and algorithms that solve them using framework of formal language theory 17

18. Polynomial Time & Problem Formalization (continued) Solvable in Polynomial Time Solvable in Polynomial Time i.e.  (n k ) for some positive constant k i.e.  (n k ) for some positive constant k Tractable: k is typically small (e.g. 1, 2, 3) Tractable: k is typically small (e.g. 1, 2, 3) Typically holds across sequential computation models Typically holds across sequential computation models Closure properties (chaining/composition) Closure properties (chaining/composition) Abstract Problem: Abstract Problem: Binary relation on set of problem instances & solutions Binary relation on set of problem instances & solutions Decision Problem: yes/no answer Decision Problem: yes/no answer impose bound on optimization problem to recast impose bound on optimization problem to recast e.g. Dominating Set e.g. Dominating Set 18

19. Dominating Set source: Garey & Johnson parameter K introduced to recast optimization problem as a decision problem 19

20. Polynomial Time & Problem Formalization (continued) Encoding of set S of abstract objects is mapping e from S to set of binary strings Encoding of set S of abstract objects is mapping e from S to set of binary strings Concrete Problem: instance set is set of binary strings Concrete Problem: instance set is set of binary strings Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time  (n k ) for some positive constant k Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time  (n k ) for some positive constant k Complexity Class P = set of polynomial-time solvable concrete decision problems Complexity Class P = set of polynomial-time solvable concrete decision problems Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) If solution to abstract-problem instance If solution to abstract-problem instance then solution to concrete-problem instance is also Q(i) then solution to concrete-problem instance is also Q(i) 20

21. Polynomial Time & Problem Formalization (continued) is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input, produces f(x) as output. is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input, produces f(x) as output. For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f 12, f 21 such that For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f 12, f 21 such that If 2 encodings e 1, e 2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent: If 2 encodings e 1, e 2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent: Goal: Extend definition of polynomial-time solvability from concrete to abstract using encodings as bridge Lemma 34.1: Let Q be an abstract decision problem on an instance set I and let e 1, e 2 be polynomially related encodings on I. Then, 21

22. Polynomial Time & Problem Formalization (continued) Formal Language Framework Formal Language Framework Alphabet  is finite set of symbols: Alphabet  is finite set of symbols: Language L over  is any set of strings consisting of symbols from  Language L over  is any set of strings consisting of symbols from  Set of instances for decision problem Q is Set of instances for decision problem Q is View Q as language L over  = {0,1} View Q as language L over  = {0,1} Express relation between decision problems and algorithms Express relation between decision problems and algorithms Algorithm A accepts string if, given input x, output A(x) is 1 Algorithm A accepts string if, given input x, output A(x) is 1 Language accepted by A is set of strings A accepts Language accepted by A is set of strings A accepts 22

23. Polynomial Time & Problem Formalization (continued) Formal Language Framework (continued) Formal Language Framework (continued) Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A accepts x in time O(n k ) Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A accepts x in time O(n k ) Language L is decided in polynomial time by algorithm A if exists constant k such that, for any length-n string, A correctly decides in time O(n k ) if Language L is decided in polynomial time by algorithm A if exists constant k such that, for any length-n string, A correctly decides in time O(n k ) if 23

24. Polynomial Time & Problem Formalization (continued) Formal Language Framework (continued) Formal Language Framework (continued) Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. We already defined: Complexity Class P = set of polynomial-time solvable concrete decision problems We already defined: Complexity Class P = set of polynomial-time solvable concrete decision problems Language Framework gives other definitions: Language Framework gives other definitions: P = { : there exists an algorithm A that decides L in polynomial time} decides L in polynomial time} = { L : L is accepted by a polynomial-time algorithm} = { L : L is accepted by a polynomial-time algorithm} 24

25. Polynomial-Time Verification Examine algorithms that “verify” membership in languages Examine algorithms that “verify” membership in languages Define class of decision problems whose solutions can be verified in polynomial time Define class of decision problems whose solutions can be verified in polynomial time 25

26. Polynomial-Time Verification (continued) Verification Algorithms Verification Algorithms algorithm A verifies language L if for any string certificate y that A can use to prove that algorithm A verifies language L if for any string certificate y that A can use to prove that also, for any string certificate y that A can use to prove that also, for any string certificate y that A can use to prove that language verified by verification algorithm A is language verified by verification algorithm A is e.g. Dominating Sete.g. Dominating Set certificate 26

27. Polynomial-Time Verification (continued) The Complexity Class NP The Complexity Class NP languages that can be verified by a polynomial-time algorithm A languages that can be verified by a polynomial-time algorithm A A verifies language L in polynomial time A verifies language L in polynomial time e.g. why? e.g. why? The Complexity Class co-NP The Complexity Class co-NP set of languages L such that set of languages L such that Recall that Recall that 27

28. Polynomial-Time Verification (continued) source: 91.503 textbook Cormen et al. P P P 34.3 28

29. NP-Completeness & Reducibility Polynomial-Time Reducibility Polynomial-Time Reducibility Define complexity class of NP-complete problems Define complexity class of NP-complete problems Relationships among complexity classes Relationships among complexity classes Circuit satisfiability as an NP-complete problem Circuit satisfiability as an NP-complete problem 29

30. NP-Completeness & Reducibility (continued) Q can be reduced to Q’ if any instance of Q can be “easily rephrased” as an instance of Q’, the solution to which provides a solution to instance of Q. Q can be reduced to Q’ if any instance of Q can be “easily rephrased” as an instance of Q’, the solution to which provides a solution to instance of Q. L 1 is polynomial-time reducible to L 2 : if exists polynomial-time computable function L 1 is polynomial-time reducible to L 2 : if exists polynomial-time computable function source: 91.503 textbook Cormen et al. 34.4 30

31. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. 34.5 34.3 31

32. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. Language is NP-complete if and without this condition, language is only NP-hard NP set of all NP- complete languages popular theoretical computer science view Theorem: - If any NP-complete problem is polynomial-time solvable, then P=NP. - Equivalently, if any problem in NP is not polynomial-time solvable, then no NP-complete problem is polynomial-time solvable. (contrapositive) 32

33. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT Need to show that starting problem is in NP First NP-complete problem: Boolean Formula Satisfiability (Cook ’71) 33

34. NP-Completeness & Reducibility (continued): Circuit Satisfiability Satisfying Assignment: truth assignment inducing output = 1 Satisfying Assignment: truth assignment inducing output = 1 GOAL: Show Circuit Satisfiability is NP-complete GOAL: Show Circuit Satisfiability is NP-complete source: 91.503 textbook Cormen et al. Boolean Combinational Gate Types: AND, NOT, OR single circuit output circuit inputs Language Def: CIRCUIT-SAT = { :C is a satisfiable boolean combinational circuit.} 34.8 34

35. NP-Completeness & Reducibility (continued) Circuit Satisfiability CIRCUIT-SAT is in NP: CIRCUIT-SAT is in NP: Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT Input 1: encoding of circuit C Input 1: encoding of circuit C Input 2: certificate = assignment of boolean values to all wires of C Input 2: certificate = assignment of boolean values to all wires of C Algorithm A : Algorithm A : For each logic gate A check if certificate value for gate’s output wire correctly computes gate’s function based on input wire values For each logic gate A check if certificate value for gate’s output wire correctly computes gate’s function based on input wire values If output of entire circuit is 1, algorithm outputs 1 If output of entire circuit is 1, algorithm outputs 1 Otherwise, algorithm outputs 0 Otherwise, algorithm outputs 0 For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 For unsatisfiable circuit, no certificate can cause A to output 1 For unsatisfiable circuit, no certificate can cause A to output 1 Algorithm A runs in polynomial time Algorithm A runs in polynomial time source: 91.503 textbook Cormen et al. 35

36. NP-Completeness & Reducibility (continued) Circuit Satisfiability CIRCUIT-SAT is NP-hard: CIRCUIT-SAT is NP-hard: L represents any language in NP L represents any language in NP Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that Must exist algorithm A verifying L in polynomial time. (A from NP) Must exist algorithm A verifying L in polynomial time. (A from NP) F uses A to compute f. F uses A to compute f. Represent computation of A as sequence of configurations Represent computation of A as sequence of configurations F constructs single circuit that computes all configurations produced by initial configuration F constructs single circuit that computes all configurations produced by initial configuration F runs in time polynomial in n=x F runs in time polynomial in n=|x| Certificate y’s length is in O(n k ) source: 91.503 textbook Cormen et al. Circuit is tailor-made for instance string x. T ( n ) copies certificate 36

37. NP-Completeness Proofs (continued) Proving a Language NP-Complete Proving a Language NP-Complete Proving a Language NP-Hard Proving a Language NP-Hard all steps except (1) all steps except (1) source: 91.503 textbook Cormen et al. Why not just pick any language in NP? 37

38. NP-Completeness Proofs (continued) Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: n boolean variables: x 1, x 2,..., x n n boolean variables: x 1, x 2,..., x n m boolean connectives: boolean function with 1 or 2 inputs and 1 output m boolean connectives: boolean function with 1 or 2 inputs and 1 output e.g. AND, OR, NOT, implication, iff e.g. AND, OR, NOT, implication, iff parentheses parentheses truth, satisfying assignments notions apply truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al. Example: Is there a satisfying assignment for this? (recall logical equivalent of implication) 38

39. NP-Completeness Proofs (continued) Show: SAT is NP-Complete Show: SAT is NP-Complete SAT is in NP via argument similar to circuit case SAT is in NP via argument similar to circuit case Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al. 39

40. NP-Completeness Proofs (continued) Show: SAT is NP-Complete (continued) Show: SAT is NP-Complete (continued) Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued) Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued) Approach: For each wire x i in circuit C, formula  has variable x i. Express gate operation. Approach: For each wire x i in circuit C, formula  has variable x i. Express gate operation. source: 91.503 textbook Cormen et al. 34.10 C 40

41. NP-Completeness Proofs (continued) Reducing Formula Satisfiability to 3-CNF- Satisfiability Reducing Formula Satisfiability to 3-CNF- Satisfiability Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  CNF = conjunctive normal form CNF = conjunctive normal form conjunction: AND of clauses conjunction: AND of clauses clause: OR of literal(s) clause: OR of literal(s) 3-CNF: each clause has exactly 3 distinct literals 3-CNF: each clause has exactly 3 distinct literals Show: 3-CNF is NP-Complete Show: 3-CNF is NP-Complete source: 91.503 textbook Cormen et al. 41

42. NP-Completeness Proofs (continued) Show: 3-CNF is NP-Complete (continued) Show: 3-CNF is NP-Complete (continued) 3CNF is in NP via argument similar to circuit case 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3-CNF-Satisfiability Reduce Formula Satisfiability to 3-CNF-Satisfiability 3 steps progressively transforming formula  closer to 3CNF 3 steps progressively transforming formula  closer to 3CNF 1) Similar to reduction 1) Similar to reduction source: 91.503 textbook Cormen et al. Binary “ParseTree” 34.11 42

43. NP-Completeness Proofs (continued) Show: 3-CNF is NP-Complete (continued) Show: 3-CNF is NP-Complete (continued) 3CNF is in NP via argument similar to circuit case 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3-CNF-Satisfiability Reduce Formula Satisfiability to 3-CNF-Satisfiability 3 steps progressively transforming formula  closer to 3CNF 3 steps progressively transforming formula  closer to 3CNF 2) Convert each clause  i ’ into conjunctive normal form 2) Convert each clause  i ’ into conjunctive normal form construct truth table for  i ’ construct truth table for  i ’ using entries for 0, build DNF formula for using entries for 0, build DNF formula for convert DNF into CNF formula  i ” by applying DeMorgan’s laws convert DNF into CNF formula  i ” by applying DeMorgan’s laws each clause has at most 3 literals each clause has at most 3 literals source: 91.503 textbook Cormen et al. truth table for  1 * * * * * use to build DNF 43

44. NP-Completeness Proofs (continued) Show: 3-CNF is NP-Complete (continued) Show: 3-CNF is NP-Complete (continued) 3CNF is in NP via argument similar to circuit case 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3-CNF-Satisfiability Reduce Formula Satisfiability to 3-CNF-Satisfiability 3 steps progressively transforming formula  closer to 3CNF 3 steps progressively transforming formula  closer to 3CNF 3) Convert each clause  i ” so it has exactly 3 distinct literals 3) Convert each clause  i ” so it has exactly 3 distinct literals add “padding variables” add “padding variables” create  i ”’ create  i ”’ Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3-CNF formula Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3-CNF formula source: 91.503 textbook Cormen et al. 44

45. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT First NP-complete problem: Boolean Formula Satisfiability (Cook ’71) 45

46. Clique source: Garey & Johnson B E C F D A Clique of an undirected graph G=(V,E) is a complete subgraph of G. clique of size 3 CLIQUE= { : graph G contains a clique of size k } source: Cormen et al. 46

47. Reducing 3-CNF-SAT to Clique x1x1x1x1 x1x1x1x1 x2x2x2x2 x2x2x2x2 x1x1x1x1 x3x3x3x3 Construct graph G such that  is satisfiable iff G has clique of size k For each clause place triple of vertices v 1 r, v 2 r, v 3 r into V. Put edge for v i r, v j s if corresponding literals consistent and r not= s (different triples). source: 91.503 textbook Cormen et al. 34.14 47

48. Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2 VERTEX-COVER= { : graph G has a vertex cover of size k } source: Cormen et al. Note difference between VERTEX-COVER and DOMINATING-SET 48

49. Reducing Clique to VertexCover source: 91.503 textbook Cormen et al. Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size |V|-k. z w u y v x 34.15 49

50. Dominating Set source: Garey & Johnson 50

51. Hamiltonian Cycle source: 91.503 textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. 34.2 NP-Complete: see textbook’s reduction from VertexCover 51

52. How to Treat NP-Hard or Complete Problems 52

53. Some Techniques for Treating NP-Complete Problems Approximation Algorithms (Chapter 35) Approximation Algorithms (Chapter 35) Heuristic Upper or Lower Bounds Heuristic Upper or Lower Bounds Greedy, Simulated Annealing, Genetic “Alg”, AI Greedy, Simulated Annealing, Genetic “Alg”, AI Mathematical Programming Mathematical Programming Linear Programming for part of problem Linear Programming for part of problem Integer Programming Integer Programming Quadratic Programming... Quadratic Programming... Search Space Exploration: Search Space Exploration: Gradient Descent, Local Search, Pruning, Subdivision Gradient Descent, Local Search, Pruning, Subdivision Randomization, Derandomization Randomization, Derandomization Leverage/Impose Problem Structure Leverage/Impose Problem Structure Leverage Similar Problems Leverage Similar Problems 53