1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2006 Lecture 6 Monday, 3/6/06 NP-Completeness

2. Overview ä NP-Completeness, Part 1 ä Graham videotape ä Review: Reduction to Establish Lower Bounds ä Basics ä NP-Completeness, Part 2 ä More basics ä How to Treat NP- Hard or Complete Problems

3. Resources - Recommended: ä Computers & Intractability ä by Garey & Johnson ä W.H.Freeman ä 1990 ä ISBN 0716710455.

4. Motivation

5. Why do we care?? ä When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(n k )) to solve it ä “Intractable” problems occur frequently in practice ä Surface differences between tractable and intractable problems are often hard to see ä e.g. MST solvable in polynomial time but Dominating Set intractable

6. Dominating Set source: Garey & Johnson

7. Industrial Motivation Supporting Apparel Manufacturing OrderedContainment Geometric Restriction Distance-BasedSubdivision MaximumRectangle Limited Gaps MinimalEnclosure Column-Based Layout Two-Phase Layout LatticePacking Maximal Cover Containment NP-complete

8. Covering Research Example: Translational 2D Covering [CCCG’01, CCCG’03 conference papers] Q3Q3 Q1Q1 Q2Q2 Sample P and Q P1P1 P2P2 Translated Q Covers P P1P1 Q1Q1 Q2Q2 Q3Q3 P2P2 Translational 2D Polygon Covering ä Input: ä Covering polygons Q = {Q 1, Q 2,..., Q m } ä Target polygons (or point-sets) P = {P 1, P 2,..., P n } ä Output:  Translations  = {  1,  2,...,  m } such that STATIC NP-COMPLETE

9. Covering Problem Taxonomy covering P: finite point sets geometric covering 2D translational covering non-geometric covering P: shapes cover partition : VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location Polynomial-time algorithms for triangulation and some tilings Q: convex Q: nonconvex BOX-COVER - -Thin coverings of the plane with congruent convex shapes - -Translational covering of a convex set by a sequence of convex shapes - -Translational covering of nonconvex set with nonidentical covering shapes - -NP-hard/complete polygon problems - -polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons - -approximation algorithms for boundary, corner covers of orthogonal polygons..... covering decomposition: Q: identical... 1D interval covered by annuli

10. Reduction to Establish Lower Bound Tightening a Lower Bound for Maximum-Area Axis-Parallel Rectangle Problem from  (n) to  (nlgn) from  (n) to  (nlgn)

11. Approach ä Go around lower bound brick wall by: ä examining strong lower bounds for some similar problems ä transforming a similar problem to the rectangle problem [this process is similar to what we do when we prove problems NP-complete] n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n

12. Lower Bound of  (n log n) by Transforming a ( seemingly unrelated ) Problem MAX-GAP instance: given n real numbers { x 1, x 2,... x n } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation specialized polygon x2x2x2x2 x4x4x4x4 x3x3x3x3 x1x1x1x1 Rectangle area is a solution to the MAX-GAP instance Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in  (n log n). Rectangle algorithm must take  (n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

13. NP-Completeness Chapter 34

14. Basic Concepts Polynomial Time & Problem Formalization Polynomial-Time Verification NP-Completeness & Reducibility NP-Completeness Proofs Expanding List of Hard Problems via Reduction

15. Polynomial Time & Problem Formalization ä Formalize notions of abstract and concrete decision problems ä Encoding a problem ä Define class of polynomial-time solvable decision problems ä Define complexity class ä Express relationship between decision problems and algorithms that solve them using framework of formal language theory

16. Polynomial Time & Problem Formalization (continued) ä Solvable in Polynomial Time  i.e.  (n k ) for some positive constant k ä Tractable: k is typically small (e.g. 1, 2, 3) ä Typically holds across computation models ä Closure properties (chaining/composition) ä Abstract Problem: ä Binary relation on set of problem instances & solutions ä Decision Problem: yes/no answer ä impose bound on optimization problem to recast ä e.g. Dominating Set

17. Dominating Set source: Garey & Johnson

18. Polynomial Time & Problem Formalization (continued) ä Encoding of set S of abstract objects is mapping e from S to set of binary strings ä Concrete Problem: instance set is set of binary strings  Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time  (n k ) for some positive constant k ä Complexity Class P = set of polynomial-time solvable concrete decision problems ä Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) ä If solution to abstract-problem instance ä then solution to concrete-problem instance is also Q(i)

19. Polynomial Time & Problem Formalization (continued) ä is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input, produces f(x) as output. ä For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f 12, f 21 such that ä If 2 encodings e 1, e 2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent: Goal: Extend definition of polynomial-time solvability from concrete to abstract using encodings as bridge Lemma 34.1: Let Q be an abstract decision problem on an instance set I and let e 1, e 2 be polynomially related encodings on I. Then,

20. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework  Alphabet  is finite set of symbols:  Language L over  is any set of strings consisting of symbols from  ä Set of instances for decision problem Q is  View Q as language L over  = {0,1} ä Express relation between decision problems and algorithms ä Algorithm A accepts string if, given input x, output A(x) is 1 ä Language accepted by A is set of strings A accepts

21. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A ä Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A accepts x in time O(n k ) ä Language L is decided in polynomial time by algorithm A if exists constant k such that, for any length-n string, A correctly decides in time O(n k ) if

22. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. ä We already defined: Complexity Class P = set of polynomial-time solvable concrete decision problems ä Language Framework gives other definitions: P = { : there exists an algorithm A that decides L in polynomial time} decides L in polynomial time} = { L : L is accepted by a polynomial-time algorithm} = { L : L is accepted by a polynomial-time algorithm}

23. Polynomial-Time Verification ä Examine algorithms that “verify” membership in languages ä Define class of decision problems whose solutions can be verified in polynomial time

24. Polynomial-Time Verification (continued) ä Verification Algorithms ä algorithm A verifies language L if for any string certificate y that A can use to prove that ä also, for any string certificate y that A can use to prove that ä language verified by verification algorithm A is

25. Polynomial-Time Verification (continued) ä The Complexity Class NP ä languages that can be verified by a polynomial-time algorithm A ä A verifies language L in polynomial time ä The Complexity Class co-NP ä set of languages L such that

26. Polynomial-Time Verification (continued) source: 91.503 textbook Cormen et al. P P P 34.3

27. NP-Completeness & Reducibility ä Polynomial-Time Reducibility ä Define complexity class of NP-complete problems ä Relationships among complexity classes ä Circuit satisfiability as an NP-complete problem

28. NP-Completeness & Reducibility (continued) ä Q can be reduced to Q’ if any instance of Q can be “easily rephrased as an instance of Q’, the solution to which provides solution to instance of Q ä L 1 is polynomial-time reducible to L 2 : if exists polynomial-time computable function source: 91.503 textbook Cormen et al. 34.4

29. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. 34.5 34.3

30. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. Language is NP-complete if and without this condition, language is only NP-hard NP set of all NP- complete languages popular theoretical computer science view Theorem: - If any NP-complete problem is polynomial-time solvable, then P=NP. - Equivalently, if any problem in NP is not polynomial-time solvable, then no NP-complete problem is polynomial-time solvable.

31. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT Need to show that starting problem is in NP

32. NP-Completeness & Reducibility (continued): Circuit Satisfiability ä Satisfying Assignment: truth assignment inducing output = 1 ä GOAL: Show Circuit Satisfiability is NP-complete source: 91.503 textbook Cormen et al. Boolean Combinational Gate Types: AND, NOT, OR single circuit output circuit inputs Language Def: CIRCUIT-SAT = { :C is a satisfiable boolean combinational circuit.} 34.8

33. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is in NP: ä Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT ä Input 1: encoding of circuit C ä Input 2: certificate = assignment of boolean values to all wires of C ä Algorithm A : ä For each logic gate A checks if certificate value for gate’s output wire correctly computes gate’s function based on input wire values ä If output of entire circuit is 1, algorithm outputs 1 ä Otherwise, algorithm outputs 0 ä For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 ä For unsatisfiable circuit, no certificate can cause A to output 1 ä Algorithm A runs in polynomial time source: 91.503 textbook Cormen et al.

34. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is NP-hard: ä L represents some language in NP ä Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that ä ä Must exist algorithm A verifying L in polynomial time. (A from NP) ä F uses A to compute f. ä Represent computation of A as sequence of configurations ä F constructs single circuit that computes all configurations produced by initial configuration source: 91.503 textbook Cormen et al.

35. NP-Completeness Proofs (continued) ä Proving a Language NP-Complete ä Proving a Language NP-Hard ä all steps except (1) source: 91.503 textbook Cormen et al.

36. NP-Completeness Proofs (continued) ä Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä n boolean variables: x 1, x 2,..., x n ä m boolean connectives: boolean function with 1 or 2 inputs and 1 output ä e.g. AND, OR, NOT, implication, iff ä parentheses ä truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al.

37. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete ä SAT is in NP via argument similar to circuit case ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability ä Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al.

38. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete (continued) ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued)  Approach: For each wire x i in circuit C, formula  has variable x i. Express gate operation. source: 91.503 textbook Cormen et al. 34.10

39. NP-Completeness Proofs (continued) ä Reducing Formula Satisfiability to 3-CNF- Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  ä CNF = conjunctive normal form ä conjunction: AND of clauses ä clause: OR of literal(s) ä 3-CNF: each clause has exactly 3 distinct literals ä Show: 3-CNF is NP-Complete source: 91.503 textbook Cormen et al.

40. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF ä 1) Similar to reduction source: 91.503 textbook Cormen et al. Binary “ParseTree” 34.11

41. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  2) Convert each clause  i ’ into conjunctive normal form  construct truth table for  i ’ ä using entries for 0, build DNF formula for  convert DNF into CNF formula  i ” by applying DeMorgan’s laws ä each clause has at most 3 literals source: 91.503 textbook Cormen et al.

42. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  3) Convert each clause  i ” so it has exactly 3 distinct literals ä add “padding variables”  create  i ”’ ä Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3-CNF formula source: 91.503 textbook Cormen et al.

43. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT

44. Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2

45. Dominating Set source: Garey & Johnson

46. Clique source: Garey & Johnson B E C F D A Clique of an undirected graph G=(V,E) is a complete subgraph of G. clique of size 3

47. Hamiltonian Cycle source: 91.503 textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. 34.2

48. Reducing 3-CNF-SAT to Clique x1x1x1x1 x1x1x1x1 x2x2x2x2 x2x2x2x2 x1x1x1x1 x3x3x3x3 Construct graph G such that  is satisfiable iff G has clique of size k For each clause place triple of vertices v 1 r, v 2 r, v 3 r into V. Put edge for v i r, v j s if corresponding literals consistent and r not= s. source: 91.503 textbook Cormen et al. 34.14

49. Reducing Clique to VertexCover source: 91.503 textbook Cormen et al. Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size |V|-k. z w u y v x 34.15

50. How to Treat NP-Hard or Complete Problems

51. How to Treat NP-Complete Problems?? ä Approximation Algorithms ä Heuristic Upper or Lower Bounds ä Greedy, Simulated Annealing, Genetic “Alg”, AI ä Mathematical Programming ä Linear Programming for part of problem ä Integer Programming ä Quadratic Programming... ä Search Space Exploration: ä Gradient Descent, Local Search, Pruning, Subdivision ä Randomization, Derandomization ä Leverage/Impose Problem Structure ä Leverage Similar Problems