1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 6 Tuesday, 10/16/01 NP-Completeness, Part 2

2. Overview ä NP-Completeness ä Motivation ä Basics ä Literature ä Reduction to Establish Lower Bounds ä Preview of Coming Attractions... ä Homework #3 ä Midterm Exam ä Chapter Dependencies

3. NP-Completeness Chapter 36

4. Motivation

5. Why do we care?? ä When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(n k )) to solve it ä “Intractable” problems occur frequently in practice ä Surface differences between tractable and intractable problems are often hard to see

6. UMass Lowell Algorithms Lab Telecommunications Data Mining & Visualization Manufacturing Design Analyze Apply PhD research Undergraduate & Masters research OS 305 Prof. Daniels

7. Industrial Motivation Supporting Apparel Manufacturing OrderedContainment Geometric Restriction Distance-BasedSubdivision MaximumRectangle Limited Gaps MinimalEnclosure Column-Based Layout Two-Phase Layout LatticePacking Maximal Cover Containment NP-complete

8. Dynamic Covering Motivation sensor coverage sensor coverage clustering clustering shape recognition shape recognition channel channel assignment assignment production allocation production allocation facility location facility location layout layout

9. New Covering Research: Objective ä Practical solutions to dynamic hard core covering problems ä dynamic instance = sequence of static instances ä hard : even static problem is computationally “intractable” ä core : useful in multiple applications ä covering ~ assignment ~ allocation ä Practical : solve static instance i quickly if it is “easy” ä combinatorially “small” or ä clearly infeasible (for decision problem) or ä clearly feasible (for decision problem) or ä a “small” change from instance i-1 static instance 0 static instance 1 static instance i...

10. State-of-the-Art: Dynamic Algorithms & Data Structures ä Polynomial-time problems: ä Data structures supporting fast updates ä Self-adjusting (e.g. splay trees) [Sleator, CMU] ä Kinetic Data Structures [Guibas, Stanford] ä Hard problems: ä Polynomial-time approximation schemes for optimization problems : “Greedy” can be sensitive to small changes [Gao, et al.] ä Meta-heuristics: no theoretical guarantee ä Practical algorithms for small and/or easy instances: ä Containment decision problem: [Daniels, Milenkovic] ä successful building block for layout problems ä leverages strong combinatorial optimization techniques ä Distributed heuristics ä Primarily for dynamic network routing or channel assignment ä [Wu et al.] connected dominating sets for network routing ä no theoretical guarantee; some empirical support ä Randomized, distributed: ä [Gao, et al.] discrete mobile centers (clustering) ä theoretical framework; assumes knowledge of change trajectories static

11. State-of-the-Art: Covering Problems covering P: finite point sets geometric covering 2D translational covering non-geometric covering P: shapes cover partition : VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location Polynomial-time algorithms for triangulation and some tilings Q: convex Q: nonconvex BOX-COVER - -Thin coverings of the plane with congruent convex shapes - -Translational covering of a convex set by a sequence of convex shapes - -Translational covering of nonconvex set with nonidentical covering shapes - -NP-hard/complete polygon problems - -polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons - -approximation algorithms for boundary, corner covers of orthogonal polygons..... covering decomposition: Q: identical... 1D interval covered by annuli

12. New Covering Research Example 1: Translational 2D Covering [recent work with grad student Rajasekhar Inkulu (CCCG conference paper)] Q3Q3 Q1Q1 Q2Q2 Sample P and Q P1P1 P2P2 Translated Q Covers P P1P1 Q1Q1 Q2Q2 Q3Q3 P2P2 Translational 2D Polygon Covering ä Input: ä Covering polygons Q = {Q 1, Q 2,..., Q m } ä Target polygons (or point-sets) P = {P 1, P 2,..., P n } ä Output:  Translations  = {  1,  2,...,  m } such that STATIC NP-COMPLETE

13. New Covering Research Example 2: 2D Channel Assignment [recent work with Masters student Sa Liu (GLOBECOM conference paper) ] ä Input: ä r x c cell grid ä q channels (frequencies) ä D(i,j) = cell(i,j) traffic demand ä # cell-phone calls ä Co-channel interference constraints based on interference threshold value ä Output: ä Feasible assignment of channels to cells that ä satisfies demand and co-channel interference constraints ä uses minimum number of channels e.g. no channel repetition within any 2 x 2 square STATIC NP-COMPLETE

14. Basic Concepts Polynomial Time & Problem Formalization Polynomial-Time Verification NP-Completeness & Reducibility NP-Completeness Proofs Expanding List of Hard Problems via Reduction

15. Polynomial Time & Problem Formalization ä Formalize notions of abstract and concrete decision problems ä Encoding a problem ä Define class of polynomial-time solvable decision problems ä Define complexity class ä Express relationship between decision problems and algorithms that solve them using framework of formal language theory

16. Polynomial Time & Problem Formalization (continued) ä Solvable in Polynomial Time  i.e.  (n k ) for some positive constant k ä Tractable; k is typically small (e.g. 1, 2, 3) ä Typically holds across computation models ä Closure properties (chaining/composition) ä Abstract Problem: denote by Q ä binary relation on a set I of problem instances and set S of problem solutions ä Decision Problem: yes/no answer ä impose bound on optimization problem to recast

17. Polynomial Time & Problem Formalization (continued) ä Encoding of set S of abstract objects is mapping e from S to set of binary strings ä Concrete Problem: instance set is set of binary strings  Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time  (n k ) for some positive constant k ä Complexity Class P = set of polynomial-time solvable concrete decision problems ä Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) ä If solution to abstract-problem instance ä then solution to concrete-problem instance is also Q(i)

18. Polynomial Time & Problem Formalization (continued) ä is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input, produces f(x) as output. ä For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f 12, f 21 such that ä If 2 encodings e 1, e 2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent: Lemma 36.1: Let Q be an abstract decision problem on an instance set I and let e 1, e 2 be polynomially related encodings on I. Then, Goal: Extend definition of polynomial-time solvability from concrete to abstract using encodings as bridge

19. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework  Alphabet  is finite set of symbols  Language L over  is any set of strings consisting of symbols from  ä Set of instances for decision problem Q is  View Q as language L over  = {0,1} ä Algorithm A accepts string if, given input x, output A(x) is 1 ä Language accepted by A is set of strings A accepts

20. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A ä Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A accepts x in time O(n k ) ä Language L is decidable in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A correctly decides in time O(n k ) if

21. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. ä We already defined: Complexity Class P = set of polynomial- time solvable concrete decision problems ä Language Framework gives other definitions: P = { : there exists an algorithm A that decides L in polynomial time} decides L in polynomial time} = { L : L is accepted by a polynomial-time algorithm} = { L : L is accepted by a polynomial-time algorithm}

22. Polynomial-Time Verification ä Examine algorithms that “verify” membership in languages ä Define class of decision problems whose solutions can be verified in polynomial time

23. Polynomial-Time Verification (continued) ä Verification Algorithms ä algorithm A verifies language L if for any string certificate y that A can use to prove that ä also, for any string certificate y that A can use to prove that ä language verified by verification algorithm A is

24. Polynomial-Time Verification (continued) ä The Complexity Class NP ä languages that can be verified by a polynomial- time algorithm ä A verifies language L in polynomial time ä The Complexity Class co-NP ä set of languages such that

25. Polynomial-Time Verification (continued) source: 91.503 textbook Cormen et al. P P P

26. NP-Completeness & Reducibility ä Polynomial-Time Reducibility ä Define complexity class of NP-complete problems ä Relationships among complexity classes ä Circuit satisfiability as an NP-complete problem

27. NP-Completeness & Reducibility (continued) ä Q can be reduced to Q’ if any instance of Q can be “easily rephrased as an instance of Q’, the solution to which provides solution to instance of Q ä L 1 is polynomial-time reducible to L 2 if exists polynomial-time computable function source: 91.503 textbook Cormen et al.

28. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al.

29. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. Language is NP-complete if and without this condition, language is only NP-hard NP set of all NP- complete languages popular theoretical computer science view Theorem: - If any NP-complete problem is polynomial-time solvable, then P=NP. - Equivalently, if any problem in NP is not polynomial-time solvable, then no NP-complete problem is polynomial-time solvable.

30. NP-Completeness & Reducibility (continued): Circuit Satisfiability ä Satisfying Assignment: truth assignment inducing output = 1 ä GOAL: Show Circuit Satisfiability is NP-complete source: 91.503 textbook Cormen et al. Boolean Combinational Gate Types: AND, NOT, OR single circuit output circuit inputs Language Def: CIRCUIT-SAT = { :C is a satisfiable boolean combinational circuit.}

31. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is in NP: ä Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT ä Input 1: encoding of circuit C ä Input 2: certificate = assignment of boolean values to all wires of C ä Algorithm A : ä For each logic gate A checks if certificate value for gate’s output wire correctly computes gate’s function based on input wire values ä If output of entire circuit is 1, algorithm outputs 1 ä Otherwise, algorithm outputs 0 ä For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 ä For unsatisfiable circuit, no certificate can cause A to output 1 ä Algorithm A runs in polynomial time source: 91.503 textbook Cormen et al.

32. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is NP-hard: ä L represents some language in NP ä Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that ä ä Must exist algorithm A verifying L in polynomial time. (A from NP) ä F uses A to compute f. ä Represent computation of A as sequence of configurations ä F constructs single circuit that computes all configurations produced by initial configuration source: 91.503 textbook Cormen et al.

33. NP-Completeness Proofs (continued) ä Proving a Language NP-Complete ä Proving a Language NP-Hard ä all steps except (1) source: 91.503 textbook Cormen et al.

34. NP-Completeness Proofs (continued) ä Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä n boolean variables: x 1, x 2,..., x n ä m boolean connectives: boolean function with 1 or 2 inputs and 1 output ä e.g. AND, OR, NOT, implication, iff ä parentheses ä truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al.

35. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete ä SAT is in NP via argument similar to circuit case ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability ä Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al.

36. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete (continued) ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued)  Approach: For each wire x i in circuit C, formula  has variable x i. Express gate operation. source: 91.503 textbook Cormen et al.

37. NP-Completeness Proofs (continued) ä Reducing Formula Satisfiability to 3-CNF- Satisfiability (continued)  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä literal: variable or its negation ä CNF = conjunctive normal form ä conjunction: AND of clauses ä clause: OR of literal(s) ä 3-CNF: each clause has exactly 3 distinct literals ä Show: 3-CNF is NP-Complete source: 91.503 textbook Cormen et al.

38. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF ä 1) Similar to reduction source: 91.503 textbook Cormen et al.

39. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF ä 1) Similar to reduction source: 91.503 textbook Cormen et al. Binary “ParseTree”

40. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  2) Convert each clause  i ’ into conjunctive normal form  construct truth table for  i ’ ä using entries for 0, build DNF formula for  convert DNF into CNF formula  i ” by applying DeMorgan’s laws ä each clause has at most 3 literals source: 91.503 textbook Cormen et al.

41. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  3) Convert each clause  i ” so it has exactly 3 literals ä add “padding variables” ä Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and source: 91.503 textbook Cormen et al.

42. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT

43. Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 3

44. Clique source: Garey & Johnson B E C F D A Clique of an undirected graph G=(V,E) is a complete subgraph of G. clique of size 3

45. Hamiltonian Cycle source: 91.503 textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V.

46. Reducing 3-CNF-SAT to Clique x1x1x1x1 x1x1x1x1 x2x2x2x2 x2x2x2x2 x1x1x1x1 x3x3x3x3 Construct graph G such that is satisfiable iff G has clique of size k For each clause place triple of vertices v 1 r, v 2 r, v 3 r into V. Put edge for v i r, v j s if corresponding literals consistent and r not= s. source: 91.503 textbook Cormen et al.

47. Reducing Clique to VertexCover source: 91.503 textbook Cormen et al. Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size |V|-k. z w u y v x

48. Reducing 3SAT to Vertex Cover source: Garey & Johnson

49. Literature

50. “Optimal Packing and Covering in the Plane are NP-Complete” [Fowler et al.] source: Fowler et al.

51. Reduction to Establish Lower Bound Tightening a Lower Bound for Maximum-Area Axis-Parallel Rectangle Problem from  (n) to  (nlgn) from  (n) to  (nlgn)

52. Approach ä Go around lower bound brick wall by: ä examining strong lower bounds for some similar problems ä transforming a similar problem to our problem [this process is similar to what we do when we prove problems NP- complete] n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n

53. Lower Bounds for Related Problems n1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 SmallestOuterRectangle:   (n) SmallestOuterCircle:   (n) LargestInnerRectangle:   (n) LargestInnerCircle:   (n log n) point set, polygon point set polygon LargestInnerRectangle:   (n log 2 (n))polygon worst-case bounds on our problem

54. Lower Bound of  (n log n) by Transforming a ( seemingly unrelated ) Problem MAX-GAP instance: given n real numbers { x 1, x 2,... x n } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation specialized polygon x2x2x2x2 x4x4x4x4 x3x3x3x3 x1x1x1x1 Rectangle area is a solution to the MAX-GAP instance Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in  (n log n). Rectangle algorithm must take  (n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

55. Preview of Coming Attractions...

56. Average-Case Complexity: “Hard on Average” ä Prof. Jie Wang, new CS faculty, Math Colloquium ä Wednesday, 17 Oct. 2:30-3:30 OS402 ä ABSTRACT: “Average case analysis always seem more relevant than the worst case. While NP-completeness is a robust notion for measuring computational intractability, it is only a worst- case concept. We are interested in identifying problems that are intractable on average, and distinguish (standard, worst-case) NP- complete problems that are "easy on average" from those that are "difficult on average." Such results can be used in areas (such as cryptography) where average hardness is a frequent assumption. In this talk, we will discuss important issues in forming robust notions of average time and feasible reductions that are suitable for studying average-case NP-completeness.”

57. How to Treat NP-Complete Problems?? ä Approximation Algorithms ä Heuristic Upper or Lower Bounds ä Greedy, Simulated Annealing, Genetic “Alg”, AI ä Mathematical Programming ä Linear Programming for part of problem ä Integer Programming ä Quadratic Programming... ä Search Space Exploration: ä Gradient Descent, Local Search, Pruning, Subdivision ä Randomization, Derandomization ä Leverage/Impose Problem Structure ä Leverage Similar Problems

58. HW#3

59. Dominating Set source: Garey & Johnson

60. Midterm Exam

61. Chapter Dependencies Ch 1-6 Math Review: Asymptotics, Recurrences, Summations, Sets, Graphs, Counting, Probability, Calculus, Proofs Techniques (e.g. Inductive) Logarithms Ch 7-10 Sorting Ch 11-14, 15 Data Structures Ch 16, 17, 18 Advanced Design & Analysis Techniques Ch 19-22 Advanced Data Structures Ch 23-25,26,27 Graph Algorithms Ch 28, 29 Parallel Comparison-Based Sorting Networks, Arithmetic Circuits Ch 30 Parallel Comparison-Based Sorting Networks Ch 31 Matrix Operations Math: Linear Algebra Ch 35 Computational Geometry Math: Geometry (High School Level) Ch 36 NP-Completeness Ch 37 Approximation Algorithms You’re responsible for all sections of this chapter, plus extra material from Garey & Johnson

62. Backups

70. “Optimal Packing and Covering in the Plane are NP-Complete” [Fowler et al.] source: Fowler et al.