1. 1 Ch 10 - NP-completeness Tractable and intractable problems Decision/Optimization problems Deterministic/NonDeterministic algorithms Classes P and NP Polynomial reductions NP-complete problems –Cook-Levin theorem –CLIQUE (via SAT  CLIQUE reduction) –TSP (via HC  TSP reduction)

2. 2 Classifying problems Classify problems as tractable or intractable. Problem is tractable if there exists at least one polynomial bound algorithm to solve it. An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

3. 3 Intractable problems Problem is intractable if it is not tractable. All algorithms that solve such a problem are not polynomial bound. It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem. For intractable problems the bounds are:

4. 4 Why is this classification useful? If problem is intractable, no point in trying to find an efficient algorithm All algorithms will be too slow for large inputs.

5. 5 Hard practical problems There are many practical problems for which no one has yet found a polynomial bound algorithm. Examples: traveling salesperson, 0/1 knapsack, graph coloring, bin packing etc. Most design automation problems such as testing and routing. Many networks, database and graph problems.

6. 6 The theory of NP completeness The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet) So any algorithm that solves it will probably be very slow for large inputs

7. 7 Decision Problems vs. Optimization Problems Decision Problem: Yes/no answer Optimization Problem: Maximization or minimization of a certain quantity When studying NP-Completeness, it is easier to deal with decision problems than optimization problems

8. 8 Element Uniqueness Problem Decision Problem: Element Uniqueness –Input: A sequence of integers S –Question: Are there two elements in S that are equal? Optimization Problem: Element Count –Input: A sequence of integers S –Output: An element in S of highest frequency What is the algorithm to solve this problem? How much does it cost?

9. 9 Coloring a Graph Decision Problem: Coloring –Input: G=(V,E) undirected graph and k , k > 0. –Question: Is G k-colorable? Optimization Problem: Chromatic Number –Input: G=(V,E) undirected graph –Output: The chromatic number of G,  (G) i.e. the minimum number  (G) of colors needed to color a graph in such a way that no two adjacent vertices have the same color.

10. 10 Traveling Salesman Given a finite set C={c 1,...,c m } of cities, a distance function d(c i, c j ) of nonnegative numbers and a bound B Decision Problem –Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B? Optimization Problem –Find the length of the minimum distance tour which includes every city exactly once

11. 11 Cliques Definition: A clique of size k in G, for some +ve integer k, is a complete subgraph of G with k vertices. Decision Problem –Input: –Question: Optimization Problem –Input: –Output:

12. 12 The relation between If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”. Therefore if the optimization problem is tractable so is the decision problem If the decision problem is “hard” the optimization problem is also “hard” –If the optimization was easy then the decision problem is easy.

13. 13 Deterministic Algorithms Definition: Let A be an algorithm to solve problem . A is called deterministic if, when presented with an instance of the problem , it has only one choice in each step throughout its execution. –If we run A again and again, is there a possibility that the output may change? What type of algorithms did we have so far?

14. 14 The Class P Definition: The class of decision problems P consists of those whose yes/no solution can be obtained using a deterministic algorithm that runs in polynomial time of steps, i.e. O(n k ), where k is a non-negative integer and n is the input size.

15. 15 Examples Sorting: Given n integers, are they sorted in non-decreasing order? Set Disjointness: Given two sets of integers, are they disjoint? Shortest path: 2-coloring: –Theorem: A graph G is 2-colorable if and only if G is bipartite if and only if G has no odd length cycle

16. 16 Non-Deterministic Algorithms A non-deterministic algorithm A on input x consists of two phases: –Guessing: An arbitrary “string of characters y” is (called certificate) generated. It may or may not correspond to a solution Not be in proper format of a solution Differ from one run to another –Verification: A deterministic algorithm verifies The generated “string of characters y” is a solution

17. 17 Non-Deterministic Algorithms Definition: Let A be a nondeterministic algorithm for a problem . We say that A accepts an instance I of  if and only if on input I, there exists a guess that leads to a yes answer. –Does it mean that if an algorithm A on a given input I leads to an answer of no for a certain guess, that it does not accept it?

18. 18 The Class NP Definition: The class of decision problems NP consists of those decision problems for which there exists a nondeterministic algorithm that runs in polynomial time. –The running time of a nondeterministic algorithm, is the sum of the running times of the guessing and verification phases. –Both phases need run in polynomial time. –In particular, the verification phase.

19. 19 Example - TSP Show that the traveling salesman problem belongs to the class of NP problems Given a problem instance x for TSP and a certificate y –Check that y is indeed a cycle that includes every vertex exactly once –Verify that the length of the cycle is at most B Is the verification algorithm polynomial?

20. 20 P and NP Problems What is the difference between P problems and NP Problems? We can solve problems in P using deterministic algorithms that run in polynomial time We can check or verify the solution of NP problems in polynomial time using a deterministic algorithm What is the set relationship between the classes P and NP?

21. 21 Summary: P and NP Summary so far: –P = problems that can be solved in poly time –NP = problems for which a solution can be verified in polynomial time –P  NP –Unknown whether P = NP (most suspect not) We’ve seen problems that belong to NP that may not belong to P –Hamiltonian path/cycle, TSP problems are in NP –Cannot solve in polynomial time –Easy to verify solution in polynomial time

22. 22 NP-Complete Problems We will see that NP-Complete problems are the “hardest” problems in NP: –If any one NP-Complete problem can be solved in polynomial time… –…then every NP-Complete problem can be solved in polynomial time… –…and in fact every problem in NP can be solved in polynomial time (showing P = NP) –Thus: solve hamiltonian-cycle in O(n 100 ) time, you’ve proved that P = NP. Retire rich & famous.

23. 23 Reduction The crux of NP-Completeness is reducibility –Informally, a problem A can be reduced to another problem B if any instance of A can be “easily rephrased” as an instance of B, the solution to which provides a solution to the instance of A This rephrasing is called transformation –Intuitively: If A reduces to B, A is “no harder to solve” than B Total cost: cost of transformation + cost to solve B

24. 24 Reduction: A reduces to B An example: –A: Given a set of Booleans, is at least one TRUE? –B: Given a set of integers, is their sum positive? –Transformation: (x 1, x 2, …, x n ) = (y 1, y 2, …, y n ) where y i = 1 if x i = TRUE, y i = 0 if x i = FALSE IAIA A solver IBIB OBOB OAOA Transformer B solver

25. 25 NP-Hard and NP-Complete If A is polynomial-time reducible to B, we denote this A  p B Definition of NP-Hard and NP-Complete: –If all problems X  NP are reducible to A, then A is NP-Hard –We say A is NP-Complete if A is NP-Hard and A  NP If A  p B and A is NP-Complete, B is also NP-Complete

26. 26 Why Prove NP-Completeness? Though nobody has proven that P ≠ NP, if you prove a problem NP-Complete, most people accept that it is probably intractable Therefore it can be important to prove that a problem is NP-Complete –Don’t need to come up with an efficient algorithm –Can instead work on approximation algorithms

27. 27 Proving NP-Completeness What steps do we have to take to prove a problem A is NP-Complete? –Pick a known NP-Complete problem B –Reduce B to A Describe a transformation that maps instances of B to instances of A, s.t. “yes” for A = “yes” for B Prove the transformation works Prove it runs in polynomial time –Oh yeah, prove A  NP

28. 28 Examples of NP-Complete problems Cook’s theorem –Satisfiability is NP-complete This was the first problem shown to be NP- complete Other problems –Graph coloring (= register allocation) –Hamiltonian path –Traveling salesman –Knapsack

29. 29 Conjunctive Normal Form (CNF) A logical (Boolean) variable is a variable that may be assigned the value true or false (p, q, r and s are Boolean variables) A literal is a logical variable or the negation of a logical variable (p and  q are literals) A clause is a disjunction of literals ( (p  q  s) and (  q  r) are clauses)

30. 30 Conjunctive Normal Form (CNF) A logical (Boolean) expression is in Conjunctive Normal Form if it is a conjunction of clauses. The following expression is in conjunctive normal form: (p  q  s)  (  q  r)  (  p  r)  (  r  s)  (  p  s  q)

31. 31 The Satisfiability problem Is there a truth assignment to the n variables of a logical expression in Conjunctive Normal Form which makes the value of the expression true? –For the answer to be “yes”, all clauses must evaluate to true –Otherwise the answer is “no” p=T, q=F, r=T and s=T is a truth assignment for: (p  q  s)  (  q  r)  (  p  r)  (  r  s)  (  p  s  q)

32. 32 The Satisfiability problem Theorem: Satisfiability is an NP-Complete problem. Proof: –Satisfiability  NP: Given a truth assignment of a Boolean formula consisting of n literals, it is clear that we can evaluate it in poly time. –Satisfiability  NP-Hard: The proof involves reducing every problem in NP to Satisfiability in polynomial time. See Garey & Johnson.

33. 33 Proving NP-Completeness SAT 3-CNF-SAT Subset-Sum CliqueHamiltonian CycleVertex-CoverTraveling Salesman

34. 34 NP completeness of CLIQUE SAT reduces to CLIQUE –Given any input to SAT, we create a corresponding input to CLIQUE that will help us solve the original SAT problem –Specifically, for a SAT formula with K clauses, we construct a CLIQUE input that has a clique of size K if and only if the original Boolean formula is satisfiable

35. 35 NP completeness of CLIQUE SAT reduces to CLIQUE –Associate a person to each variable occurrence in each clause

36. 36 NP completeness of CLIQUE SAT reduces to CLIQUE –Associate a person to each variable occurrence in each clause –”Two people” know each other except if: they come from the same clause they represent t and t’ for some variable t

37. 37 NP completeness of CLIQUE SAT reduces to CLIQUE –Two people know each other except if: they come from the same clause they represent t and t’ for some variable t –Clique of size 4  satisfiable assignment set variable in clique to ”true” (x, y, z) = (true, true, false)

38. 38 NP completeness of CLIQUE SAT reduces to CLIQUE –Two people know each other except if: they come from the same clause they represent t and t’ for some variable t –Clique of size 4  satisfiable assignment –Satisfiable assignment  clique of size 4 (x, y, z) = (false, false, true) choose one true literal from each clause

39. 39 CLIQUE is NP-complete –CLIQUE is in NP –SAT is in NP-complete –SAT reduces to CLIQUE Hundreds of problems can be shown to be NP-complete that way…

40. 40 NP completeness of CLIQUE Show that the traveling salesman problem is NP-complete, assuming that the Hamiltonian cycle problem is NP-complete. –Prove that TSP  NP (already done) Guess: a sequence of cities Verify: that it is a tour of cost less than B –Reduce the undirected Hamiltonian cycle problem to the TSP So if we had a TSP-solver, we could use it to solve the Hamilitonian cycle problem in polynomial time

41. 41 HamCycle  p TSP? Transform input for HamCycle into input for TSP –HamCycle: Given unweighted graph G 1, does it have a Hamiltonian cycle? –TSP: Given weighted graph G 2 and B, is there a Hamiltonian cycle with total cost less than B? Must convert unweighted graph to weighted

42. 42 HamCycle  p TSP? Convert unweighted graph to weighted –Add edges to G 1 to make a complete graph G 2 –Add weights as follows: wt(i,j) is 0 if edge i,j is in G 1 (original graph for HamCycle) wt(i,j) is 1 if edge i,j is not in G 1 G 1 has ham. cycle iff G 2 has TSP with B=0 –Can you see why? –Is this transformation polynomial?

43. 43 The Main Question Is P = NP? –Is the original DECISION problem as easy as VERIFICATION? Most important open problem in theoretical computer science.

44. 44 The Big Picture Summarizing: it is not known whether NP problems are tractable or intractable But, there exist provably intractable problems –Even worse – there exist problems with running times unimaginably worse than exponential! More bad news: there are provably non- computable (undecidable) problems –There are no (and there will not ever be!) algorithms to solve these problems