2. 1 NPC NP-Complete Problems

3. 2 We can solve any problem(!)(?) … but perhaps not with a computer! Some problems are not computable: The Halting Problem: Write a function boolean halts(Program p, Input i); which returns true if p halts on input i, and false if it doesn ’ t. A bonus homework exercise?

4. 3 If we could write the halts function, then we could also write: boolean loopIfHalts(Program p, Input i) { if (halts(p,i)) while (true) ; else return true; } boolean testSelf(Program p) { return loopIfHalts(p,p); } What is testSelf(testSelf) ?

5. 4 Proof by contradiction: boolean halts(Program p,Input i); which returns true if p halts on input i, and false if it doesn ’ t. boolean loopIfHalts(Program p, Input i) { if (halts(p,i)) while (true) ; else return true; } boolean tS(Program p) { return loopIfHalts(p,p); } Assume: tS(tS) halts and returns true. loopIfHalts(tS,tS) = true halts(tS,tS) = false tS(tS) doesn't halt which is a contradiction Assume: tS(tS) does not halt halts(tS,tS) = true contradiction

6. 5 We can solve any computable problem! … but perhaps not with an “ efficient ” algorithm. Of course we could solve any problem by: –Enumerating all possible solutions; –Testing each one to see if it is a solution. That may take a long time … … but sometimes it seems that ’ s the best we can do!

7. 6 Circuit Satisfiability: Find an assignment of truth values to the inputs of a logic circuit, that results in an output of 1. x1 x2 x3

8. 7 Hamiltonian Cycles: Given a graph G, find a simple cycle that contains each vertex in G.

9. 8 Subset Sum: Given a finite set of natural numbers S and a target natural number t, find a subset S ’ of S whose sum is t. For example, find: S ’  {1,3,6,34,93,243,654,1040,1436,1879} with sum S ’ = 2361.

10. 9 What is a problem? Abstractly, a problem is a set of pairs (I,S), where I is an instance of the problem and S is a solution. A decision problem is a problem in which the solutions are either yes or no. (i.e., 0 or 1) An optimization problem O is a subset of a problem P in which, if (I,S)  O and (I,S ’ )  P, then S  S ’, with respect to some ordering . (I.e. the solution S is better than S')

11. 10 Decision Problems: To solve a decision problem on a computer: –Provide the computer with an instance; –Wait for a yes/no answer. In most cases, we will need to encode the instance in a form that the computer can manipulate. We will assume that instances are coded as strings of bits: i.e., by elements of {0,1}*

12. 11 Encodings, e Abstract Problems Concrete Problems Is G a connected graph? Does e(G) represent a connected graph? General picture: For example:

13. 12 Problem Solving: An algorithm solves a problem in time O(T(n)) if, for any instance I, the algorithm can produce a solution S in time O(T(|I|)). A problem is polynomial-time solvable if there is an algorithm to solve it in time O(n k ), for some constant k.

14. 13 Polynomial time = Tractable? The computable polynomial-time algorithms that we encounter in practice usually have relatively low values of k. The set of polynomial-time problems is the same in many reasonable models of computation. The set of polynomial-time algorithms has nice closure properties.

15. 14 The set of concrete decision problems that are solvable in polynomial time.

16. 15 Polynomial Time Solvability for Abstract Algorithms: We would like to extend these ideas to abstract algorithms: An abstract decision problem Q is solvable in polynomial time if the encoding e(Q) is in P. This depends on the encoding used! Encodings, e Abstract Problems Concrete Problems

17. 16 Not all encodings are equal! Suppose we have A(k) =  (k) –for example: A(k) = 1+2+3+... +k If we code the integer k in binary notation, then we require n = log k bits of input. If we code an integer k in unary notation, then we require n = k+1 bits of input. Then algorithm A will have: –Exponential complexity,  (2 n ), if we use the binary representation; –Linear complexity,  (n), if we use the unary representation.

18. 17 This seems counter intuitive: The "better" the encoding the worse the complexity! In reality the same work is happening in both cases, we just inflated the size of the input (in the unary encoding) so the complexity appears better. Moral of the Story: If comparing two Abstract decision problems be sure the two encodings you are using are comparable!

19. 18 Relating Encodings: A function f : {0,1}*  {0,1}* is polynomial- time computable if there is an algorithm that, given any input x, produces f(x) as output in polynomial time. Encodings e 1 and e 2 are polynomially related if there are polynomial-time computable functions f and g mapping between them. {0,1}* Instances e1e1 e2e2 f g

20. 19 If e 1 and e 2 are polynomially related encodings on the instances of an abstract decision problem Q, then: e 1 (Q)  P  e 2 (Q)  P So, as long as we stick to reasonable, concise encodings, the exact encoding doesn ’ t alter our ability to solve a problem in polynomial time.

21. 20 Accepting and Deciding: Given a string x  {0,1}*, an algorithm A: –accepts x, if it outputs 1 given input x. –rejects x, if it outputs 0 given input x. –decides x, if it either accepts or rejects x. –is there another choice? A language is a set of strings x  {0,1}*. The set of strings x  {0,1}* for which an algorithm A outputs 1 is called the language accepted by A. Similar definitions for the languages rejected or decided by an algorithm …

22. 21 Languages correspond to concrete decision problems: –The language for a problem Q is just the set of strings that have solution 1. In this setting, P is the set of all problems that can be decided in polynomial time.

23. 22 This is the same as the set of languages that can be accepted in polynomial-time! –Suppose L can be accepted in polynomial time by an algorithm A. Then A will accept L after at most cn k steps for some constants c, k. –To decide L in polynomial time, simulate cn k steps in the execution of A. (A polynomial time overhead.) If it hasn ’ t terminated with an answer after that many steps, then it won ’ t ever be accepted, and so we can return 0.

24. 23 Write down 5 problems that are in P: 1. 2. 3. 4. 5. How many decision problems can you think of that are not in P?

25. 24 Verifying Solutions: It is often much easier to verify that a given value is a solution of a problem, than it is to calculate a solution directly. A verification algorithm takes two inputs: –A string x, representing a problem instance; –A string y, called a certificate. Algorithm A verifies a language L if L is precisely the set of strings x  L for which there exists a certificate y such that A(x,y) returns 1.

26. 25 The set of concrete decision problems that are verifiable in polynomial time. NP stands for non-deterministic polynomial

27. 26 “ Nondeterministic Polynomial ” : If a decision problem is in NP, then we can calculate solutions for an instance x by: –Enumerating all possible answers y; –Verifying each one using A(x,y) until we find a solution. If there are N possible solutions, this will take O(Nn k ) time. But often N is O(c n ), and so the overall time is O(c n ) too (since c n grows much faster than n k as n goes to infinity). The best case time is O(n k ) --- if the first answer we try is a solution.

28. 27 P  NP: If L is in P, then we can construct a polynomial-time verification algorithm A(x,y) that simply ignores the certificate y, and accepts precisely those strings that it determines to be in L. Is P=NP? After more than three decades, this is still an open question … but many researchers believe that they are not equal.

29. 28 Reductions

30. 29 Motivation Reductions are our main tool for comparison between problems.

31. 30 Introduction Objectives: –To formalize the notion of “reductions”. Overview: –Karp reductions –HAMPATH  p HAMCYCLE –Closeness under reductions –Cook reductions –Completeness

32. 31 Reductions – Concept A pp B IF we can translate problem A to problem B… THEN B is at least as hard as A.

33. 32 Reductions – Formal Definition Definition: Language A is polynomial time reducible to language B if… I.e – there exists a polynomial time TM which halts with f(w) on its tape when given input w. where for every w, w  A  f(w)  B f is called the polynomial time reduction of A to B. Denote: A  p B a polynomial time computable function f:  *  * exists, SIP 250

34. 33 Reductions and Algorithms ** A  * - A f ** B  * - B

35. 34 Reductions and Algorithms f polynomial time algorithm for A polynomial time algorithm for B w f(w)  B ? f(w) wA?wA?

36. 35 How to Reduce? 1.Construct f. 2.Show f is polynomial time computable. 3.Prove f is a reduction, i.e show: 1.If w  A then f(w)  B 2.If f(w)  B then w  A

37. 36 Terminology The type of reductions we’ve just described is also called: –Polynomial-time mapping reduction –Polynomial-time many-one reduction –Polynomial-time Karp reduction We’ll always refer to such reductions unless otherwise specified.

38. 37 Example To demonstrate a reduction, we’ll revisit the example we gave in the introduction. We’ll rephrase and formalize the seating and tour problems And demonstrate a slightly different reduction between them.

39. 38 Hamiltonian Path Instance: a directed graph G=(V,E) and two vertices s  t  V. Problem: To decide if there exists a path from s to t, which goes through each node once. In the version presented in the introduction we asked for some path, without specifying the start and end vertices.

40. 39 Hamiltonian Cycle Instance: a directed graph G=(V,E). Problem: To decide if there exists a simple cycle in the graph which goes through each node exactly once.

41. 40 f( ) = HAMPATH  p HAMCYCLE pp f( ) = s t s t n

42. 41 Correctness (Completeness) If there exists a Hamiltonian path (v 0 =s,v 1,…,v n =t) in the original graph, then (u,v 0 =s,v 1,…,v n =t,u) is a Hamiltonian cycle in the new graph.

43. 42 Correctness (Soundness) (u,s) and (t,u) must be in any Hamiltonian cycle in the constructed graph, thus removing u yields a Hamiltonian path from s to t.

44. 43 How to Reduce? 1.Construct f. 2.Show f is polynomial time computable. 3.Prove f is a reduction, i.e show: 1.If w  HAMPATH then f(w)  HAMCYCLE 2.If f(w)  HAMCYCLE then w  HAMPATH easy to verify

45. 44 Closeness Under Reductions Definition: A complexity class C is closed under reductions if, whenever L is reducible to L’ and L’  C, then L is also in C.  L L’ C

46. 45 Observation Theorem: P, NP, PSPACE and EXPTIME are closed under polynomial-time Karp reductions. Proof: Filling in the exact details is left to the reader. (See sketch)See sketch

47. 46 Other Types of Reductions Cook Reduction: Use an efficient “black box” (procedure) that decides B, to construct a polynomial-time machine which decides A. polynomial time algorithm for A polynomial time algorithm for B

48. 47 Cook reduction of HAMCYCLE to HAMPATH. For each edge (u,v)  E, –if there’s a Hamiltonian path from v to u in the graph where (u,v) is removed, output ‘YES’, exit Output ‘NO’ HAMPATH oracle Make sure it’s efficient!

49. 48 Reducibility: Problem Q can be reduced to problem Q ’ if every instance of Q can be rephrased as an instance of Q ’. A language L 1 is polynomial-time reducible to a language L 2, written L 1  P L 2, if there is a polynomial-time computable function f such that: x  L 1  f(x)  L 2 f is a reduction function; an algorithm that computes f is a reduction algorithm.

50. 49 If L 1  P L 2 and L 2 is in P, then so is L 1. F xf(x) Reduction algorithm A2A2 f(x)  L 2 Algorithm for deciding L 2 xL1xL1 Algorithm for deciding L 1

51. 50 NP-completeness: A language L is NP-complete if: –L is in NP; and –L ’  P L for every L ’ in NP. A language that satisfies just this property is said to be NP-hard.

52. 51 NP-completeness and P=NP: If any NP-complete problem is polynomial time solvable, then P=NP: –Suppose L  P is NP-complete. Then for any L ’  NP, we have L ’  P L, and hence L ’  P. If any problem in NP is not in P, then no NP- complete problem is in P. –Suppose L  NP and L  P. Now, if there is an L ’  P that is NP-complete, then L  P L ’, and hence L  P, which is a contradiction.

53. 52 To disprove P=NP: Just find a polynomial-time algorithm for an NP-complete problem … Fame and fortune will be yours... But nobody has found one yet …

54. 53 NPC Contains thousands of distinct problem  exponential algorithms  efficient algorithms ? each reducible to all others

55. 54 How Can Studying Complexity Make You a Millionaire? This is the most fundamental open question of computer science. Resolving it would grant the solver a great honor … as well as substantial fortune… www.claymath.org/prizeproblems/pvsnp.htm www.claymath.org/prizeproblems/pvsnp.htm

56. 55 P = NP  co-NP ? Any language in NP  co-NP - NP ? P = NP = co-NP NP = co-NP P co-NP NP P = NP  co-NP co-NP NP NP  co-NP P Four possible relationships between complexity classes

57. 56 Circuit Satisfiability is NP- complete: Of course, the first step is to find an NP-complete algorithm … We will show that circuit satisfiability is NP-complete, using the following two steps: –Prove that it is NP; –Prove that it is NP-hard.

58. 57 Circuit Satisfiability is in NP: Certificate supplies value at each input and output wire; We check the values at each gate; Return the value at the final gate’s output.

59. 58 We need to show that every language in NP is polynomial-time reducible to circuit-satisfiability. So suppose that L is in NP. We will describe a polynomial time reduction function f such that x  L if and only if f(x) is a circuit satisfiable problem. Circuit Satisfiability is NP-hard:

60. 59 Basic idea of proof Represent the computation A as a sequence of configurations and M is a combinational circuit that implement the mapping of one configuration to another. Configuration : the program, the program counter, working storage, machine states.

61. 60 There is an algorithm A that verifies L in polynomial time. Let T(n) = worst-case time for A on input strings of length n. Each step in execution of A can be represented as a transformation from one machine configuration to the next: Apcstatexy Apcstatexy M Logic circuit

62. 61 2. Every NP Problem ≤ P CIRCUIT-SAT (CIRCUIT-SAT is NP-hard) (2)

63. 62 C i --M--> C i+1 If A run at most T(n) steps the output appears in C T(n) Construct M that computes all configurations. F : given x computes C = f(x) that is satisfiable iff there exists a certificate y such that A(x,y) = 1. when F obtains x, it first computes n = |x| and construct C’ consists of T(n) copies of M.

64. 63 Proof 1.F correctly computes f. C is satisfiable iff there exists y such that A(x,y) = 1. 2.F runs in polynomial time. Part 1 if part Suppose there exists y, length O(n k ) such that A(x,y) = 1. Apply y to the input of C, the output of C is C(y) = A(x,y) = 1. Thus if a certificate exists then C is satisfiable.

65. 64 Part 1 only if Suppose C is satisfiable, hence there exists an input y to C such that C(y) = 1, from which we conclude that A(x,y) =1. Part 2 (A runs in polynomial time) The number of bits required to represent a configuration is polynomial in n (n = |x|). Program A has constant size

66. 65 The length of input x is n The length of the certificate y is O(n k ) The algorithm runs at most O(n k ) steps, the amount of working storage required by A is polynomial of n. The length of a configuration is polynomial in O(n k ) M has the size polynomial in the length of a configuration, hence is polynomial in n.

67. 66 The circuit C consists of at most t = O(n k ) copies of M, hence it has size in polynomial of n. The construction of C from x can be accomplished in polynomial time by the reduction algorithm F, since each step of construction takes polynomial time. Theorem 36.7 The circuit-satisfiability problem is NP-complete

68. 67 Formula Satisfiability (SAT)  =x 10  (x 4   x 3 )  (x 5  (x 1  x 2 ))  (x 6   x 4 )  (x 7  (x 1  x 2  x 4 ))  (x 8  (x 5  x 6 ))  (x 9  (x 6  x 7 ))  (x 10  (x 7  x 8  x 9 ))

69. 68 Dictionary literal: (negated or not) Boolean variable Examples: x,  x clause: several literals connected with  Example: (x  y  z) CNF (Conjunctive Normal Form): several clauses connected with  Example: (x  y)  (x  y  z) 3CNF: a CNF formula with three literals in each clause. Example: (x  y  z)  (x  y  z) negation: not (  ) conjunction: and (  ) disjunction: or (  )

70. 69 New Base Problems The only NP-Complete problem we currently know of is SAT. Unfortunately, it’s not very comfortable to work with. Thus we’ll start by introducing several useful variants of SAT. We’ll use them as our base problems.

71. 70 3SAT Instance: a 3CNF formula3CNF Problem: To decide if the formula is satisfiable. (x  y  z)  (x  y  z) (x  x  x)  (  x  x  x) A satifiable 3CNF formula An unsatifiable 3CNF formula

72. 71 3SAT is NP-Complete 3SAT is a special case of SAT, and is therefore clearly in NP. In order to show it’s also NP-Complete, we’ll alter the proof of SAT’s NP- Completeness, so it produces 3CNF formulas. Why would that be enough? SIP 259-260

73. 72 CNF  3CNF (x  y)  (x 1  x 2 ...  x t ) ... clauses with 1 or 2 literals (xyx)(xyx) replication clauses with more than 3 literals split (x 1  x 2  c 11 )  (  c 11  x 3  c 12 ) ...  (  c 1t-3  x t-1  x t )

74. 73 3SAT is NP-Complete Since we’ve shown a reduction from any NP problem to 3SAT, and 3SAT is in NP, 3SAT is NP-Complete.

75. 74 CLIQUE Instance: A graph G=(V,E) and a goal k. Problem: To decide if there is a set of nodes C={v 1,...,v k }  V, s.t for any u,v  C: (u,v)  E.

76. 75 CLIQUE is in NP On input G=(V,E),k: Guess C={v 1,...,v k }  V For any u,v  C: verify (u,v)  E Reject if one of the tests fail, accept otherwise. The length of the certificate: O(n) (n=|V|) Time complexity: O(n 2 )

77. 76 CLIQUE is NP-Complete Proof: We’ll show 3SAT  p CLIQUE. SIP 251-253

78. 77 The Reduction   for any clause (  )  connected iff   |V| = formula’s length K= no. of clauses

79. 78 Proof of Correctness 1...k1...k a clique of size k must contain one node from every layer. NOT connected!

80. 79 Correctness...... given a k-clique, an assignment which satisfies all the literals of the clique satisfies the formula given a satisfying assignment, a set comprising of one satisfied literal of each clause forms a k-clique.

81. 80 INDEPENDENT-SET Instance: A graph G=(V,E) and a goal k. Problem: To decide if there is a set of nodes I={v 1,...,v k }  V, s.t for any u,v  I: (u,v)  E.

82. 81 INDEPENDENT-SET  NP On input G=(V,E),k: Guess I={v 1,...,v k }  V For any u,v  C: verify (u,v)  E Reject if one of the tests fail, accept otherwise. The length of the certificate: O(n) (n=|V|) Time complexity: O(n 2 )

83. 82 INDEPENDENT-SET is NPC Proof: By the previous claim and a trivial reduction from CLIQUE. previous claim there’s a clique of size k in a graph there’s an IS of size k in its complement

84. 83 SUBSET-SUM Instance: A set of numbers denoted S and a target number t. Problem: To decide if there exists a subset Y  S, s.t  y  Y y=t.

85. 84 SUBSET-SUM is in NP On input S,t: Guess Y  S Accept iff  y  Y y=t. The length of the certificate: O(n) (n=|S|) Time complexity: O(n)

86. 85 SUBSET-SUM is NP- Complete Proof: We’ll show 3SAT  p SUBSET- SUM. SIP 269-271

87. 86 Examples SUBSET-SUM … because 2+8=10 … because 11 cannot be made out of {2,4,8}

88. 87 Reducing 3SAT to SubSet Sum Proof idea: Choosing the subset numbers from the set S corresponds to choosing the assignments of the variables in the 3CNF formula. The different digits of the sum correspond to the different clauses of the formula. If the target t is reached, a valid and satisfying assignment is found.

89. 88 10001001 10000100 1001100 1000010 101000 100011 10100 10001 1000 1000 100 100 10 10 1 1 + 11113333 +x1+x1 –x1–x1 +x2+x2 –x2–x2 +x3+x3 –x3–x3 +x4+x4 –x4–x4 1234 clause: Subset Sum 3CNF formula: Make the number table, and the ‘target sum’ t dummies

90. 89 Reducing 3SAT to SubSet Sum Let  3CNF with k clauses and variables x 1, …,x. Create a Subset-Sum instance by: 2 +2k elements of S  = {y 1,z 1, …,y,z,g 1,h 1, …,g k,h k } –y j indicates positive x j literals in clauses –z j indicates negated x j literals in clauses –g j and h j are dummies –and

91. 90 Subset Sum 10001001 10000100 1001100 1000010 101000 100011 10100 10001 1000 1000 100 100 10 10 1 1 11113333 +x1+x1 –x1–x1 +x2+x2 –x2–x2 +x3+x3 –x3–x3 +x4+x4 –x4–x4 Note 1: The “1111” in the target forces a proper assignment of the x i variables. Note 2: The target “3333” is only possible if each clause is satisfied. (The dummies can add maximally 2 extra.)

92. 91 Subset Sum is a satisfying assignment 10001001 10000100 1001100 1000010 101000 100011 10100 10001 1000 1000 100 100 10 10 1 1 11113333 10001001 1000010 100011 10100 1000 1000 100 100 10 1 11113333 +x1+x1 –x1–x1 +x2+x2 –x2–x2 +x3+x3 –x3–x3 +x4+x4 –x4–x4 +

93. 92 Subset Sum is not a satisfying assignment 10001001 10000100 1001100 1000010 101000 100011 10100 10001 1000 1000 100 100 10 10 1 1 11113333 10000100 1000010 100011 10100 1000 1000 ??? 11112??? +x1+x1 –x1–x1 +x2+x2 –x2–x2 +x3+x3 –x3–x3 +x4+x4 –x4–x4 +

94. 93 Proof 3SAT  P Subset Sum For every 3CNF , take target t=1 … 13 … 3 and the corresponding set S . If  3SAT, then the satisfying assignment defines a subset that reaches the target. Also, the target can only be obtained via a set that gives a satisfying assignment for .

95. 94 Finding the Solution If the decision problem Subset-Sum is solvable in polynomial time, then we can also find the subset that reaches the target in poly-time. How? By asking smart questions about several variants of the problem. That way, we can determine which variables x i are involved in the solution.

96. 95 Directed Hamiltonian Path Given a directed graph G, does there exist a Hamiltonian path, which visits all nodes exactly once? Two Examples: this graph has a Hamiltonian path but this graph has no Hamiltonian path

97. 96 3SAT to Hamiltonian Path Proof idea: Given a 3CNF , make a graph G  such that … a Hamiltonian path is possible if and only if there is a zig-zag path through G , … where the directions (zig or zag) determine the assignments (true or false) of the variables x 1, …,x L of .

98. 97 “ Zig-Zag Graphs ” s t There are 2 L different Hamiltonian paths from s to the target t. For every i, “Zig” means x i =True … while “Zag” means x i =False

99. 98 Accessing the Clauses If  has K clauses,  = C 1  C k, then x i has K “hubs”: stands for Idea: Make K extra nodes C j that can only be visited via the hub- path that defines an assignment of x i which satisfies the clause (using zig/zag = T/F).

100. 99 Connecting the Clauses Let the clause C j contain x i : If then x i =True=“zig” reaches node C j If then x i =False=“zag” reaches node C j

101. 100 Proof 3SAT  P HamPath Given a 3CNF  (x 1, …,x L ) with K clauses Make graph G  with a zig/zag levels for every x i Connect the clauses C j via the proper “ hubs ” If  SAT, then the satisfying assignment defines a Hamiltonian path, and vice versa.

102. 101 Example 4 variables… 4 clauses… Clauses connected via zig-zag “hubs”

103. 102 Example assignment… …satifies all four clauses; hence it defines a Hamiltonian Path

104. 103 Example assignment… …does not satify first clause; hence the path misses the C 1 node

105. 104 More on Hamiltonian Path Useful for proving NP-completeness of “ optimal routing problems ” Typical example: NP-completeness of “ Traveling Salesman Problem ” Issue of directed versus undirected graphs

106. 105 Forcing Directions Given a directed graph with s,x 1,…,x k,t Replace s with s out, the t with t in,and every x i with the triplet “x i,in —x i,mid — x i,out ” Redraw the original directed edges with edges going from out-nodes to in-nodes. If and only if the directed graph has a HamPath from s to t, then so has this graph.

107. 106 Example: Directions s y x s y x “Undirected HamPath” is NP-complete x in s out x mid x out becomes y mid y out y in x in s out x mid x out becomes y mid y out y in

108. 107 Minimizing Miles Given k cities, with distance graph G “ Is there a route that visits all k cities with less than t miles of traveling? ” The Traveling Salesman Problem is in NP. Close connection with Undirected-Ham- Path One can show: TSP is NP-complete

109. 108 CIRCUIT-SAT 3-CNF-SAT SAT TSP HAM-CYCLE CLIQUE VERTEX-COVER SUBSET-SUM

110. 109 Solving hard problems Relax the problem -- use approx. algo. Relax the method -- use probabilistic algo and give up total correctness Relax the architecture -- use parallel Relax the machine -- use analog computer ?