1 Design and Analysis of Algorithms Yoram Moses Lecture 11 June 3,

2 Nondeterministic Polynomial Time (NP)

3 Shortest Path: Search, Existence, Verification Search problem:  Input: (G,w,s,t): a directed graph G with weight function w, a source s, and a sink t.  Goal: find a shortest path from s to t. (or reject if none exists)  Complexity: our solution runs in O(VE) = O(n 2 ) (Notice n = size of input = O(V+E)) Existence problem:  Input: (G,w,s,t,k): G,w,s,t are as before + a number k.  Goal: decide whether there is a path from s to t of length ≤ k.  Complexity: our solution runs in O(VE) = O(n 2 ) Verification problem:  Input: (G,w,s,t,k,p): G,w,s,t,k as before. p is a path in G.  Goal: decide whether p is a simple path from s to t of length ≤ k.  Complexity: O(V) = O(n).

4 Max Flow: Search, Existence, Verification Search problem:  Input: (G,c,s,t): a directed graph G with capacity function c, a source s, and a sink t.  Goal: find a maximum flow in G. (or reject if none exists)  Complexity: O(VE 2 ) = O(n 3 ) Existence problem:  Input: (G,c,s,t,k): G,c,s,t as before + a number k.  Goal: decide whether there is a flow in G with value ≥ k.  Complexity: O(VE 2 ) = O(n 3 ) Verification problem:  Input: (G,c,s,t,k,f): G,c,s,t,k as before. f is a function from edges of G to real numbers.  Goal: decide whether f is a legal flow with value ≥ k.  Complexity: O(E) = O(n).

5 Hamiltonian Cycle: Search, Existence, Verification Search problem:  Input: an undirected graph G.  Goal: find a Hamiltonian cycle in G (or reject if none exists).  Complexity: O(VxV!) = O(n2 n log n ) Existence problem:  Input: an undirected graph G.  Goal: decide whether G has a Hamiltonian cycle.  Complexity: O(VxV!) = O(n2 n log n ) Verification problem:  Input: (G,p): an undirected graph G and a sequence of nodes p.  Goal: decide whether p is a Hamiltonian cycle in G.  Complexity: O(V) = O(n).

6 3-Coloring: Search, Existence, Verification Search problem:  Input: G: an undirected graph  Goal: find a 3-Coloring of G. (or reject if none exists)  Complexity: O(E 3 V ) = O(n2 n log 3 ) Existence problem:  Input: G: as before.  Goal: decide whether G has a 3-Coloring.  Complexity: O(E 3 V ) = O(n2 n log 3 ) Verification problem:  Input: (G,  ): G as before and  : V  {1,2,3}.  Goal: decide whether  is a 3-Coloring of G.  Complexity: O(E) = O(n).

7 Search and Existence vs. Verification Conclusion: in many natural examples:  Search and existence are computationally equivalent  Verification is easier Sometimes it’s just a little easier (Shortest Path, Max flow) Sometimes it’s a lot easier (Hamiltonian cycle, 3-Coloring)

8 Verification Relations Language: L  {0,1} * Definition: A verification relation for L is a relation R  {0,1} *  {0,1} * s.t. for all x  {0,1} * :  x  L  there is at least one y  {0,1} * s.t. (x,y)  R.  x  L  there is no y  {0,1} * s.t. (x,y)  R. y is called the “certificate” for x  A.k.a. its “witness” or “proof” Remarks:  Every input x  L has at least one certificate y.  If (x,y)  R, then y is a certificate for x.  An input x  L may have several certificates.  A language L has many verification relations.

9 Verification Relations: Examples Shortest path:  x = (G,w,s,t,k), y = a path p  Language: {(G,w,s,t,k): G has an s-t path of length ≤ k}  Certificate: s-t path of length ≤ k  Verification relation: {((G,w,s,t,k),p): p is an s-t path of length ≤ k in G} Hamiltonian cycle:  x = undirected graph G, y = a path p  Language: G that has a Hamiltonian cycle  Certificate: a Hamiltonian cycle in G  Verification relation: {(G,p): p is a Hamiltonian cycle in G}

10 Nondeterministic Polynomial Time Definition: A binary relation R is polynomially bounded, if there exists some c > 0 s.t. for every (x,y)  R, |y| ≤ |x| c. Definition: L is polynomial-time verifiable, if it has a verification relation R, which satisfies both:  R is polynomially bounded, and  R is polynomial-time decidable. Definition: The class NP (Nondeterministic Polynomial Time) is the set of all polynomial-time verifiable languages.

11 NP: Examples Examples of languages in NP:  Decision Shortest Path, Decision Max Flow, Decision LP  Hamiltonian Cycle, TSP, 3-Coloring, SAT, k-SAT, Clique Examples of languages not known to be in NP:  HC-complement: given a graph G, decide whether G has no Hamiltonian cycles.

12 Definition: A nondeterministic algorithm is an algorithm N that, on input x,  First, N “nondeterministically” guesses a “witness” y.  Then, N runs a deterministic “verification” algorithm on (x,y).  Note: N may make different nondeterministic guesses in different runs on the same input x. Nondeterministic Algorithms Nondeterministic guess Nondeterministic Algorithm N Verification x y yes/no (x,y)(x,y)

13 Decision by Nondeterministic Algorithms Definition: A nondeterministic algorithm N is said to decide language L if:  For every input x  L, there is at least one guess y s.t. N accepts (x,y).  For every input x  L, the verification algorithm N rejects (x,y), for all guesses y. A polynomial-time nondeterministic algorithm is one in which  The guesses (y’s) are of polynomial size (in |x|), and  The verification algorithm runs in polynomial time. Lemma: L  NP iff L is decidable by a polynomial-time nondeterministic algorithm.

14 An NP Algorithm for Clique Nondeterministic guess (input: x = (G,k)) 1.for i = 1,…,k 2. v i  nondeterministic guess of a node in V=V(G) 3.output y = (v 1,…,v k ) Verification algorithm (input: (x,y)) 1.If x is not a valid encoding of a graph G and an integer k, reject. 2.If y is not a valid encoding of k nodes v 1,…,v k in G, reject. 3.If v 1,…,v k are not distinct, reject. 4.for i  1,…,k-1 do 5. for j  i+1,…,k do 6. if {v i,v j }  E reject. 7.accept

15 Lemma: P  NP Biggest open problem of computer science: is P = NP? Two possibilities: Current belief: P  NP  Search & Existence strictly harder than Verification. P = NP? P vs. NP P = NP P NP

16 f: N  N: a complexity measure. Time(f(n)) = all languages decidable in time O(f(n)). Lemma: Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n), then Time(f(n))  Time(g(n)). Theorem (Time Hierarchy) Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n) 1/2, then Time(f(n))  Time(g(n)). Time Hierarchy

17 P Definition: Lemma: P  EXP but P  EXP Lemma: NP  EXP (exercise) Open problem: is NP = EXP? 3 Possibilities: P, NP, and EXP P EXP NP P EXP = NP NP = P EXP

18 NP-Completeness (NPC) Problems in NP not known to be in P: Hamiltonian Cycle, Clique, SAT, k-SAT (k ≥ 3), k-Coloring (k ≥ 3), TSP, …. (many others) All of these are “NP-Complete” NP-Complete Problems:  Belong to NP  If any of them belongs to P, then NP = P. Two possibilities: NPC P NP NP = NPC = P

19 NP-Hardness (NPH) Definition: A language L is NP-hard if L ’ ≤ p L holds for all L ’  NP. NPH = class of all NP-hard problems. Lemma: If any NP-hard problem belongs to P, then NP = P.  If one NPH problem is easy, then all of NP is easy. Lemma: If L  NPH and L ≤ p L ’, then L ’  NPH.

20 NP-Completeness Definition: A language L is NP-complete if both  L  NP and  L is NP-hard NPC = class of NP-complete problems NPC = NP  NPH Theorem:  If some NPC language is in P, then P = NP. (P  NPC    NP = P = NPC).  If some NPC language is not in P, then no NPC language is in P. (NPC  P  P  NPC =   NP  P).

21 NP-Completeness NPC: “hardest” problems in NP Behave as a “single block”: either all in P or all outside P Lemma: If L 1,L 2  NPC, then both L 1 ≤ p L 2 and L 2 ≤ p L 1.

22 Proving NP-Completeness How to prove that a given language L is NPC?  Show that L  NP, and  Show that L ’ ≤ p L holds for every L ’  NP. Easier alternative:  Show that L  NP, and  Find some NPC problem L ’ and show L ’ ≤ p L. How do we obtain the first NPC problem?  Using the first alternative  Cook-Levin theorem: Circuit-SAT is NP-complete.

23 NP-Completeness: the Full Recipe To show that L is NPC:  Prove L is in NP Show a polynomial time nondeterministic algorithm for L  Select an NPC problem L ’  Show a polynomial-time reduction f from L ’ to L Prove that x  L ’ iff f(x)  L Show a polynomial-time algorithm to compute f

24 Example: Clique is NPC Clique is in NP (seen today) 3-SAT is NPC (will show this later on) 3-SAT ≤ p Clique (seen in previous lecture) Therefore: Clique is also NP-Complete!

25 End of Lecture 11