NP-completeness Class of hard problems

Jaruloj ChongstitvatanaNP-complete Problems2 Outline  Introduction  Problems and Languages Turing machines and algorithms P and NP  Satisfiability problem  Reduction  Other NP-complete problems

Jaruloj ChongstitvatanaNP-complete Problems3 Problems  A problem can be defined as a mapping from the instance to the answer.  Examples: Shortest path problem { }  {a shortest path P} 0/1 Knapsack problem { }  {the maximum value} Closest-pair-of-points problem { }  {, which is the closest pair of points in P}. Satisfiable problem { }  {T, F}

Jaruloj ChongstitvatanaNP-complete Problems4 Abstract problems A set of instances A set of solutions

Jaruloj ChongstitvatanaNP-complete Problems5 Decision Problems  A decision problem is a problem whose answer is either “yes” or “no”.  Example: Satisfiable problem { }  {T, F}  Any problem can be described in term of a decision problem.

Jaruloj ChongstitvatanaNP-complete Problems6 Reformulate Problems as Decision Problems  Shortest path problem { }  {T, F} The answer is T iff there is no path between u and v, which is shorter than k.  0/1 Knapsack problem { }  {T, F} The answer is T iff there is no selection of objects fitted in the sack which yields higher value than V.

Jaruloj ChongstitvatanaNP-complete Problems7 Encoding of Decision Problems  An encoding of a problem is a mapping from the set of instances of the problem to a set of strings.  Example: shortest path The encoding of in the Shortest path problem is a string which is used to describe G, u, v, and p.  G can be described by a string of its adjacency matrix.  u and v can be described by two strings, indicating the node labels.  p can be described by a sequence of node in the path.

Jaruloj ChongstitvatanaNP-complete Problems8 Abstract/ Concrete problems Mapping (encoding) Mapping (encoding) A set of instances A set of solutions Abstract problem Concrete problem

Jaruloj ChongstitvatanaNP-complete Problems9 Encoding of Problems  Is X a prime ? f:{X |X>0 and is a binary number}{Y,N} f(x) = Y if X is prime f(x) = N otherwise  Is a 3-CNF expression X satisfiable? f:{X | X is a 3-CNF expression}  {Y, N} f(x) = Y if X is satisfiable f(x) = N otherwise

Jaruloj ChongstitvatanaNP-complete Problems10 Why encoding is important?  Encoding determines the instance size.  Example: an integer n n encoded as one number => instance size = 1 n encoded as binary number => instance size = log 2 n n encoded as unary number => instance size = n

Problems and Languages

Jaruloj ChongstitvatanaNP-complete Problems12 Languages and Decision Problems  Let  be an alphabet.  A language over  is a set of strings created from symbols in . B={0,1}* is the set of all binary strings. { is a palindrome} is a language over B.  A yes-instance (or no-instance) is an instance whose answer is “yes” (or “no”).  An instance must be either a yes- or a no- instance.

Jaruloj ChongstitvatanaNP-complete Problems13 Languages and Decision Problems  A decision problem can be encoded in a language {| is an encoding of a yes- instance}. The shortest path problem can be encoded as {e(G, u, v, p>)| p is the shortest path between u and v in the graph G}.

Turing Machines

Jaruloj ChongstitvatanaNP-complete Problems15 Turing machines (TM)  A Turing machine is a machine which takes a string on its input tape, and decides whether to accept or to reject the string.  A Turing machine contains A finite-state control with  A start state  A halt state  Transition function: state x symbol  state x symbol x {R,L} A tape with a Read/Write head which can move left and right.

Jaruloj ChongstitvatanaNP-complete Problems16 Deterministic/ Nondeterministic Turing machines  Difference A transition is deterministic or nondeterministic.  What is nondeterminism? Given a state of a machine/program, there is more than one possible next step.  Examples Deterministic: functions. Nondeterministic: relations.

Jaruloj ChongstitvatanaNP-complete Problems17 Example of a DTM 1/,R q2 h q1 /,L s p1 p4 p2 p3 0/,R 0/0,R 1/1,R /,L 0/,L 0/0,L 1/1,L /,R

Jaruloj ChongstitvatanaNP-complete Problems18 How a DTM works 1/,R q2 h q1 /,L s p1 p4 p2 p3 0/,R 0/0,R 1/1,R /,L 0/,L 0/0,L 1/1,L /,R 01   1 On the input , the TM halts.

Jaruloj ChongstitvatanaNP-complete Problems19 How a DTM works 1/,R q2 h q1 /,L s p1 p4 p2 p3 0/,R 0/0,R 1/1,R /,L 0/,L 0/0,L 1/1,L /,R 00   On the input , the TM hangs.

Jaruloj ChongstitvatanaNP-complete Problems20 Example of NTM  Let L={ww| w{0,1} * } s p u q0q0 t0t0 r0r0 h 0/0,L 0/ ,L 1/ ,L 0/0,L 1/1,L /,L / ,R 0/0,R 1/1,R / ,R 0/ ,L 1/ ,L q1q1 t1t1 0/0,L 1/1,L /,L 0/0,R 1/1,R / v 0/0,R 1/1,R

Jaruloj ChongstitvatanaNP-complete Problems21 Accept/Decide  Let T be a Turing machine.  For a string  over , T accepts  iff T halts on  with output “1”. T rejects  iff T halts on  with output “0”. T hangs on  iff T dose not halts on .  For a language L over , T accepts L iff, for any string  in L, T accepts . T decides  iff  for any string  in L, T accepts , and  for any string  inL, T rejects .

Jaruloj ChongstitvatanaNP-complete Problems22 Time complexity  A language L is accepted/decided in O(f(n)) by a Turing machine T if, for any length-n string  in L,  is accepted/decided in k·f(n) by a Turing machine T, for some constant k.  A language L is accepted/decided in polynomial time by a Turing machine T if, L is accepted/decided in O(n k ) by a Turing machine T, for some constant k.

Jaruloj ChongstitvatanaNP-complete Problems23 Church-Turing Thesis  If there is a Turing machine deciding a problem  in O(f), then there is an algorithm solving  in O(f).  Turing machine  algorithm

Complexity classes

Jaruloj ChongstitvatanaNP-complete Problems25 Class of P  If  is a problem in P, given any instance I of , there is an algorithm which solves I in O(n k ), or there is a Turing machine which accepts e(I) in O(n k ) steps, where n is the size of e(I) and k is a constant.  What does this mean? If  is in P,  is easy because it can be solved in polynomial time.

Jaruloj ChongstitvatanaNP-complete Problems26 Class of NP  If  is a problem in P, given any instance I and any certificate C of , there is an algorithm which verifies, in O(n k ), that C is the answer of I, for any C and I. there is a Turing machine which verifies, in O(n k ), that C is the answer of I, for any C and I. there is a nondeterministic Turing machine which accepts e(I), in O(n k ), for any I. where n is the size of e(I) and k is a constant. What does this mean? If  is in NP,  is not necessarily easy because it can only be verified in polynomial time.

Jaruloj ChongstitvatanaNP-complete Problems27 Easy problems / Hard problems  Can be solved in polynomial time Prove by showing the algorithm.  Cannot be solved in polynomial time Prove by showing that there is no polynomial- time algorithm.  Can verify the answer in polynomial time.  Can usually be solved by an exponential-time algorithm. Brute-force algorithm: try every possible answers.

Jaruloj ChongstitvatanaNP-complete Problems28 Class of co-NP  L is in co-NP ifL is in NP. NPco-NP P

Reduction Is X more difficult than Y?

Jaruloj ChongstitvatanaNP-complete Problems30 Reduction  Let L 1 and L 2 be languages over  1 and  2, respectively. L 1 is (polynomial-time) reducible to L 2, denoted by L 1 L 2, if there is a TM M computing a function f:  1 *  2 * such that wL 1  f(w)L 2 in polynomial time.  Let P 1 and P 2 be problems. P 1 is (polynomial-time) reducible to P 2, denoted by P 1 P 2, if there is an algorithm computing a function f:  1 *  2 * such that w is a yes-instance of P 1  f(w) is a yes-instance of P 2 in polynomial time.

Jaruloj ChongstitvatanaNP-complete Problems31 Meaning of Reduction P 1 is reducible to P 2 if  DTM computing, in polynomial time, a function f:  1 *  2 * such that w is a yes-instance of P 1  f(w) is a yes-instance of P 2.  If you can map yes-instances of problem A to yes- instances of problem B, then we can solve A in polynomial time if we can solve B in polynomial time. it doesn’t mean we can solve B in polynomial time if we can solve A in polynomial time.

Example of Reductuion

Jaruloj ChongstitvatanaNP-complete Problems33 Satisfiability problem (SAT)  Given a Boolean expression , is  satisfiable?  (A~BC)(~A~C)B ~C is not satisfiable. ABC A~BC~A~C

Jaruloj ChongstitvatanaNP-complete Problems34 3CNF-SAT Problem  Given an expression  in 3CNF, is  satisfiable?  Given an expression  in 3CNF, is there a truth assignment of propositions in  which makes  true?  Example: =(PQR)(PQR)(PSR)(PSR)  is satisfiable when  P=t, Q=f, R=f, S=t or  P=f, Q=f, R=f, S=t

Jaruloj ChongstitvatanaNP-complete Problems35 Clique or Complete Subgraph  A graph G=(V,E) has a clique if there is W V such that for all nodes a and b in W, there is an edge between a and b. A B C D E F

Jaruloj ChongstitvatanaNP-complete Problems36 Clique Problem  Given a graphG=(V,E) and an integer k, is there a clique of size k in G?  Example: Is there a clique of size 3, 4, or 5, in the graph below? A B C D E F

Jaruloj ChongstitvatanaNP-complete Problems37 3CNF-SAT  Clique Proof: To prove that 3CNF-SAT  Clique, Find a function f such that  a 3CNF expression  is satisfiable  a graph G has a clique of size k, where =f() and  there is a TM computing f in polynomial time. f can be defined as follows: Let = ( 11  12  13 )( 21  22  23 )…( n1  n2  n3 ) f() =, where G=(V, E),  V={ ij | 0<i<n+1, 0<j<4}, and  E={( ij,  km )| i ≠ k and  ij ≠ ~ km }

Jaruloj ChongstitvatanaNP-complete Problems38 Building a graph from a 3CNF exp. (P  Q  R)  (P  S  R)  (  P  S  R) PR ~Q P S ~R ~P ~R ~S

Jaruloj ChongstitvatanaNP-complete Problems39 3CNF-SAT  Clique (cont’d)  Show  is satisfiableG has clique of size n  Let G has a clique of size n.  From the definition of G, there is no edge between vertices in the same disjunctive clause, and representing a literal and its negation.  A link between two nodes means that the two literals can be true at the same time, and they are in different disjunctive clauses.

Jaruloj ChongstitvatanaNP-complete Problems40 Satisfiability and Clique (P  Q  R)  (P  S  R)  (  P  S  R) PR ~Q P S ~R ~P ~R ~S

Jaruloj ChongstitvatanaNP-complete Problems41 Satisfiability and Clique (P  Q  R)  (P  Q  R)  (  P  Q  R)  (  P  Q  R) PR Q P Q ~R ~P R Q ~R Q

Jaruloj ChongstitvatanaNP-complete Problems42 Satisfiability and Clique (P  Q  R)  (P  Q  R)  (  P  Q  R)  (  P  Q  R) PR Q P Q ~R ~P R Q ~R Q

Jaruloj ChongstitvatanaNP-complete Problems43 3CNF-SAT  Clique (cont’d)  There is a TM computing f in polynomial time.

Properties of Reductions

Jaruloj ChongstitvatanaNP-complete Problems45 Reflexivity of reduction Theorem: Let L be a language over . L  L. Proof: Let L be a language over . Let f be an identity function from  *  *. Then, there is a TM computing f. Because f is an identity function, wL  f(w)=wL. By the definition, L  L.

Jaruloj ChongstitvatanaNP-complete Problems46 Property of reduction Theorem: Let L 1 and L 2 be languages over . If L 1 L 2, thenL 1 L 2. Proof: Let L 1 and L 2 be languages over . B ecause L 1 L 2, t here are a function f such that wL 1  f(w)L 2, and a TM T computing f in polynomial time. wL 1  f(w)L 2. By the definition,L 1 L 2.

Jaruloj ChongstitvatanaNP-complete Problems47 Transitivity of reduction Theorem:Let L 1, L 2 and L 3 be languages over . If L 1  L 2 and L 2  L 3, then L 1  L 3. Proof: Let L 1, L 2 and L 3 be languages over . T here is a function f such that wL 1  f(w)L 2, and a TM T1 computing f in polynomial time because L 1  L 2. T here is a function g such that wL 2  g(w)L 3, and a TM T2 computing g in polynomial time because L 2  L 3. wL 1  f(w)L 2  g(f(w))L 3, and T1T2 computes g(f(w)) in polynomial time. By the definition, L 1  L 3.

Jaruloj ChongstitvatanaNP-complete Problems48 Using reduction to prove P/NP Theorem: If L 2 is in P/NP, and L 1 L 2, then L 1 is also P/NP. Proof: Let L 1 and L 2 be languages over , L 1 L 2, and L 2 be in P/NP. Because L 2 is in P/NP, there is a DTM/NTM T 2 accepting L 2 in polynomial time. Because L 1 L 2, there is a DTM T 1 computing a function f such that wL 1  f(w)L 2 in polynomial time. Construct a DTM/NTM T= T 1 T 2. We show that T accepts L 1 in polynomial time. If wL 1, T 1 in T computes f(w)L 2 and T 2 in T accepts f(w), in polynomial time. If wL 1, T 1 in T computes f(w) L 2 and T 2 in T does not accept f(w) in polynomial time. Thus, L 1 is also in P/NP.

Jaruloj ChongstitvatanaNP-complete Problems49 Using reduction to prove co-NP. Theorem: If L 2 is in co-NP, and L 1 L 2, then L 1 is also in co-NP. Proof: Let L 1 and L 2 be languages over , L 1 L 2, and L 2 be in co-NP. Because L 2 is in co-NP,L 2 is in NP. Because L 1 L 2,L 1 L 2. Then,L 1 is in NP. Thus, L 1 is co-NP.

NP-completeness

Jaruloj ChongstitvatanaNP-complete Problems51 NP-completeness  A language (or problem) L is NP-hard if, for any language L' in NP, L' ≤ L.  A language (or problem) L is NP-complete if it is in NP and is NP-hard.

Jaruloj ChongstitvatanaNP-complete Problems52 Why interested in NP-complete problem  Implication on the problem “is P=NP ?”  If an NP-complete problem is in P, then P=NP.  Why? If L is NP-complete, any language in NP is reducible to L. Since L is in P, any language in NP is reducible to a language in P. Then, any language in NP is in P.

Jaruloj ChongstitvatanaNP-complete Problems53 Lemma If L is a language such that L’ ≤ L for some L’ ∈ NPC, then L is NP-hard. Moreover, if L ∈ NP, then L ∈ NPC. Proof Since L’ is NP-complete, for all L’’ ∈ NP, we have L’’ ≤ L’. Because L’ ≤ L, L’’ ≤ L by transitivity. Then, L is NP-hard. If L ∈ NP, we also have L ∈ NPC.

Ground Reduction SAT is NP-complete

Jaruloj ChongstitvatanaNP-complete Problems55 SAT is NP-complete  To prove that SAT is NP-complete: Prove SAT is in NP Prove any language in NP is reducible to SAT.  SAT is in NP: Informal proof Given a certificate which is a truth assignment of propositions which makes the expression true, there is an algorithm that can verify the answer in polynomial time.

Jaruloj ChongstitvatanaNP-complete Problems56 SAT is NP-complete (cont’d) Prove any language in NP is reducible to SAT. Proof: Let L be any language in NP. Then, there is an NTM T which accepts L in polynomial time p(n), where n is the input size. To show that L ≤ SAT, we find a function f such that x L  f(x)  SAT. That is, to find a function f such that x L  f(x) is satisfiable.

Jaruloj ChongstitvatanaNP-complete Problems57 SAT is NP-complete (cont’d) Proof (cont’d): That is, to find a function f such that T accepts x  f (x) is satisfiable. Let f (x)=g 1 (x)g 2 (x)g 3 (x)g 4 (x)g 5 (x)g 6 (x)g 7 (x). f (x) describes the sequence of configuration changes which leads to the halt state of T. f(x) contains propositions Q i,j, H i,k, and S i,k,j. Q i,j : T is in state q j after i moves. H i,k : T’s tape head is on square k after i moves. S i,k,j : Symbol  j is on square k after i moves.

Jaruloj ChongstitvatanaNP-complete Problems58 SAT is NP-complete (cont’d) Proof (cont’d): g 1 (x): the initial configuration of T at the beginning. g 2 (x): accepting configuration of T after N steps. g 3 (x): possible transitions of T. g 4 (x): hanging configuration. g 5 (x): describe that T is in exactly one state at a time. g 6 (x): describe that each square of the tape contains exactly one symbol. g 7 (x): describe that change can only made on the tape square on which the tape head is located.

Jaruloj ChongstitvatanaNP-complete Problems59 SAT is NP-complete (cont’d) Proof (cont’d): g 1 (x): describe the initial configuration of T at the beginning. n N Q 0,0  H 0,0  S 0,0,0   S 0,k, i k   S 0,k, 0 k=1 k=n+1 ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ q0q0

Jaruloj ChongstitvatanaNP-complete Problems60 SAT is NP-complete (cont’d) Proof (cont’d): g 2 (x): describe accepting configuration of T after N steps. Q N,a haha

Jaruloj ChongstitvatanaNP-complete Problems61 SAT is NP-complete (cont’d) Proof (cont’d): g 3 (x): describe possible transitions of T. For each transition δ(q j,  l )= (q t1,  z1, D 1 ), δ(q j,  l )= (q t2,  z2, D 2 ) (Q i,j H i,k S i,k,l )((Q i+1,t1  H i+1,d1(k)  S i+1,k,z1 ) V (Q i+1,t2  H i+1,d2(k)  S i+1,k,z2 ))

Jaruloj ChongstitvatanaNP-complete Problems62 SAT is NP-complete (cont’d) Proof (cont’d): g 4 (x): describe hanging configuration. For each transition δ(q j,  l )= ø (Q i,j  H i,k  S i,k,l )  (Q i+1,j  H i+1,k  S i+1,k,l )

Jaruloj ChongstitvatanaNP-complete Problems63 SAT is NP-complete (cont’d) Proof (cont’d): g 5 (x): T is in exactly one state at a time. For each time i and states q j0, q j1, q j2, …, q j0 ≠ q j1 ≠ q j2 ≠ … : (Q i,j0 v Q i,j1 v Q i,j2 v …) At least one state (~Q i,j0 v~Q i,j1 )(~Q i,j1 v~Q i,j2 )(~Q i,j0 v~Q i,j2 )… Never two state at the same time

Jaruloj ChongstitvatanaNP-complete Problems64 SAT is NP-complete (cont’d) Proof (cont’d): g 6 (x): state that each square of the tape contains exactly one symbol. For each time i, square k, and symbols  0,  1,  2, …,  0 ≠  1 ≠  2 ≠ … : (S i,k,0 v S i,k,1 v S i,k,2 v …) At least one symbol in a cell (~S i,k,0 v~S i,k,1 )  (~S i,k,1 v~S i,k,2 )  (~S i,k,0 v~S i,k,2 ) … Never two symbols in a cell at the same time

Jaruloj ChongstitvatanaNP-complete Problems65 SAT is NP-complete (cont’d) Proof (cont’d): g 7 (x): state that change can only made on the tape square on which the tape head is located. For each i, k, and l : (~H i,k  S i,k,l )  (S i+1,k,l ) H i,k v ~S i,k,l v S i+1,k,l

Jaruloj ChongstitvatanaNP-complete Problems66 SAT is NP-complete (cont’d) Proof (cont’d): From the definition of f, f(x) is true iff T accepts x. f(x) can be computed in polynomial time. Thus, SAT is NP-complete.

Jaruloj ChongstitvatanaNP-complete Problems67 3CNF-SAT is NP-complete First, show 3CNF-SAT is in NP. Let  be a 3CNF expression, and truth assignments C be a certificate for . There is an algorithm to verify that  is true under the assignments C, by substituting the value of each literal and evaluating the expression. This algorithm takes polynomial time of the length of .

Jaruloj ChongstitvatanaNP-complete Problems68 3CNF-SAT is NP-complete Second, show SAT ≤ 3CNF-SAT. Let  be a Boolean expression. We will construct a 3CNF expression f() such that f() is satisfiable iff  is satisfiable. To create f(): (A) Create an abstract syntax tree of  so that internal nodes are logical operators , v, ~, , or , and leaf nodes are literals (x or ~x). Then, create , from the abstract syntax tree which is satisfiable iff  is satisfiable.

Jaruloj ChongstitvatanaNP-complete Problems69 3CNF-SAT is NP-complete  = ((p  q)  (((~s v r)  (q  ~r)) v ~s) = y1  (y1  (y2  y3))  (y2  (p  q ))  (y3  (y4 v ~s))  (y4  (y5  y6))  (y5  (~s v r ))  (y6  (q  ~r))  is satisfiable iff  is satisfiable. :y1  :y2 v:y5 v:y3  :y4 :y6 pq q~r~sr

Jaruloj ChongstitvatanaNP-complete Problems70 3CNF-SAT is NP-complete (B) Convert  into CNF. P  Q is transformed to (P  Q)  (Q  P). P  Q is transformed to ~P v Q. ~(P v Q) is transformed to ~P  ~Q. ~(P  Q) is transformed to ~P v ~Q.  = (y1  (y2  y3)) =(y1  (y2  y3))  ((y2  y3)  y1) =(~y1 v (y2  y3))  (~y2 v ~y3 v y1) = (~y1 v y2)  (~y1 v y3)  (y1 v ~y2 v ~y3) = (~y1 v y2 v y3)  (~y1 v y2 v ~y3)  (~y1 v y2 v y3)  (~y1 v ~y2 v y3)  (y1 v ~y2 v ~y3)   

Jaruloj ChongstitvatanaNP-complete Problems71 3CNF-SAT is NP-complete f() = , and f() is satisfiable iff  is satisfiable. Next, we need to show that f is computable in polynomial time. In step (A): the abstract syntax tree can be created in polynomial time of the length of , the expression  can be created also in polynomial time of the number of node in the tree and the length of , which is also a polynomial time of the length of . In step (B):  can be created, using Boolean logic identities, in polynomial time of the length of , which is also a polynomial time of the length of . That is, SAT ≤ 3CNF-SAT.

Jaruloj ChongstitvatanaNP-complete Problems72 CLIQUE is NP-complete Proof: 3CNF-SAT is NP-complete. 3CNF-SAT ≤ CLIQUE. CLIQUE is NP-complete.