CSC 3130: Automata theory and formal languages Andrej Bogdanov The Chinese University of Hong Kong NP-complete problems Fall 2009

Polynomial-time reductions Language L polynomial-time reduces to L’ if there exists a polynomial-time computable map R that takes an instance x of L into instance y of L’ s.t. x ∈ L if and only if y ∈ L’ LL’(CLIQUE)(IS) xy (G, k) (G’, k’) x ∈ Lx ∈ L y ∈ L’y ∈ L’ (G has clique of size k)(G’ has IS of size k’) R

The Cook-Levin Theorem Every L ∈ NP reduces to SAT SAT = {f: f is a satisfiable Boolean formula } (x 1 ∨ x 2 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) ∧ (x 1 ) f = is satisfiable x 1 = T x 2 = F x 3 = T x 4 = T (x 1 ∨ x 2 ) ∧ (x 1 ) ∧ (x 2 ) f = is not satisfiable

NP-hardness Language L is NP -hard if every L’ in NP reduces to L Intuitively, NP -hard means “harder than all of NP ” The Cook-Levin Theorem says SAT is NP -hard P SAT NP

NP-complete L is NP -complete if L is NP -hard and L ∈ NP Intuitively, NP -complete means “hardest in NP ” Recall that SAT ∈ NP, so SAT is NP -complete If SAT ∈ P, then P = NP P SAT NP

Proof of Cook-Levin Theorem To prove it, we have to describe a reduction R : Every L ∈ NP reduces to SAT w Boolean formula f w ∈ Lw ∈ L f is satisfiable R

Proof of Cook-Levin Theorem All we know about L : It has a poly-time NTM M –Let’s look at computation tableau of M on input w M w w1w1 q0q0 q acc … S- th configuration symbol at time T S T Since M is nondeterministic, there may be many possible tableaus # … # # # #

Proof of Cook-Levin Theorem S T n = length of input w height of tableau is p(n) for some polynomial p width is at most p(n) k possible tableau symbols x T, S, u = true,if the (T, S) cell of tableau contains u if not false, u 1 ≤ S ≤ p(n) 1 ≤ T ≤ p(n) 1 ≤ u ≤ k w1w1 q0q0 … ## q acc … # # #

Proof of Cook-Levin Theorem We will design a formula f such that: w Boolean formula f w ∈ Lw ∈ L f is satisfiable R variables of f : assignment to x T, S, u way to fill up the tableau satisfying assignment accepting computation tableau f is satisfiable x T, S, u M accepts w

Proof of Cook-Levin Theorem We want to construct (in time poly(n) ) a formula f : x T, S, u = true,if the (T, S) cell of tableau contains u if not false, 1 ≤ S ≤ p(n) 1 ≤ T ≤ p(n) 1 ≤ u ≤ k f(x 1, 1, 1,..., x p(n), p(n), k ) = true,if the x s represent a valid accepting tableau if not false, Symbols in computation tableau come from some alphabet {a 1,...,a k }

Proof of Cook-Levin Theorem f = f cell ∧ f 0 ∧ f move ∧ f acc f cell : “Every cell contains exactly one symbol” “The first row is #q 0 w 1 w 2...w k ☐... ☐ # ” f0:f0: f move : “The moves between rows follow the transitions of M ” f acc : “ q acc appears somewhere in the last row” S T u w1w1 q0q0 … ## q acc … # # #

Proof of Cook-Levin Theorem Desired meaning Implementation: f cell : “Every cell contains exactly one symbol” f cell = f cell 1, 1 ∧... ∧ f cell p(n), p(n) where f cell T, S = (x T, S, 1 ∨... ∨ x T, S, k ) ∧ (x T, S, 1 ∧ x T, S, 2 ) ∧ (x T, S, 1 ∧ x T, S, 3 ) ∧... ∧ (x T, S, k-1 ∧ x T, S, k ) at least one symbol no two symbols or: “Exactly one of x S, T, 1 ∨... ∨ x S, T, k is true”

Proof of Cook-Levin Theorem Desired meaning Implementation: f0:f0: “The first row is #q 0 w 1 w 2...w k ☐... ☐ # ” f acc : “ q acc appears somewhere in the last row” f 0 = x 1, 1, # ∧ x 1, 1, q0 ∧ x 1, 1, w1 ∧... ∧ x 1, p(n),# f acc = x p(n), 1, qacc ∨ x p(n), 2, qacc ∨... ∨ x p(n), p(n), qacc

Valid and invalid windows … 6 a 3 b 0 x 0 … … 0 a 6 b 0 x 0 … 0 valid windows … 6 # 3 b 0 a 0 … … 0 # 6 b 0 q 5 … 0 … 6 a 3 q 2 a 0 … … 0 q 5 a 6 x 0 … 0 … 6 q 2 a 0 b 0 … … 0 a 6 b 0 q 2 … 0 invalid windows … 6 a 3 a 0 ☐ 0 … … 0 x 6 a 0 ☐ 0 … 0 q2q2 q5q5 a/xL … 6 a 3 q 2 a 0 … … 0 q 5 a 6 b 0 … 0 … 6 q 2 q 2 a 0 … … 0 q 2 q 2 x 3 … 0 … 6 a 3 q 2 a 0 … … 0 a 6 q 5 x 0 … 0

Proof of Cook-Levin Theorem Desired meaning Implementation: f move : “The moves between rows follow transitions of M ” a q0q0 q acc … f move = f move 1, 1 ∧... ∧ f move p(n)-3, p(n)-3 b q3q3 a b q7q7 cb f move 2, 2 f move T, S = over all (x T, S, a1 ∧ x T, S+1, a2 ∧ x T, S+2, a3 ∧ x T+1, S, a4 ∧ x T+1, S+1, a5 ∧ x T+2, S+1, a6 ) ∨ valid windows a1a2a3 a4a5a6 # # # ## #

Other NP- complete problems CLIQUE = {(G, k): G is a graph with a clique of k vertices } IS = {(G, k): G is a graph with an independent set of k vertices } VC = {(G, k): G is a graph with a vertex cover of k vertices } CLIQUE, IS and VC are NP -complete SAT NP CLIQUEIS VC

Proving NP -hardness To show L is NP -hard, it is enough to reduce from some L’ we already know is NP -hard –For now we can take L’ = SAT To show L is NP -complete, we also need to argue that L is in NP –This is usually the easy part roadmap: SAT 3SAT IS CLIQUE VC

3SAT SAT = {f: f is a satisfiable Boolean formula } 3SAT = {f: f is a satisfiable Boolean formula in conjunctive normal form with 3 literals per clause } CNF: AND of ORs of literals literal: x i or x i (conjunctive normal form) 3CNF: CNF with 3 literals per clause (x 1 ∨ x 2 ∨ x 2 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) clause literals (x 2 ∨ (x 1 ∧ x 2 )) ∧ (x 1 ∧ (x 1 ∨ x 2 )) gates (repetitions are allowed)

NP-hardness of 3SAT Theorem Proof: We describe a reduction R from SAT Boolean formula f 3CNF formula f’ f’ is satisfiable R f is satisfiable 3SAT is NP -hard

Reducing SAT to 3SAT Example: f = (x 2 ∨ (x 1 ∧ x 2 )) ∧ (x 1 ∧ (x 1 ∨ x 2 )) We give extra variables to every gate (“wire”) x3x3 x6x6 x7x7 x8x8 x9x9 x4x4 x5x5 x 10 AND ORNOT AND NOTOR AND NOT x2x2 x1x1 x2x2 x1x1 x1x1 x2x2 x 7 = x 4 ∧ x 5 x 4 x 5 x 7 T T T T T T F F T F T F T F F T F T T F F T F T F F T F F F F T (x4∨x5∨x7)(x4∨x5∨x7) (x4∨x5∨x7)(x4∨x5∨x7) (x4∨x5∨x7)(x4∨x5∨x7) (x4∨x5∨x7)(x4∨x5∨x7) (x4∨x5∨x7)(x4∨x5∨x7) ∧(x4∨x5∨x7)∧(x4∨x5∨x7) ∧(x4∨x5∨x7)∧(x4∨x5∨x7) ∧(x4∨x5∨x7)∧(x4∨x5∨x7)

Turning gates into 3CNFs AND T T T T T T F F T F T F T F F T F T T F F T F T F F T F F F F T z = y ∧ x xy z xy z (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) OR T T T T T T F F T F T T T F F F F T T T F T F F F F T F F F F T z = y ∨ x xy z xy z (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) ∧ (x ∨ y ∨ z) NOT T T F T F T F T T F F F z = xx z x z (x∨z)∧(x∨z)(x∨z)∧(x∨z) (x∨x∨z)∧(x∨x∨z)(x∨x∨z)∧(x∨x∨z) Gj:Gj: fj:fj:

Reducing SAT to 3SAT Boolean formula f 3CNF formula f’ R f’ = f n+1 ∧ f n+2 ∧... ∧ f t ∧ (x n+t ∨ x n+t ∨ x n+t ) requires that output of f is true On input f, where f is a boolean formula Construct and output the following 3CNF formula f’: R:R: Add variable x n+j for each gate G j in f Write 3CNF f j for each gate G j, j = {1,..., t} Let

Reducing SAT to 3SAT Every satisfying assignment of f extends uniquely to a satisfying assignment of f’ Conversely, every satisfying assignment of f’ must contain a satisfying assignment of f Boolean formula f 3CNF formula f’ R f’ is satisfiable f is satisfiable

Clique Theorem CLIQUE = {(G, k): G is a graph with a clique of k vertices } A clique is a subset of vertices so that all pairs are connected {1, 2, 3}, {1, 4}, {4} are cliques CLIQUE is NP -hard ✓ SAT 3SAT IS CLIQUE VC

Reducing 3SAT to CLIQUE Proof: We give a reduction from 3SAT to CLIQUE 3SAT = {f: f is a satisfiable Boolean formula in 3CNF } CLIQUE = {(G, k): G is a graph with a clique of k vertices } 3CNF formula f (G, k) G has a clique of size k R f is satisfiable

Reducing 3SAT to CLIQUE Example: f = (x 1 ∨ x 1 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 3 ) Put an edge for every consistent pair x1x1 x1x1 x2x2 x1x1 x2x2 x2x2 x1x1 x2x2 x3x3 Put a vertex for every literal

Reducing 3SAT to CLIQUE On input f, where f is a 3CNF formula with m clauses Construct the following graph G: G has 3m vertices, divided into m groups, one for each literal in f 3CNF formula f (G, k) R R:R: If a and b are in different groups and a ≠ b, put an edge (a, b) Output (G, m)

Reducing 3SAT to CLIQUE 3CNF formula f (G, m) G has a clique of size m R f is satisfiable f = (x 1 ∨ x 1 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 3 ) x1x1 x1x1 x2x2 x1x1 x2x2 x2x2 x1x1 x2x2 x3x3 TTFFFTFFT

Reducing 3SAT to CLIQUE 3CNF formula f (G, m) G has a clique of size m R f is satisfiable f = (x 1 ∨ x 1 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 2 ) ∧ (x 1 ∨ x 2 ∨ x 3 ) x1x1 x1x1 x2x2 x1x1 x2x2 x2x2 x1x1 x2x2 x3x3 FTTFFTTTF

✓ SAT 3SAT IS CLIQUE VC Reducing 3SAT to CLIQUE Every satisfying assignment of f gives a clique of size m in G Conversely, every clique of size m in G gives a consistent satisfying assignment of f. 3CNF formula f (G, m) G has a clique of size m R f is satisfiable ✓ ✓

Vertex cover ✓ SAT 3SAT IS CLIQUE ✓ ✓ Theorem VC is NP -hard A vertex cover is a set of vertices that touches (covers) all edges {2, 4}, {3, 4}, {1, 2, 3} are vertex covers VC = {(G, k): G is a graph with a vertex cover of size k} VC

Reducing CLIQUE to VC Proof: We describe a reduction from IS to VC Example (G’, k’) G’ has a VC of size k’ R (G, k) G has an IS of size k ∅, {1}, {2}, {3}, {4}, {1, 2}, {1, 3} {2, 4}, {3, 4}, {1, 2, 3}, {1, 2, 4}, {1, 3, 4}, {2, 3, 4}, {1, 2, 3, 4} independent sets vertex covers

Reducing IS to VC Claim Proof S is an independent set of G if and only if S is a vertex cover of G S is an independent set of G no edge has both endpoints in S every edge has an endpoint in S S is a vertex cover of G ∅ {1} {2} {3} {4} {1, 2} {1, 3} {2, 4} {3, 4} {1, 2, 3} {1, 2, 4} {1, 3, 4} {2, 3, 4} {1, 2, 3, 4} IS VC

Reducing IS to VC (G’, k’) R (G, k) G has a VC of size n – kG has an IS of size k On input (G, k), R:R: Output (G, n – k). ✓ SAT 3SAT IS CLIQUE VC ✓ ✓ ✓

The ubiquity of NP-complete problems We saw a few examples of NP-complete problems, but there are many more A surprising fact of life is that most CS problems are either in P or NP- complete A 1979 book by Garey and Johnson lists 100+ NP -complete problems