1. Theory of Computing Lecture 17 MAS 714 Hartmut Klauck

2. Nondeterministic Turing Machine A nondeterministic Turing machine is a 7-tuple (Q, ¡, b, §, q 0, A, ± ) Q: set of states of the machine ¡ : tape alphabet b 2 ¡ : blank symbol § µ ¡ -{b}: input alphabet q 0 2 Q: initial state A µ Q: accepting states ± : Q- A £ ¡  Subset of {Set of all {Q £ ¡ £ {left,stay,right}}: transition

3. Notes NTIME(f(n)) is the class of all languages that have nondeterministic time complexity at most O(f(n)) NP is the class of languages L such that the time complexity of L can be upper bounded by a polynomial in n

4. NP Guess and check definition: A language L is in NP, if there is a language R 2 P such that for all x: [There is a poly(|x|) length string y s.t. x#y 2 R], x 2 L y is called proof or witness

5. Relation to other classes P µ NP by definition NP µ EXP – enumerate all witnesses and check each Better: NP µ PSPACE – PSPACE: languages decidable by TM using polynomial space – enumerate all witnesses (re-use the same space)

6. Completeness We will identify a class of problems that capture the hardness of NP P=NP if and only if one of these problems is in P – None are known to be in P These are the problems L in NP, such that any other problem in NP can be solved by a deterministic polynomial time TM given a free subroutine that solves L

7. Completeness Informally, L reduces to S, if, given an `oracle’ that decides S for free we can compute L in polynomial time (deterministically) NP-complete: L is in NP and all S in NP reduce to L – L is hardest in NP

8. Cook-Reductions Named after Stephen Cook L reduces to S by a Cook-reduction, if there is an algorithm that solves L, using calls to a subroutine to S, and runs in polynomial time Formal definition uses a model of oracle Turing machine

9. Karp-Reductions We will use a simpler definition of reductions Also called ``many-one reductions’’ Definition: – Let L µ ¡ * and S µ § * – A Karp-reduction from L to S is a polynomial time computable function f that sends x 2 ¡ * to f(x) 2 § * such that x 2 L, f(x) 2 S Clearly: a polynomial time algorithm for S implies a polynomial time algorithm for L Notation: L · P S Vice versa: if L is not in P, then S is not in P

10. Definition: NP-complete Definition: a language L is NP-hard, if every problem S 2 NP reduces to L ( S · P L ) Definition: a language L is NP-complete, if L is NP- hard and L 2 NP We use Karp reductions. Cook reductions may result in a larger set of NP-complete problems (this is an open problem)

11. NP-completeness Theorem: P=NP iff there is an NP- complete problem in P Counterpositive: If P  NP then no NP- complete problem is in P

12. Proof Assume an NP-complete problem L is in P: – For any (other) problem S in NP we can perform the reduction and then decide L – This is a polytime algorithm for all S in NP P=NP: implies that all problems in NP are in P

13. How to identify a hard problem Given a language L, is it NP-complete? Find a known NP-complete problem S Reduce S to L Then L is hard under the assumption that S is hard How do we get our first NP-complete problem?

14. Finding an NP-complete Problem We will define the problem CircuitSAT and show how every problem in NP reduces to it Then we can show further NP-hardness results by reducing from CircuitSAT, or any other NP-hard problem we already know Important property of reductions: Lemma: If L · p S and S · p T then L · p T [Transitivity] Proof: exercise

15. Reducing from any L 2 NP Given L 2 NP we need to find a reduction to CircuitSAT L 2 NP ) there is a language S 2 P such that x 2 L iff 9 y: (x#y) 2 S – y is short and S has a polytime TM M – the length of y is known given x We will model the computation of M using Boolean circuits

16. Circuits A Boolean circuit is a directed acyclic graph where each inner node has in-degree 1 or 2. The sources are labeled by variables from {x 1,…,x n }. Inner nodes are labeled with the gate functions AND, OR, NOT Notation of AND: Æ OR: Ç NOT: : The sink nodes are the outputs of the circuit

17. Example

18. Computation of a circuit Inputs x 1,…,x n are instantiated to some Boolean values Value of the source nodes are the values of the corresponding variables Value of the inner nodes: – AND(x,y)=1 iff x=1 and y=1 – OR(x,y)=1 iff x=1 or y=1 – NOT(x)=1 iff x=0 Process nodes in topological order

19. Circuits A circuit computes a function f:{0,1} n  {0,1} k if the output gates are correct for every input A family of circuits is an infinite sequence C1,C 2, C 3,… of circuits, such that C n has n inputs A family computes a function f:{0,1}*  {0,1} if each C i computes f restricted to {0,1} n

20. Uniformity A family of circuits is uniform, if each circuit C n is easy to compute, given n – This means a TM can output a description of C n in time poly(n) Otherwise the circuit model is too powerful The size of a circuit family is the function that maps the input length n to the number of nodes of C n Note: P=class of Boolean functions computable by polynomial size uniform circuit families.

21. CircuitSAT Input: a Boolean circuit C with n inputs and 1 output and size m – Encoded suitably Output: accept if and only if there is an input x that is accepted by C Circuit Satisfiablity – circuit is satisfiable if there is an input that will make it accept/compute 1

22. Circuit simulating Turing Machine Theorem: – If M is a polytime Turing machine, and n is the fixed input length we consider, then there is a (uniform) circuit C n, that decides the same language as M on inputs of length n. The size of C n is polynomial in n

23. Proof Sketch 1.For simplicity we consider only TM with alphabet {0,1} (original symbols and blanks are encoded as strings ) 2.Change the Turing machine so it moves obliviously (left to right all the way and back etc.) 3.Construct a circuit for the transition function 4.`Glue’ copies of the transition circuit together into a circuit that simulates the TM

24. Pieces of the proof: A Turing machine is sweeping, if it moves (on every input) from the left end of the space used to the right end and back until it stops on the left end Lemma: Making M sweeping increases the running time from T(n) to O(T 2 (n)) Proof: Leave a marker for the head position on the tape. For each time step of M sweep the whole tape region used from left to right and back twice. Move the marker according to the computation of M.

25. Pieces of the proof: Every Boolean function {0,1} n  {0,1} can be computed by a Boolean circuit [Shannon] Then: the transition function can be computed by circuits Proof: exercise Note: most functions need exponential size circuits. Transition function is of constant size. Hence the circuits are of constant size.

26. Pieces of the proof: Gluing the circuits together: – We use T(n) 2 copies of the transition function circuit. Arrange them as a grid. – Copy i,j performs the computation of the transition function when the TM is on cell j of the tape during the i-th traversal of the tape – Inputs of copy i,j: tape symbol from copy i-1, state from the copy i, j-1 or i, j+1 – Outputs: tape symbol goes to copy i+1,j ; state goes to copy i,j+1 or i,j-1 – Special connections at j being left and right end of the tape

27. Conclusion: The size of the circuit is O(T(n) 2 ) The circuit is easy to construct from the TM, i.e., if we do this construction for all n we get a uniform polynomial size family of circuits

28. NP-hardness Theorem: CircuitSAT is NP-hard Take any L 2 NP Use the TM M for S 2 P from the guess and check definition of NP x 2 L iff x#y 2 S For x 2 {0,1} m find the circuit C n that simulates M on length n inputs (n=length of x and y combined) Fix the source nodes of C n reading x i variables to the actual values of these variables Now: x 2 L iff C n accepts some y (with x fixed) iff C n (with fixed x) is in CircuitSAT Karp reduction to CircuitSAT !

29. NP-complete Theorem: CircuitSAT is NP-complete Left to show: CircuitSAT 2 NP This is easy to see: Guess an input x, simulate the circuit C on a TM to check that C(x)=1 – exercise (circuit given as adjacency list)

30. Simplifying CircuitSAT A literal is a variable x i or its negation A clause is a term of the form (l 1 Ç l 2 Ç … l k ) A formula in conjunctive normal form (CNF) is a AND of m clauses (of any length) on n variables Special case of circuits CNF-SAT: set of all CNF formulae that are satisfiable (i.e., there is a setting of the variables that makes the formula accept)

31. CNF-SAT Theorem: – CNF-SAT is NP-complete Proof: Easy to see that CNF-SAT is in NP Hardness: reduction from CircuitSAT Let C be a circuit. We need to find a CNF formula F that is satisfiable iff C is satisfiable – Also F cannot be too long

32. The reduction Assume that C has s gates and n variables x 1,...,x n Assume that all negations are at the sources of C (why can we do this? De Morgan’s laws) Then F will have n+s variables x 1,…,x n and g 1,…,g s The CNF F is a conjunction (AND) of clauses There are 3 clauses for every gate of C Idea: g i guesses the value of gate i

33. The clauses Gate i is the AND of gates j and k: We want to express that g i = g j Æ g k Clauses: (g i Ç : g j Ç : g k ) AND ( : g i Ç g j ) AND ( : g i Ç g j ) Gate i is the OR of gates j and k: We want that g i =g j Ç g k Clauses: (g i Ç: g j ) AND (g i Ç: g k ) AND ( : g i Ç g j Ç g k ) Last clause: (g O ), the output gate

34. Correctness We need to show that C is satisfiable iff F is Assume C is satisfied by x Then the correct values of the gates make C accept Then these values (for the g-variables) together with x satisfy F Conversely, given the values of x and g variables that satisfy F, x must satisfy C Hence we have a Karp reduction (clearly F is of linear size in C)

35. Note Since in CircuitSAT all gates have in-degree 2 all clauses concern only 3 gates We have shown NP-completeness of 3-SAT: – a k-CNF is a CNF in which all clauses have 3 literals (or less) – k-SAT={set of all k-CNF that are satisfiable} Notes: Not hard to give a reduction from CNF-SAT to 3-SAT 2-SAT is in P (and hence not NP-complete unless P=NP) 3-SAT is the basis of our reductions from now on

36. Reductions We will show reductions from 3-SAT to a few problems Due to transitivity we can use reductions from any of the problems for which we already have a reduction from 3-SAT – Often this is easier than directly reducing from 3- SAT