Fall 2013 CMU CS Computational Complexity Lecture 2 Diagonalization, 9/12/2013

2 Languages Let ∑ be any finite alphabet. L µ ∑ * L is a language. The decision problem corresponding to L is the function: f:∑ * ! {yes, no} deciding membership in L, i.e., {x | f(x)=‘yes}

9/12/20133 Model for this class; Multi-tape Turing Machines Read-only Input tape; write only output... finite control abab aa bbcd... k tapes δ:Q x ∑ k ! Q x ∑ k x {L,R,-} k (input tape)

9/12/20134 Turing Machines Q finite set of states ∑ alphabet including blank: “_” q start, q accept, q reject in Q δ:Q x ∑ k ! Q x ∑ k x {L,R,-} k transition fn. input written on tape, head on 1 st square, state q start sequence of steps specified by δ if reach q accept or q reject then halt

9/12/20135 TIME and SPACE TIME(f(n)) = languages decidable by a multi- tape TM in at most f(n) steps, where n is the input length, and f :N ! N SPACE(f(n)) = languages decidable by a multi- tape TM that touches at most f(n) squares of its work tapes, where n is the input length, and f :N ! N

9/12/20136 Time and Space Classes L = SPACE(log n) PSPACE =  k SPACE(n k ) P =  k TIME(n k ) EXP =  k TIME(2 n k )

9/12/20137 Outline Why big-oh? Linear Speedup Theorem Hierarchy Theorems Robust Time and Space Classes Relationships between classes Some complete problems

9/12/20138 Linear Speedup Theorem: Suppose TM M decides language L in time f(n). Then for any  > 0, there exists TM M’ that decides L in time  f(n) + n + 2. Proof: –simple idea: increase “word length” –M’ will have one more tape than M m-tuples of symbols of M ∑ new = ∑ old  ∑ old m many more states

9/12/20139 Linear Speedup part 1: compress input onto fresh tape... ababbaaa ababbaaa_

9/12/ Linear Speedup part 2: simulate M, m steps at a time bbaababaaab... abbaababaaababa... mm –4 (L,R,R,L) steps to read relevant symbols, “remember” in state –2 (L,R or R,L) to make M’s changes

9/12/ Linear Speedup accounting: –part 1 (copying): n + 2 steps –part 2 (simulation): 6 (f(n)/m) –set m = 6/  –total:  f(n) + n + 2 Theorem: Suppose TM M decides language L in space f(n). Then for any  > 0, there exists TM M’ that decides L in space  f(n) + 2. Proof: same.

9/12/ Time and Space Moral: big-oh notation necessary given our model of computation –Recall: f(n) = O(g(n)) if there exists c such that f(n) ≤ c g(n) for all sufficiently large n. –TM model incapable of making distinctions between time and space usage that differs by a constant. In general: interested in course distinctions not affected by model –e.g. simulation of k-string TM running in time f(n) by single-string TM running in time O(f(n) 2 )

9/12/ Hierarchy Theorems Does genuinely more time permit us to decide new languages? how can we construct a language L that is not in TIME(f(n))… idea: same as “HALT undecidable” diagonalization and simulation

9/12/ Recall proof for Halting Problem Turing Machines inputs Y n Y n n Y n YnYYnnY H’ : box (M, x): does M halt on x? The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.

9/12/ Time Hierarchy Theorem Turing Machines inputs Y n Y n n Y n YnYYnnY D : box (M, x): does M accept x in time f(n)? TM SIM tells us yes/no for each box in time g(n) rows include all of TIME(f(n)) construct TM D running in time g(2n) that is not in table

9/12/ Time Hierarchy Theorem Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n: TIME(f(n)) ( TIME(f(2n) 3 ). more on “proper complexity functions” later…

9/12/ Proof of Time Hierarchy Theorem Proof: –SIM is TM deciding language { : M accepts x in ≤ f(|x|) steps } –Claim: SIM runs in time g(n) = f(n) 3. –define new TM D: on input if SIM accepts, reject if SIM rejects, accept –D runs in time g(2n)

9/12/ Proof of Time Hierarchy Theorem Proof (continued): –suppose M in TIME(f(n)) decides L(D) M( ) = SIM( ) ≠ D( ) but M( ) = D( ) –contradiction.

9/12/ Proof of Time Hierarchy Theorem Claim: there is a TM SIM that decides { : M accepts x in ≤ f(|x|) steps} and runs in time g(n) = f(n) 3. Proof sketch: SIM has 4 work tapes contents and “virtual head” positions for M’s tapes M’s transition function and state f(|x|) “+”s used as a clock scratch space

9/12/ Proof of Time Hierarchy Theorem contents and “virtual head” positions for M’s tapes M’s transition function and state f(|x|) “+”s used as a clock scratch space –initialize tapes –simulate step of M, advance head on tape 3; repeat. –can check running time is as claimed. Important detail: need to initialize tape 3 in time O(f(n))

9/12/ Proper Complexity Functions Definition: f is a proper complexity function if –f(n) ≥ f(n-1) for all n –there exists a TM M that outputs exactly f(n) symbols on input 1 n, and runs in time O(f(n) + n) and space O(f(n)).

9/12/ Proper Complexity Functions includes all reasonable functions we will work with –log n, √n, n 2, 2 n, n!, … –if f and g are proper then f + g, fg, f(g), f g, 2 g are all proper can mostly ignore, but be aware it is a genuine concern: Theorem: 9 non-proper f such that TIME(f(n)) = TIME(2 f(n) ).

9/12/ Hierarchy Theorems Does genuinely more space permit us to decide new languages? Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n: SPACE(f(n)) ( SPACE(f(n) log f(n)). Proof: same ideas.

9/12/ Best Hierarchy Theorems Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n: TIME(f(n)) ( TIME(  (f(n)log(f(n))). Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n: SPACE(f(n)) ( SPACE(  f(n)). 

9/12/ Time and Space Classes L = SPACE(log n) PSPACE =  k SPACE(n k ) P =  k TIME(n k ) EXP =  k TIME(2 n k )

9/12/ Hierarchy Separations Time Hierarchy Theorem implies P ( EXP –P µ TIME(2 n ) ( TIME(2 (2n)3 ) µ EXP Space Hierarchy Theorem implies L ( PSPACE –L = SPACE(log n) ( SPACE(log 2 n) µ PSPACE

9/12/ TIME(f(n))  SPACE(f(n)) Any time step can only affect constant number of tape cells : P µ PSPACE

9/12/ SPACE(f(n))  TIME(2 O(f(n)) ) T in TIME(f(n)) says yes or no – never loops So T can’t repeat a configuration. log(n) + klog(f(n))+ c + O(f(n) bits = O(f(n)) bits if f(n)>=log(n) 28 input-tape head position work-tape head position state work-tape contents

SPACE(f(n))  TIME(2 O(f(n)) ) L µ P PSPACE µ EXP So far: L µ P µ PSPACE µ EXP Open: L = P ??? P = PSPACE ?? 9/12/201329

9/12/ Reductions reductions are the main tool for relating problems to each other given two languages L 1 and L 2 we say “L 1 reduces to L 2 ” and we write “L 1 ≤ L 2 ” to mean: –there exists an efficient (logspace) algorithm that computes a function f s.t. x  L 1 implies f(x)  L 2 x  L 1 implies f(x)  L 2

9/12/ Reductions positive use: given new problem L 1 reduce it to L 2 that we know to be in P. Conclude L 1 in P (how?) –e.g. bipartite matching ≤ max flow –formalizes “L 1 as easy as L 2 ” yes no yes no L1L1 L2L2 f f

9/12/ Reductions negative use: given new problem L 2 reduce L 1 (that we believe not to be in P) to it. Conclude L 2 not in P if L 1 not in P (how?) –e.g. satisfiability ≤ graph 3-coloring –formalizes “L 2 as hard as L 1 ” yes no yes no L1L1 L2L2 f f

9/12/ Reductions Example reduction: –3SAT = { φ : φ is a 3-CNF Boolean formula that has a satisfying assignment } (3-CNF = AND of OR of ≤ 3 literals) –IS = { (G, k) | G is a graph with an independent set V’ µ V of size ≥ k } (ind. set = set of vertices no two of which are connected by an edge)

9/12/ Ind. Set is as easy as 3-SAT The reduction f: given φ = (x  y   z)  (  x  w  z)  …  (…) we produce graph G φ : x y zz  x x wz one triangle for each of m clauses edge between every pair of contradictory literals set k = m …

9/12/ Reductions φ = (x  y   z)  (  x  w  z)  …  (…) Claim: φ has a satisfying assignment if and only if G has an independent set of size at least k –proof? Conclude that 3SAT ≤ IS. x y zz  x x wz …

9/12/ Hardness L is C-hard if –every language in C reduces to L

9/12/ Completeness complexity class C language L is C-complete if –L is in C –L is C-hard