1. Fall 2013 CMU CS 15-855 Computational Complexity Lecture 7 Alternating Quantifiers, PH, PSPACE. These slides are mostly a resequencing of Chris Umans’ slides From CS 151: Complexity Theory (Spring 2013) at Caltech.CS 151: Complexity Theory (Spring 2013) 10/1/2013

2. 2 A motivating question Central problem in logic synthesis: Complexity of this problem? –NP-hard? in NP? in coNP? in PSPACE? –complete for any of these classes?   x1x1 x2x2   x3x3 …xnxn  given Boolean circuit C, integer k is there a circuit C’ of size at most k that computes the same function C does?

3. 10/1/20133 Oracle Turing Machines Oracle Turing Machine (OTM): –multitape TM M with special “query” tape –special states q ?, q yes, q no –on input x, with oracle language A –M A runs as usual, except… –when M A enters state q ? : y = contents of query tape y  A  transition to q yes y  A  transition to q no

4. 10/1/20134 Oracle Turing Machines Nondeterministic OTM –defined in the same way –(transition relation, rather than function) oracle is like a subroutine, or function in your favorite programming language –but each call counts as single step e.g.: given φ 1, φ 2, …, φ n are even # satisfiable? –poly-time OTM solves with SAT oracle

5. 10/1/20135 Oracle Turing Machines Shorthand #1: applying oracles to entire complexity classes: –complexity class C –language A C A = {L decided by OTM M with oracle A with M “in” C} –example: P SAT

6. 10/1/20136 Oracle Turing Machines Shorthand #2: using complexity classes as oracles: –OTM M –complexity class C –M C decides language L if for some language A  C, M A decides L Both together: C D = languages decided by OTM “in” C with oracle language from D exercise: show P SAT = P NP

7. 10/1/20137 The Polynomial-Time Hierarchy can define lots of complexity classes using oracles the classes on the next slide stand out –they have natural complete problems –they have a natural interpretation in terms of alternating quantifiers –they help us state certain consequences and containments (more later)

8. 10/1/20138 The Polynomial-Time Hierarchy Σ 0 = Π 0 = P Δ 1 =P P Σ 1 =NPΠ 1 =coNP Δ 2 =P NP Σ 2 =NP NP Π 2 =coNP NP Δ i+1 =P Σ i Σ i+i =NP Σ i Π i+1 =coNP Σ i Polynomial Hierarchy PH =  i Σ i

9. 10/1/20139 The Polynomial-Time Hierarchy Σ 0 = Π 0 = P Δ i+1 =P Σ i Σ i+i =NP Σ i Π i+1 =coNP Σ i Example: –MIN CIRCUIT: given Boolean circuit C, integer k; is there a circuit C’ of size at most k that computes the same function C does? –MIN CIRCUIT  Σ 2

10. 10/1/201310 The Polynomial-Time Hierarchy Σ 0 = Π 0 = P Δ i+1 =P Σ i Σ i+i =NP Σ i Π i+1 =coNP Σ i Example: –EXACT TSP: given a weighted graph G, and an integer k; is the k-th bit of the length of the shortest TSP tour in G a 1? –EXACT TSP  Δ 2

11. 10/1/201311 The PH PSPACE: generalized geography, 2-person games… 3rd level: V-C dimension… 2nd level: MIN CIRCUIT, BPP… 1st level: SAT, UNSAT, factoring, etc… P NP coNP Σ3Σ3 Π3Π3 Δ3Δ3 PSPACE EXP PH Σ2Σ2 Π2Π2 Δ2Δ2

12. 10/1//2013 Useful characterization Recall: L  NP iff expressible as L = { x | 9 y, |y| ≤ |x| k, (x, y)  R } where R  P. Corollary: L  coNP iff expressible as L = { x | 8 y, |y| ≤ |x| k, (x, y)  R } where R  P.

13. 10/1//2013 Useful characterization Theorem: L  Σ i iff expressible as L = { x | 9 y, |y| ≤ |x| k, (x, y)  R } where R  Π i-1. Corollary: L  Π i iff expressible as L = { x | 8 y, |y| ≤ |x| k, (x, y)  R } where R  Σ i-1.

14. 10/1//2013 Alternating quantifiers Nicer, more usable version: L  Σ i iff expressible as L = { x | 9 y 1 8 y 2 9 y 3 …Qy i (x, y 1,y 2,…,y i )  R } where Q= 8 / 9 if i even/odd, and R  P L  Π i iff expressible as L = { x | 8 y 1 9 y 2 8 y 3 …Qy i (x, y 1,y 2,…,y i )  R } where Q= 9 / 8 if i even/odd, and R  P

15. 10/1//2013 Alternating quantifiers Proof: –(  ) induction on i –base case: true for Σ 1 =NP and Π 1 =coNP –consider L  Σ i : L = {x | 9 y 1 (x, y 1 )  R’ }, for R’  Π i-1 L = {x | 9 y 1 8 y 2 9 y 3 …Qy i ((x, y 1 ), y 2,…,y i )  R} L = {x | 9 y 1 8 y 2 9 y 3 …Qy i (x, y 1,y 2,…,y i )  R} –same argument for L  Π i –(  ) exercise.

16. 10/1//2013 Alternating quantifiers Pleasing viewpoint: P NP coNP Δ3Δ3 Σ2Σ2 Π2Π2 Δ2Δ2 “9”“9” “8”“8” “ 98 ”“ 89 ” Σ3Σ3 Π3Π3 PSPACE PH “ 989 ” “ 898 ” “ 9898989 …” ΣiΣi ΠiΠi “ 989 …” “ 898 …” const. # of alternations poly(n) alternations

17. 10/1//2013 Complete problems three variants of SAT: –QSAT i (i odd) = {3-CNFs φ(x 1, x 2, …, x i ) for which 9 x 1 8 x 2 9 x 3 … 9 x i φ(x 1, x 2, …, x i ) = 1} –QSAT i (i even) = {3-DNFs φ(x 1, x 2, …, x i ) for which 9 x 1 8 x 2 9 x 3 … 8 x i φ(x 1, x 2, …, x i ) = 1} –QSAT = {3-CNFs φ for which 9 x 1 8 x 2 9 x 3 … Qx n φ(x 1, x 2, …, x n ) = 1}

18. 10/1//2013 QSAT i is Σ i -complete Theorem: QSAT i is Σ i -complete. Proof: (clearly in Σ i ) –assume i odd; given L  Σ i in form { x | 9 y 1 8 y 2 9 y 3 … 9 y i (x, y 1,y 2,…,y i )  R } …x… …y 1 … …y 2 … …y 3 … … …y i … C CVAL reduction for R 1 iff (x, y 1,y 2,…,y i )  R

19. 10/1//2013 QSAT i is Σ i -complete –Problem set: can construct 3-CNF φ from C: 9 z φ(x,y 1,…,y i, z) = 1  C(x,y 1,…,y i ) = 1 –we get: 9 y 1 8 y 2 … 9 y i 9 z φ(x,y 1,…,y i, z) = 1  9 y 1 8 y 2 … 9 y i C(x,y 1,…,y i ) = 1  x  L …x… …y 1 … …y 2 … …y 3 … … …y i … C CVAL reduction for R 1 iff (x, y 1,y 2,…,y i )  R

20. 10/1//2013 QSAT i is Σ i -complete Proof (continued) –assume i even; given L  Σ i in form { x | 9 y 1 8 y 2 9 y 3 … 8 y i (x, y 1,y 2,…,y i )  R } …x… …y 1 … …y 2 … …y 3 … … …y i … C CVAL reduction for R 1 iff (x, y 1,y 2,…,y i )  R

21. 10/1//2013 QSAT i is Σ i -complete –Problem set: can construct 3-DNF φ from C: 8 z φ(x,y 1,…,y i, z) = 1  C(x,y 1,…,y i ) = 1 –we get: 9 y 1 8 y 2 … 8 y i 8 z φ(x,y 1,y 2,…,y i, z) = 1  9 y 1 8 y 2 … 8 y i C(x,y 1,y 2,…,y i ) = 1  x  L …x… …y 1 … …y 2 … …y 3 … … …y i … C CVAL reduction for R 1 iff (x, y 1,y 2,…,y i )  R

22. 10/1//2013 QSAT is PSPACE-complete Theorem: QSAT is PSPACE-complete. Proof: 8 x 1 9 x 2 8 x 3 … Qx n φ(x 1, x 2, …, x n )? –in PSPACE: 9 x 1 8 x 2 9 x 3 … Qx n φ(x 1, x 2, …, x n )? –“ 9 x 1 ”: for each x 1, recursively solve 8 x 2 9 x 3 … Qx n φ(x 1, x 2, …, x n )? if encounter “yes”, return “yes” –“ 8 x 1 ”: for each x 1, recursively solve 9 x 2 8 x 3 … Qx n φ(x 1, x 2, …, x n )? if encounter “no”, return “no” –base case: evaluating a 3-CNF expression –poly(n) recursion depth –poly(n) bits of state at each level

23. 10/1//2013 QSAT is PSPACE-complete –given TM M deciding L  PSPACE; input x –2 n k possible configurations –single START configuration –assume single ACCEPT configuration –define: REACH(X, Y, i)  configuration Y reachable from configuration X in at most 2 i steps.

24. 10/1//2013 QSAT is PSPACE-complete REACH(X, Y, i)  configuration Y reachable from configuration X in at most 2 i steps. –Goal: produce 3-CNF φ(w 1,w 2,w 3,…,w m ) such that 9 w 1 8 w 2 …Qw m φ(w 1,…,w m )  REACH(START, ACCEPT, n k )

25. 10/1//2013 QSAT is PSPACE-complete –for i = 0, 1, … n k produce quantified Boolean expressions ψ i (A, B, W) 9 w 1 8 w 2 … ψ i (A, B, W)  REACH(A, B, i) –convert ψ n k to 3-CNF φ add variables V 9 w 1 8 w 2 … 9 V φ(A, B, W, V)  REACH(A, B, n k ) –hardwire A = START, B = ACCEPT 9 w 1 8 w 2 … 9 V φ(W, V)  x  L

26. 10/1//2013 QSAT is PSPACE-complete –ψ o (A, B) = true iff A = B or A yields B in one step of M … … STEP config. A config. B Boolean expression of size O(n k )

27. 10/1//2013 QSAT is PSPACE-complete –Key observation #1: REACH(A, B, i+1)  9 Z [ REACH(A, Z, i)  REACH(Z, B, i) ] –cannot define ψ i+1 (A, B) to be 9 Z [ ψ i (A, Z)  ψ i (Z, B) ] (why?)

28. 10/1//2013 QSAT is PSPACE-complete –Key idea #2: use quantifiers –couldn’t do ψ i+1 (A, B) = 9 Z [ ψ i (A, Z)  ψ i (Z, B) ] –define ψ i+1 (A, B) to be 9 Z 8 X 8 Y [ ((X=A  Y=Z)  (X=Z  Y=B))  ψ i (X, Y) ] –ψ i (X, Y) is preceded by quantifiers –move to front (they don’t involve X,Y,Z,A,B)

29. 10/1//2013 QSAT is PSPACE-complete ψ o (A, B) = true iff A = B or A yields B in 1 step ψ i+1 (A, B) = 9 Z 8 X 8 Y [ ((X=A  Y=Z)  (X=Z  Y=B))  ψ i (X, Y) ] –|ψ 0 | = O(n k ) –|ψ i+1 | = O(n k ) + |ψ i | –total size of ψ n k is O(n k ) 2 = poly(n) –logspace reduction

30. 10/1//2013 PH collapse Theorem: if Σ i = Π i then for all j > i Σ j = Π j = Δ j = Σ i “the polynomial hierarchy collapses to the i-th level” Proof: –sufficient to show Σ i = Σ i+1 –then Σ i+1 = Σ i = Π i = Π i+1 ; apply theorem again P NP coNP Σ3Σ3 Π3Π3 Δ3Δ3 PH Σ2Σ2 Π2Π2 Δ2Δ2

31. 10/1//2013 PH collapse –recall: L  Σ i+1 iff expressible as L = { x | 9 y (x, y)  R } where R  Π i –since Π i = Σ i, R expressible as R = { (x,y) | 9 z ((x, y), z)  R’ } where R’  Π i-1 –together: L = { x | 9 (y, z) (x, (y, z))  R’} –conclude L  Σ i

32. 10/1//2013 Oracles vs. Algorithms A point to ponder: given poly-time algorithm for SAT –can you solve MIN CIRCUIT efficiently? –what other problems? Entire complexity classes? given SAT oracle –same input/output behavior –can you solve MIN CIRCUIT efficiently?

33. 10/1//2013 Natural complete problems We now have versions of SAT complete for levels in PH, PSPACE Natural complete problems? –PSPACE: games –PH: almost all natural problems lie in the second and third level

34. 10/1//2013 Natural complete problems in PH –MIN CIRCUIT good candidate to be  2 -complete, still open –MIN DNF: given DNF φ, integer k; is there a DNF φ’ of size at most k computing same function φ does? Theorem (U): MIN DNF is Σ 2 -complete.

35. 10/1//2013 Natural complete problems in PSPACE General phenomenon: many 2-player games are PSPACE-complete. –2 players I, II –alternate pick- ing edges –lose when no unvisited choice pasadena athens auckland san francisco oakland davis GEOGRAPHY = {(G, s) : G is a directed graph and player I can win from node s}

36. 10/1//2013 Natural complete problems in PSPACE Theorem: GEOGRAPHY is PSPACE- complete. Proof: –in PSPACE easily expressed with alternating quantifiers –PSPACE-hard reduction from QSAT

37. 10/1//2013 Natural complete problems in PSPACE 9 x 1 8 x 2 9 x 3 …(  x 1  x 2  x 3 )  (  x 3  x 1 )  …  (x 1  x 2 ) true false truefalse truefalse … x1x1 x2x2 x3x3 I II I I I I alternately pick truth assignment pick a clause I II I pick a true literal?

38. 10/1//2013 Karp-Lipton we know that P = NP implies SAT has polynomial-size circuits. –(showing SAT does not have poly-size circuits is one route to proving P ≠ NP) suppose SAT has poly-size circuits –any consequences? –might hope: SAT  P/poly  PH collapses to P, same as if SAT  P

39. 10/1//2013 Karp-Lipton Theorem (KL): if SAT has poly-size circuits then PH collapses to the second level. Proof: –suffices to show Π 2  Σ 2 –L  Π 2 implies L expressible as: L = { x : 8 y 9 z (x, y, z)  R} with R  P.

40. 10/1//2013 Karp-Lipton L = { x : 8 y 9 z (x, y, z)  R} –given (x, y), “ 9 z (x, y, z)  R?” is in NP –pretend C solves SAT, use self-reducibility –Claim: if SAT  P/poly, then L = { x : 9 C 8 y [use C repeatedly to find some z for which (x, y, z)  R; accept iff (x, y, z)  R] } poly time

41. 10/1//2013 Karp-Lipton L = { x : 8 y 9 z (x, y, z)  R} {x : 9 C 8 y [use C repeatedly to find some z for which (x,y,z)  R; accept iff (x,y,z)  R] } –x  L: some C decides SAT  9 C 8 y […] accepts –x  L: 9 y 8 z (x, y, z)  R  9 C 8 y […] rejects

42. 10/1//201342 Randomized complexity classes model: probabilistic Turing Machine –deterministic TM with additional read-only tape containing “coin flips” BPP (Bounded-error Probabilistic Poly-time) –L  BPP if there is a p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ 2/3 x  L  Pr y [M(x,y) rejects] ≥ 2/3 –“p.p.t” = probabilistic polynomial time

43. 10/1//2013 BPP  PH Recall: don’t know BPP different from EXP Theorem (S,L,GZ): BPP  (Π 2  Σ 2 ) don’t know Π 2  Σ 2 different from EXP but believe much weaker

44. 10/1//2013 BPP  PH Proof: –BPP language L: p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ 2/3 x  L  Pr y [M(x,y) rejects] ≥ 2/3 –strong error reduction: p.p.t. TM M’ use n random bits (|y’| = n) # strings y’ for which M’(x, y’) incorrect is at most 2 n/3 (can’t achieve with naïve amplification)

45. 10/1//2013 BPP  PH view y’ = (w, z), each of length n/2 consider output of M’(x, (w, z)): 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 0 1 1 1 0 1 0 0 1 1 0 1 11 1 1 1 1 1 1 1 1 0 0 0 1 1 00 0 1 0 1 1 0 1 1 1 1 1 1 1 0 11 0 0 0 0 0 0 1 0 0 1 0 0 0 0 00 00 0 1 0 0 0 0 0 0 0 0 0 0 1 0 00 0 0 0 0 0 0 0 0 0 0 0 0 00 00 0 0 0 0 1 0 0 1 0 0 0 0 00 0 w = 000…00 000…01 000…10 … 111…11 … … xLxL xLxL so many ones, some disk is all ones so few ones, not enough for whole disk

46. 10/1//2013 BPP  PH proof (continued): –strong error reduction: # bad y’ < 2 n/3 –y’ = (w, z) with |w| = |z| = n/2 –Claim: L = {x : 9 w 8 z M’(x, (w, z)) = 1 } – x  L: suppose 8 w 9 z M’(x, (w, z)) = 0 implies  2 n/2 0’s; contradiction – x  L: suppose 9 w 8 z M’(x, (w, z)) = 1 implies  2 n/2 1’s; contradiction

47. 10/1//2013 BPP  PH –given BPP language L: p.p.t. TM M: x  L  Pr y [M(x,y) accepts] ≥ 2/3 x  L  Pr y [M(x,y) rejects] ≥ 2/3 –showed L = {x : 9 w 8 z M’(x, (w, z)) = 1} –thus BPP  Σ 2 –BPP closed under complement  BPP  Π 2 –conclude: BPP  (Π 2  Σ 2 )

48. 10/1//2013 New Topic The complexity of counting

49. 10/1//2013 Counting problems So far, we have ignored function problems –given x, compute f(x) important class of function problems: counting problems –e.g. given 3-CNF φ how many satisfying assignments are there?

50. 10/1//2013 Counting problems #P is the class of function problems expressible as: input x f(x) = |{y : (x, y)  R}| where R  P. compare to NP (decision problem) input x f(x) =  y : (x, y)  R ? where R  P.

51. 10/1//2013 Counting problems examples –#SAT: given 3-CNF φ how many satisfying assignments are there? –#CLIQUE: given (G, k) how many cliques of size at least k are there?

52. 10/1//2013 Reductions Reduction from function problem f 1 to function problem f 2 –two efficiently computable functions Q, A x (prob. 1) y (prob. 2) f 2 (y) f 1 (x) Q A f2f2 f1f1

53. 10/1//2013 Reductions problem f is #P-complete if –f is in #P –every problem in #P reduces to f “parsimonious reduction”: A is identity –many standard NP-completeness reductions are parsimonious –therefore: if #SAT is #P-complete we get lots of #P-complete problems x (prob. 1) y (prob. 2) f 2 (y) f 1 (x) Q A f2f2 f1f1

54. 10/1//2013 #SAT #SAT: given 3-CNF φ how many satisfying assignments are there? Theorem: #SAT is #P-complete. Proof: –clearly in #P: (φ, A)  R  A satisfies φ –take any f  #P defined by R  P

55. 10/1//2013 #SAT –add new variables z, produce φ such that  z φ(x, y, z) = 1  C(x, y) = 1 –for (x, y) such that C(x, y) = 1 this z is unique –hardwire x –# satisfying assignments = |{y : (x, y)  R}| …x… …y… C CVAL reduction for R 1 iff (x, y)  R f(x) = |{y : (x, y)  R}|

56. 10/1//2013 Relationship to other classes To compare to classes of decision problems, usually consider P #P which is a decision class… easy: NP, coNP  P #P easy: P #P  PSPACE Toda’s Theorem : PH  P #P.