1. Eric Allender Rutgers University Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds DCFS 2008

2. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 2 >< 2 > Today’s Goal:  To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.  And to show that this is useful.  To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.

3. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 3 >< 3 > Kolmogorov Complexity  C(x) = min{|d| : U(d) = x}  Important property – Invariance: The choice of the universal Turing machine U is unimportant.  x is random if C(x) ≥ |x|.  C A (x) = min{|d| : U A (d) = x}

4. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 4 >< 4 > Circuit Complexity  Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.  Size(f) = min{Size(D) : D computes f}  We may allow oracle gates for a set A, along with AND and OR gates.  Size A (f) = min{Size(D) : D A computes f}

5. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 5 >< 5 > K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions. What are some differences?  A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. – Given any string x, let f x be the function whose truth table is the string of length 2 log|x|, padded out with 0’s, and define Size(x) to be Size(f x ).

6. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 6 >< 6 > K-complexity ≈ Circuit Complexity  There are some obvious similarities in the definitions. What are some differences?  A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.  A more fundamental difference: – C(x) is not computable; Size(x) is.  The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.

7. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 7 >< 7 > MCSP  MCSP is in NP, but is not known to be NP- complete.  MCSP is not believed to be in P. – Factoring is in BPP MCSP. – Every cryptographically-secure one-way function can be inverted in P MCSP /poly.

8. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 8 >< 8 > So how can K-complexity and Circuit complexity be the same?  C(x) ≈ Size H (x), where H is the halting problem.  For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for f x, where f x (i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.  For the other direction, let Size H (f x ) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate D H and produce f x

9. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 9 >< 9 > So how can K-complexity and Circuit complexity be the same?  C(x) ≈ Size H (x), where H is the halting problem.  So there is a connection between C(x) and Size(x) …  …but is it useful?  First, let’s look at decidable versions of Kolmogorov complexity.

10. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity  The usual definition:  C t (x) = min{|d| : U(d) = x in time t(|d|)}.  Problems with this definition – No invariance! If U and U’ are different universal Turing machines, C t U and C t U’ have no clear relationship. – (One can bound C t U by C t’ U’ for t’ slightly larger than t – but nothing can be done for t’=t.)  No nice connection to circuit complexity!

11. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity  Levin’s definition:  Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.  Invariance holds! If U and U’ are different universal Turing machines, Kt U (x) and Kt U’ (x) are within log |x| of each other.  Let A be complete for E = Dtime(2 O(n) ). Then Kt(x) ≈ Size A (x).

12. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Time-Bounded Kolmogorov Complexity  Levin’s definition:  Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.  Why log t? – This gives an optimal search order for NP search problems. – Adding t instead of log t would give every string complexity ≥ |x|.  …So let’s look at how to make the run-time be much smaller.

13. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Revised Kolmogorov Complexity  C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). – This is identical to the original definition.  Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}. – The new and old definitions are within O(log |x|) of each other.  Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

14. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – Branching Program Size ≈ KB(x) = min{|d|+2 s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.

15. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – Formula Size ≈ KF(x) = min{|d|+2 t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

16. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? …but is this interesting?  The result that Factoring is in BPP MCSP was first proved by observing that, in P MCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).  (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPP T for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.

17. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants.  Consider R KT, R Kt, and R C.  R KT is in coNP, and not known to be coNP hard.  R C is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? – Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in  But R C is undecidable! Perhaps H is in P relative to R C ?

18. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants.  Consider R KT, R Kt, and R C.  R KT is in coNP, and not known to be coNP hard.  R C, is not hard for NP under poly-time many-one reductions, unless P=NP. – How about more powerful reductions? – PSPACE is in P relative to R C. – NEXP is in NP relative to R C. – Proof uses PRGs, Interactive Proofs, and the fact that an element of R C of length n can be found in poly time, relative to R C [BFNV].  But R C is undecidable! Perhaps H is in P relative to R C ?

19. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Relationship between H and R C  Perhaps H is in P relative to R C ?  This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to R C [Kummer].  …but the bound t depends on the choice of U in the definition of C(x).  We also know that H is in P/poly relative to R C.

20. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? This idea has many variants.  Consider R KT, R Kt, and R C.  What about R Kt ?  R Kt, is not hard for NP under poly-time many- one reductions, unless E=NE. – How about more powerful reductions? – EXP = NP(R Kt ). – R Kt is complete for EXP under P/poly reductions. – Open if R Kt is in P!

21. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – Formula Size ≈ KF(x) = min{|d|+2 t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

22. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Kolmogorov Complexity is Circuit Complexity  C(x) ≈ Size H (x).  Kt(x) ≈ Size E (x).  KT(x) ≈ Size(x).  Other measures of complexity can be captured in this way, too: – A similar definition captures depth k threshold circuit size.  [This is the clever transition to start the discussion of circuit lower bounds…]

23. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Big Complexity Classes  NP PP .. ..  NC  L (Deterministic Logspace)

24. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  TC 0 O(1)-Depth Circuits of MAJ gates  AC 0 [6]  NC 1 Log-Depth Circuits  AC 0 can’t compute Mod 2 [FSS,A]  AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes

25. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  TC 0 O(1)-Depth Circuits of MAJ gates  AC 0 [6]  NC 1 Log-Depth Circuits  AC 0 can’t compute Mod 2 [FSS,A]  AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes

26. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  TC 0 O(1)-Depth Circuits of MAJ gates  NC 1 Log-Depth Circuits  AC 0 [2] can’t compute Mod 3 [R,S]  AC 0 [2]  AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes

27. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  NC 1 Log-Depth Circuits  TC 0 O(1)-Depth Circuits of MAJ gates  AC 0 [6]  AC 0 [2]  AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes

28. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  NC 1 poly-size formulae  TC 0 O(1)-Depth Circuits of MAJ gates  AC 0 [6]  AC 0 [2]  AC 0 O(1)-Depth Circuits of AND/OR gates The Main Objects of Interest: Small Complexity Classes

29. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  NP has complete sets (under polynomial time reducibility ≤ P )  These small classes have complete sets, too (under ≤ AC° ) Complete Problems

30. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Reductions  A ≤ AC° B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B

31. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  NC 1  TC 0  AC 0 [6]  AC 0 [2]  AC 0 Complete Problems  sorting, multiplication, division  [Naor,Reingold] Pseudorandom Generator

32. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  NC 1  TC 0  AC 0 [6]  AC 0 [2]  AC 0 Complete Problems  BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)  Word problem over S 5

33. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? The Word Problem Over S 5  A regular set complete for NC 1 =

34. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered  NP (SAT, Clique, TSP,…)  P (Linear Programming, CVP, …)  NL (Connectivity, Shortest Paths, 2SAT, …)  L (Undirected Connectivity, Acyclicity, …)  NC 1 (BFE, Regular Sets)  TC 0 (Sorting, Multiplication, Division) We’re interested in NC 1 (for instance) not because we want to build formulae for these functions…

35. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered  NP (SAT, Clique, TSP,…)  P (Linear Programming, CVP, …)  NL (Connectivity, Shortest Paths, 2SAT, …)  L (Undirected Connectivity, Acyclicity, …)  NC 1 (BFE, Regular Sets)  TC 0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.

36. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Complexity Classes are not Invented – They’re Discovered  NP (SAT, Clique, TSP,…)  P (Linear Programming, CVP, …)  NL (Connectivity, Shortest Paths, 2SAT, …)  L (Undirected Connectivity, Acyclicity, …)  NC 1 (BFE, Regular Sets)  TC 0 (Sorting, Multiplication, Division) These classes are real. They’re important.

37. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Longstanding Open Problems  Is P = NP?  Is AC 0 [6] = NP?  Is depth 3 AC 0 [6] = NP? We’ll focus on questions such as : Is BFE in TC 0 ? Is BFE in AC 0 [6]?

38. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? How Close Are We to Proving Circuit Lower Bounds?  Conventional Wisdom: Not Close At All!  No new superpolynomial size lower bounds in over two decades.  Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.  Since the [Naor, Reingold] generator is computable in TC 0, this is bad news.

39. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? More Modest Goals  Problems requiring formulae of size n 3 [Håstad]  Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]  Problems requiring depth d TC 0 circuits of size n 1+ c [Impagliazzo, Paturi, Saks]  Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]  There is little feeling that these results bring us any closer to separating complexity classes.

40. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  How close are the following two statements?  TC 0 Circuits for BFE must be of size n 1+Ω(1)  For some c >0, TC 0 Circuits for BFE must be of size n 1+ c. How Close Are We to Proving Circuit Lower Bounds?

41. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  How close are the following two statements?  TC 0 Circuits for BFE must be of size n 1+Ω(1)  For some c >0, FTC 0 Circuits for BFE must be of size n 1+ c How Close Are We to Proving Circuit Lower Bounds? This is known [IPS’97] This implies TC 0 ≠ NC 1 [A, Koucky]

42. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility  A set B is said to be “self-reducible” if B ≤ P B

43. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility  A set B is said to be “self-reducible” if B ≤ P B via a reduction that, on input x, does not ask about whether x is in B.  Very well-studied notion.  For example, φ is in SAT if and only if (φ 0 is in SAT) or (φ 1 is in SAT)

44. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility  Many of the important problems in (or near) NC 1 have a special self-reducibility property:

45. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self-Reducibility  Many of the important problems in (or near) NC 1 have a special self-reducibility property: Instances of length n are AC 0 -Turing (or TC 0 - Turing) reducible to instances of length n ½ via reductions of linear size.  Examples: – BFE – the word problem over S 5 – MAJORITY – Iterated Product of 3-by-3 Integer Matrices

46. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility  BFE A subformula near the root Subformulae near inputs

47. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility S5S5

48. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Self Reducibility  The self-reduction of S 5, on inputs of size n, uses ( n ½ + 1) oracle gates of size n ½.  Thus if S 5 has TC 0 circuits of size n k, it also has circuits of size ( n ½ + 1) n k/ 2 = O(n (k+ 1)/2 ).  Similar arguments hold for other classes (such as AC 0 [6] and NC 1 ).  More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

49. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? A Corollary  If BFE has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size.  If S 5 has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size.  If MAJ has AC 0 [6] circuits, then it has such circuits of nearly linear size. (Etc.)  Thus, e.g., to separate NC 1 from TC 0, it suffices to show that BFE requires TC 0 circuits of size n 1.0000001.

50. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? A Corollary  If BFE has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size.  If S 5 has TC 0 or AC 0 [6] circuits, then it has such circuits of nearly linear size.  If MAJ has AC 0 [6] circuits, then it has such circuits of nearly linear size. (Etc.)  How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC 0 by proving that SAT requires TC 0 circuits of size n 1.0000001 ?)

51. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility  Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC.  Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.  …but perhaps, rather than showing directly that SAT has this strong form of self- reducibility, one can argue that if SAT is in TC 0 then it has TC 0 circuits of nearly-linear size.

52. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility  Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC.

53. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility  Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d levels of oracle gates

54. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility  Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d 2 levels of oracle gates

55. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Limitations of Self-Reducibility  Any problem for which instances of length n are TC 0 -Turing reducible to instances of length n ½ via poly-size reductions lies in NC. d 3 levels of oracle gates After log log rounds, the depth is log O(1) n

56. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress  We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC 1 and TC 0. Some of these proofs are “natural”.  Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?  Not necessarily!

57. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress  The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.  In contrast, we are making use of a self- reducibility property that allows us to boost a n 1+ ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.

58. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Prospects for Progress  These observations are simple, but …  they have forever changed the way that we look at quadratic (and smaller) lower bounds.  We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC 1 ≠ TC 0.) But we do believe that this avenue deserves further exploration.

59. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.  There are many open problems in this area that I will be delighted to discuss with you in more detail.  There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page. Conclusion

60. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC 0 (depth 3). Speculation

61. Eric Allender: How Close Are We to Proving Circuit Lower Bounds?  [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized  The proof shows that NEXP is in P/poly iff every set in P contains strings of KT- complexity O(log n) iff NEXP = IP[P/poly]. Speculation

62. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? Similar techniques show:  NEXP is in nonuniform NC 1 iff every set in P contains strings of KF- complexity O(log n) iff NEXP = MIPNC 1 iff MIPNC 1 cannot be derandomized.  NEXP is in nonuniform TC 0 iff every set in P contains strings of small complexity iff NEXP = MIPTC 0 iff MIPTC 0 cannot be derandomized. Speculation

63. Eric Allender: How Close Are We to Proving Circuit Lower Bounds? What else happens in such a collapse?  If NP = uniform TC 0, then #P is not contained in non-uniform TC 0 (so NEXP is not in non-uniform TC 0 ).  So let’s consider NEXP = MIPTC 0 and NP ≠ uniform TC 0. If this “hardness assumption” were sufficient to “derandomize” MIPTC 0 then this would give the desired lower bound on NEXP…  [Fortnow, Klivans], [van Melkebeek, Santhanam] Speculation