1 Slides by Michael Lewin & Robert Sayegh. Adapted from Oded Goldreich’s course lecture notes by Vered Rosen and Alon Rosen.

2 Introduction In this lecture we will study some of the relations between Boolean circuits and Turing machines: We will define and explore the classes NC and AC Establish a strong connection between space complexity and depth of circuits.

3 Boolean Circuits definitions Definition: A Boolean Circuit is a directed acyclic graph with labeled vertices: The input vertices, labeled with a variable x i or a constant (0 or 1), and have fan-in 0. The gate vertices, have fan-in k>0, and are labeled with a Boolean function on k inputs (    ) - The (  ) gate has fan-in 1 always. The output vertices, labeled ‘output’, have fan-out Given an assignment  {0,1} m on variables x 1 …x, C(  ) denotes the value of the circuit’s output. By assigning to each vertex its Boolean-operation value.

4 Boolean Circuits definitions (2) Size(C) of a circuit, denotes the number of gates in a circuit C. Depth(C) of a circuit, denotes the maximum distance from an input to an output. A bounded fan-in circuit, is a circuit with an a-priori upper bound on the fan-in of its AND and OR gates. An unbounded fan-in circuit, is a circuit with no limitation on the fan-in of its AND and OR gates.

5 Boolean Circuits observations Any circuit with bounded fan-in K, can be trans- formed into a circuit with bounded fan-in 2, paying only a constant factor in its depth and size. Using De-Morgan’s laws, any circuit can be modified in such a way that all the negations appear only in the input layer. We can construct any unbounded fan-in circuit, in the special form where all  and  gates are organized into alternating layers, with edges only between adjacent layers

6 Boolean Circuits observations (2) For any Turing machine M running on some input x  {0,1} n in time T M (n), we can construct a circuit of size T M 2 (n) and of depth bounded by T M (n). Circuits may be organized into disjoint layers, where each layer consists of gates having equal distance from the input vertices. Such circuit presentations capture a notion of parallel programming (more on this later).

7 Families of Circuits definitions Definition: A language L is said to be decided by a family of circuits {Cn}, when Cn accepts n variables as input iff  n  x  {0,1} n Cn(x)=  L ( x ) Definition: Functions Depth D and size S of a family implies that  Cn, Size(Cn)  s(n)  Cn, Depth(Cn)  d(n) Where s(  )  S and d(  )  D

8 Log-space uniformity Definition: A family {Cn}, is called log-space uniform if there exists a DTM, M, s.t.  n M(1 n )= and M is in space log(| |) is the description of the circuit !!Note that M runs in log-space of the output size, so that we can produce circuits of super-polynomial size. By requiring uniformity we correlate the size and depth of a family {Cn}, which decides language L, with the complexity of the language’s TM !! Otherwise, The family of circuits with constant output (true or false) on input 1 n, can easily decide languages, even outside R; Simply by representing the truth table of any language.

9 Small-depth Circuits Motivation: A small depth circuit is a polynomial-size circuit whose depth is poly-logarithmic in its size That is: a circuit with size=p(n) and depth=O(log k n) Next we show that for such circuits the unbounded fan-in will not add much power. And that such circuits capture the notion of efficient parallel computation. 20.2

10 Classes NC and AC Definition: class NC For K  0, NC k is the class of languages that can be decided by families of bounded fan-in, small circuits: {Cn} s.t. size(Cn)=p(n) and depth(Cn)=O(log k n). Definition: class AC For K  0, AC k is the class of languages that can be decided by families of unbounded fan-in, small circuits: {Cn} s.t. size(Cn)=p(n) and depth(Cn)=O(log k n). NC   k NC k AC   k AC k

11 AC = NC Theorem:  k  0 NC k  AC k  NC k+1 Proof: The first inclusion is trivial The second inclusion is easy to observe: The fan-in of AC K circuits must be bound by poly(n), thus each gate can be converted to a tree of identical gates, with fan-in=2 and depth O(logn). By Transforming all AC k gates, we get a circuit with bound fan-in 2 and with a poly-factor in size and with a logarithmic-factor in depth. This is an NC k+1 circuit . Open Questions: - Does the hierarchy collapse? - What’s the inclusion between NC and P

12 AC 0  NC 1 Theorem: AC 0  NC 1 We next show a sketch for proving that the parity problem is in NC 1 but not in AC 0. Definition: Parity(x 1, …,x n )   i x i (mod 2) - The theorem implies that uniform-AC 0  P - Whereas the question uniform-NC 1  ? NP is open !

13 AC 0  NC 1 (2) Claim: Parity  NC 1 Proof: Parity can be computed by a binary tree of xor gates. We then replace each xor gate with three gates: a  b=(  a  b)  (a   b) We increased the size and depth in factors 2 and 3 respectively. Consequently, Parity is computed by circuits of logarithmic depth, and polynomial size; thus in NC 1.

14 AC 0  NC 1 (3) Claim: Parity  AC 0 Proof (sketch): We show that every constant depth circuit computing Parity, must have a sub-exponential size. Therefore, Parity cannot be in AC 0 : - First we Prove that depth 2 parity circuits must be large - Then we prove that depth d small circuits solving parity, can be converted to d-1 depth small-circuits. Thus, contradicting the induction hypothesis. Theorem:  d constant, a circuit computing Parity must have size exp(  (n 1/(d-1) )).

15 AC 0  NC 1 (4) Base (d=2): Assuming the circuit is an AND of OR gates: - Any AND gate evaluating to ‘1’ will determine the value of the circuit. - Every AND must be of fan-in = n, otherwise such AND gate evaluates to ‘1’ independently of some x i - Thus, each AND gate represents some assignment of the input variables. There must be at least 2 n-1 AND gates, otherwise there is an assignment  =x 1,…,x n s.t. Parity(  )=1, of which no AND gate evaluates to ‘1’. Applies Symmetrically for an OR of ANDs circuit

16 AC 0  NC 1 (5) The Induction step: The induction is based on the Lemma of Hastad: Given a depth 2 circuit, say AND of ORs; if one gives random values to a randomly selected subset of variables, it is possible to write the induced circuit as OR of ANDs with very high probability. Given a depth d circuit computing parity, we assign random values to a large number of its inputs. Consequently we obtain a simplified circuit with fewer variables; but still computing parity. By Virtue of the lemma, we can interchange the two layers closest to the input. Then merge the two now adjacent levels with the same connective, thus decreasing the depth of the circuit to d-1.

17 Formally: On input variables x 1,…,x n the random restriction  treats each x i independently as follows: w.p. (1-  )/2set x i =0 x i =w.p. (1-  )/2set x i =1 w.p.  leave x i as a variable - The expected number of variables is m=n . - We would like to reduce the size of the transformed circuit to be smaller than exp(o(m 1/(d-2) )) Requestingn 1/(d-1) < m 1/(d-2) we get n 1/(d-1) < m 1/(d-2) thus we choose,  =n -1/(d-1) . AC 0  NC 1 (6) This is a small-o

18 NC and Parallel Computation Definition (PRAM) A PRAM machine, consists of several independent RAMs, each having a separate set of registers. In addition, there is an infinite shared memory accessible by all RAMs. We denote as PRAM(t(  ),p(  )) the class of languages decidable by A PRAM working in parallel time t(  ) by using p(  ) processors. A parallel computation is said to be efficient, if it can obtains an exponential run-time drop, in solving a problem, comparing to sequential machines

19 NC and Parallel Computation (2) Theorem: uniform-NC = PRAM(polylog, poly) The class NC captures the notion of efficient computation by PRAM machines. Similarly to the way class P captures the notion of efficiency for the RAM machines. The class NC ignores two important aspects of parallel computation: - Communication between processors - The real bottleneck, which is the number of the processors. A PRAM(t(log 2 n),p(n)) is more likely useful than PRAM(t(logn),p(n 2 )).

20 Circuit Depth and Space Complexity Definition: Depth/Size(d(  ),s(  )) is the class of all languages that can be decided by a uniform-family of bounded fan-in circuits of depth d(  ) & size s(  ) Definition: Depth (d(  )) is the class of all languages that can be decided by a uniform-family of bounded fan-in circuits of depth d(  ) 20.3 NC is Depth/Size(polylog,poly) NC  Depth(polylog) - Actually Depth(d(  )) is Depth/Size(d(  ),2 O(d(  )) ) - Therefore, potentially Depth(polylog) contains languages that do not belong to NC

21 Circuit Depth and Space Complexity (2) Theorem: For any integer function s(  )  log(  ) NSPACE(s)  Depth/Size(O(s 2 ),2 O(s) ) Proof: (in the next slide we use a claim to continue the proof) Given a NTM with s(n)-space M, we construct a family {Cn} of depth O(s 2 ) and size 2 O(s) s.t.  X  {0,1} * C |x| (x) = M(x) Recall that the computation of M on x, can be represented by the configuration graph G M,x where M accepts x is the problem of connectivity between the initial configuration vertex to the accepting configuration vertex.

22 Circuit Depth and Space Complexity (3) Claim: CONN  NC 2 proof: - Given a directed graph G, let A be the adjacency matrix. - And let B = A+I (allowing self loops) - Let B 2 i,j =  k (B i,k  B k,j ) B 2 i,j =1 iff (i,j) are connected with a path of length  2 - Using logn such Boolean multiplications, we can compute the matrix B n, which is the adjacency matrix of the transitive closure of A. - Finally: The squaring action is in AC 0, thus in NC 1 Therefore, logn NC 1 multiplications will be in NC 2 . Corollary: NL  NC 2

23 Circuit Depth and Space Complexity (4) Proof (cont): The circuit we build is a composition of two circuits. - The first circuit, generates G M,x for input x and M (i.e. the matrix A) Given x and M, there are 2 O(s) configurations, each represented by O(s) bits; For each pair of configurations, we check if they are adjacent by comparing the contents of the work tape in the two configurations, which is a depth O(logs) operation.

24 Circuit Depth and Space Complexity (5) Proof (cont): - The second circuit, accepts as input the matrix A, and decides the CONN problem on G M,x. Since G M,x is a 2 O(s) circuit, and using the claim, then CONN problem can be decided on G M,x in depth O(s 2 ) and size 2 O(s). Overall, we obtain a circuit Cn in depth O(s 2 ) and size 2 O(s) s.t. Cn(x)=M(x). We actually proved NSPACE(s)  Depth(O(s 2 ))

25 Depth(d)  DSpace(d) Theorem: For any integer function d(  )  log(  ) Depth(d)  DSPACE(s) Proof: Given uniform family {Cn}, of depth d(n), we construct a DTM d(n)-space M, s.t.  x  {0,1} * M(x)=C |x| (X) The algorithm will be the composition of two algorithms, each using d(n) space. Lemma: Let M1, M2 be two s(n)-space Turing machines. Then, there exists an s(n)-space TM M that on input x outputs M2(M1(x)).

26 Depth(d)  DSpace(d) (2) The algorithm:(given input x  {0,1} n ) 1. Obtain a description of Cn List of gates and their predecessors The description may be exponential (as the number of gates is) 2. Evaluate Cn(x) The proof is presented by proving the following two claims: Claim: can be generated using O(d(n)) space. Proof:By uniformity of {Cn}, there exists a DTM M, s.t. M(1 n ) = (description of Cn), using log( ). Since | |  2 O(d(n)), M uses O(d(n)) space as required.

27 Depth(d)  DSpace(d) (3) Claim: Circuit evaluation for bounded fan-in circuits can be solved in space=O(circuit depth) Proof: Given circuit C of depth d and input x, we want to compute C(x). Our implementation is recursive: A natural recursion would be s.t. for every operation node op in the circuit we define Value(Cx,w) : - If the node is a leaf, simply return its value. - Value(Cx,op)= Value(Cx,v) op Value(Cx,u) where v and u are the predecessors of op. !! However, such recursion consumes space O(d 2 ) : There are O(d) recursion levels, where at each level we remember a vertex name which is also O(d) Assuming fan-in=2

28 Depth(d)  DSpace(d) (4) We represent each vertex by a path reaching it from the output vertex. - The output vertex is represented by , the empty string - Then, its right predecessor by 0, and the left one by 1 Consequently, each vertex is represented by a binary string of length O(d). Where its predecessors are achieved by concatenating 0 and 1 respectively! For a path representing a leaf or a node operation: - If path is a leaf then return its value. - Value(Cx,op) = Value(Cx, path o 1) op Value(Cx, path o 0) ! At each recursion level, path determines precisely all the previous recursion levels; Thus, space consumption is O(d).

29 Corollary We summarize the relation between Circuit depth and space complexity: NC 1  L  NL  NC 2