Presentation is loading. Please wait.

Presentation is loading. Please wait.

Random projections and depth hierarchy theorems

Similar presentations


Presentation on theme: "Random projections and depth hierarchy theorems"— Presentation transcript:

1 Random projections and depth hierarchy theorems
Rocco Servedio Columbia University Title: Two circuit lower bounds We present two recent lower bounds on small-depth circuits. The first lower bound deals with shallow monotone circuits of Majority gates (unweighted threshold gates). We show that for any d, a monotone depth-d circuit of Majority gates needs size 2^{n^{1/d}} to compute a particular explicit (monotone) weighted threshold function. This answers questions of Goldmann and Karpinski (1993) and Hastad (2010, 2014), and can be viewed as a step towards showing that monotone TC0 is not contained in monotone NC1. The second lower bound gives a near-optimal size-depth tradeoff for circuits computing the “distance-k connectivity function” on graphs. Our lower bound here extends results of Beame, Impagliazzo and Pitassi (1993) and Rossman (2014) and may be viewed as giving evidence towards the optimality of Savitch’s small-space algorithm for directed s-t connectivity. This lower bound is proved via “random projections”, an extension of the classic technique of random restrictions that has recently proved useful in other contexts in circuit complexity. Joint work with Xi Chen, Igor Oliveira, and Li-Yang Tan. St. Petersburg, Russia May 2016

2 Two small-depth-circuit lower bounds
Rocco Servedio Columbia University Title: Two circuit lower bounds We present two recent lower bounds on small-depth circuits. The first lower bound deals with shallow monotone circuits of Majority gates (unweighted threshold gates). We show that for any d, a monotone depth-d circuit of Majority gates needs size 2^{n^{1/d}} to compute a particular explicit (monotone) weighted threshold function. This answers questions of Goldmann and Karpinski (1993) and Hastad (2010, 2014), and can be viewed as a step towards showing that monotone TC0 is not contained in monotone NC1. The second lower bound gives a near-optimal size-depth tradeoff for circuits computing the “distance-k connectivity function” on graphs. Our lower bound here extends results of Beame, Impagliazzo and Pitassi (1993) and Rossman (2014) and may be viewed as giving evidence towards the optimality of Savitch’s small-space algorithm for directed s-t connectivity. This lower bound is proved via “random projections”, an extension of the classic technique of random restrictions that has recently proved useful in other contexts in circuit complexity. Joint work with Xi Chen, Igor Oliveira, and Li-Yang Tan. St. Petersburg, Russia May 2016

3 The goal of circuit complexity
Goal: Strong lower bounds on size of unrestricted circuits computing explicit functions. Holy grail: size lower bound Unfortunately, we have no idea how to prove that an explicit n-variable Boolean function cannot be computed by unrestricted circuits of size 10n. Explicit function that requires any circuit size n^{\omega(1)}. We don’t know how to prove 10n lower bound. So, we look at restricted circuits. Bound depth Monotone circuits (for monotone functions) So, we look at restricted circuits: Small-depth circuits (both results of this talk) Monotone circuits (first result)

4 This talk: Two recent circuit lower bounds
1. Lower bound for monotone small-depth circuits of Maj gates that compute the “addition” function 2. Lower bound for small-depth circuits of AND/OR/NOT gates (non-monotone) computing “small-distance connectivity.” Similarities: Both are Lower bounds against small-depth circuits Near-optimal (or actually optimal) Differences: The problems (and proofs/techniques) are totally different.

5 Columbia/Charles University
Part I: Addition is exponentially harder than counting for shallow monotone circuits Joint work with Xi Chen Columbia Igor Oliveira Columbia/Charles University

6

7 Boolean circuit complexity
A circuit with AND/OR/NOT gates: (de Morgan circuit) Size = # of wires. Depth = depth. (Unbounded fan-in.) Can consider circuits with other types of gates as well instead of AND/OR: Maj circuit: can use gates that computes the majority function “weighted threshold” circuit: can use “LTF gates” -- gates that compute “weighted threshold” functions, i.e. that output 1 iff

8 Monotone circuits: come in different flavors
Monotone de Morgan circuits: No negations, only AND/OR gates Monotone Maj circuits: No negations, only Maj gates (and constants 0,1) Monotone weighted threshold circuits: No negative weights allowed: each LTF gate computes some function where each Maj

9 What do we know about monotone circuits?
A fair bit! Celebrated lower bounds: [Razborov85,Andreev85,AlonBoppana87,KarchmerWigderson88,Tardos88,RazWigderson92,HarnikRaz00,PitassiGoos14]: strong lower bounds on monotone AND/OR-circuit size and depth for explicit monotone functions [AjtaiGurevich87]: Explicit monotone function in poly(n)-size AC0 which does not have poly(n)-size monotone-AC0 circuits O(1)-depth, poly(n)-size AND, OR, NOT gates O(1)-depth, poly(n)-size Non-monotone O(1)-depth, poly(n)-size circuit that computes a monotone function that’s not computable by any AND, OR, gates only Monotone function

10 What else do we know about monotone circuits?
We also know non-trivial upper bounds: [AjtaiKomlosSzemeredi83,Valiant84]: has poly(n)-size monotone AND/OR formulas (tree circuits) And more.

11 But we don’t know everything…
Monotone-TC0: Constant-depth, poly(n)-size circuits of monotone weighted threshold gates Monotone-NC1: poly(n)-size monotone AND/OR formulas (= poly-size, log-depth, fan-in 2 monotone formulas) (= poly-size, log-depth, fan-in 2 monotone circuits) [Yao89,HastadGoldmann90]: Monotone-NC1 not contained in monotone-TC0 Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. log(n)-depth, O(n)-size not computed by any poly(n)-size, O(1)-depth circuit of monotone weighted threshold gates .... .... .... .... .... .... .... ....

12 Is the “univeral halfspace” computable in monotone-NC1 ?
A first question: Monotone-TC0: Constant-depth, poly(n)-size circuits of monotone weighted threshold gates Monotone-NC1: poly(n)-size monotone AND/OR formulas [Yao89, HastadGoldmann90]: Monotone-NC1 not contained in monotone-TC0. Question 1: Is monotone-TC0 contained in monotone-NC1 ? Equivalent to: Is the “univeral halfspace” computable in monotone-NC1 ? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Universal halfspace:

13 A close shave… Question 1: Is monotone-TC0 contained in monotone-NC1 ?
Monotone-TC0: Constant-depth, poly(n)-size circuits of monotone weighted threshold gates Monotone-NC1: poly(n)-size monotone AND/OR formulas [Yao89, HastadGoldmann90]: Monotone-NC1 not contained in monotone-TC0. Question 1: Is monotone-TC0 contained in monotone-NC1 ? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. [BeimelWeinreb05]: “almost Yes”: monotone-TC0 is contained in nO(log n)-size monotone-NC1. But…it is believed (at least by some people) that the answer is “No” – that nO(log n) is the smallest possible size for monotone formulas computing the universal halfspace.

14 A second question Recall [AjtaiGurevich87]: There is a monotone function in AC0 which does not have poly(n)-size monotone-AC0 circuits. O(1)-depth, poly(n)-size Monotone function in AC0 AND, OR, NOT gates AND, OR, gates only O(1)-depth, poly(n)-size not computable by any Question 2: Could there be a monotone function in AC0 which does not have poly(n)-size monotone circuits, period? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. AND, OR, gates only O(1)-depth, poly(n)-size AND, OR, NOT gates not computable by any poly(n)-size, any depth Monotone function in AC0

15 Question 1: Is monotone-TC0 contained in monotone-NC1 ?
both Question 1: Is monotone-TC0 contained in monotone-NC1 ? Question 2: Could there be a monotone function in AC0 which does not have poly(n)-size monotone circuits, period? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. This talk will answer of these questions! (But it describes partial progress on both of them.) neither

16 Question 1 revisited Question 1: Can monotone-NC1 compute monotone-TC0? Possible approach, if you are an optimist: Recall [AKS,Valiant]: monotone-NC1 can compute So if constant-depth monotone circuits of Maj gates can compute monotone-TC0, then monotone-NC1 circuits can compute monotone-TC0. (constant depth  only polynomial blowup in size when we compose.) Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. First main result: This won’t work.

17 } First result Define the function as follows:
The input is a list of –bit strings (which we view as binary #s ) The output is 1 iff Example: input is For this input, have } Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. 150 + 120 + = 473 > 256 203

18 Theorem 1: Any depth-d monotone Maj circuit computing must have size
First result is computed by a single monotone threshold gate: Gates “can count” Theorem 1: Any depth-d monotone Maj circuit computing must have size Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. “Addition is exponentially harder than counting for shallow monotone circuits”

19 Theorem 1: Any depth-d monotone Maj circuit computing must have size
Discussion Theorem 1: Any depth-d monotone Maj circuit computing must have size Theorem is tight in two ways: Any n-variable monotone weighted threshold function (such as ) can be computed by a poly-size depth-2 non-monotone Maj circuit [GoldmannHastadRazborov92, GoldmannKarpinski93, Hofmeister96, AmanoMaruoka05] can be computed by a depth-d, size monotone Maj circuit Theorem answers question of [GK93,Hastad10,Hastad14] [Hofmeister92] proved the d=2 case Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

20 Second question and result
Question 2: Could there be a monotone function in AC0 which does not have poly(n)-size monotone circuits? AND, OR, gates only O(1)-depth, poly(n)-size AND, OR, NOT gates not computable by any poly(n)-size, any depth Monotone function in AC0 Motivated by Theorem: [AjtaiGurevich87] There is an n-variable monotone function such that has a poly(n)-size, constant-depth AND/OR/NOT circuit; but Any constant-depth AND/OR circuit for has size nw(1) Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

21 The Ajtai-Gurevich function
n columns Input log n rows iff (# of 1s) > (n log n)/2 Theorem: [AjtaiGurevich87] The above monotone function has a poly(n)-size, constant-depth AND/OR/NOT circuit, but any depth-d AND/OR circuit for has size at least where Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

22 Second main result: Strengthening of Ajtai-Gurevich
Theorem: There is an variable monotone function such that has a poly(n)-size, depth-3 AND/OR/NOT circuit; Any depth-d monotone-Maj circuit computing has size at least Compare with [AjtaiGurevich87]: stronger size bound (exponential for every d) works against stronger circuits (Maj gates) very different proof (no switching lemma) Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

23 Proving Theorem 2 from Theorem 1
Theorem 1: Any depth-d monotone Maj circuit computing must have size Theorem 2: There is an variable monotone function such that has a poly(n)-size, depth-3 AND/OR/NOT circuit; Any depth-d monotone Maj-circuit computing has size at least Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. The function is where

24 A glimpse at the proof of Theorem 1
painful A glimpse at the proof of Theorem 1 Theorem 1: Any depth-d Maj circuit computing must have size Proof is a induction. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Inductively construct distributions that are “hard” for deeper and deeper circuits. very careful

25 A glimpse at the proof of Theorem 1
Let Initial pair of distributions : both over Every has . Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Every has

26 Initial hardness: against depth-1 monotone MAJ circuits
Not hard to show: Any depth-1 monotone Maj circuit of size satisfies Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

27 Key step in the proof of Theorem 1
Key lemma: Suppose any depth monotone Maj circuit over of size satisfies Then any depth- monotone Maj circuit over of size satisfies where The case gives the theorem. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

28 To construct over from over
the proof actually needs three pairs of distributions: over over and over some other domain that’s intermediate between and Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Proof maintains very tight control over all three pairs of distributions.

29 Related questions: Average-case lower bounds?
Gil Kalai has asked several “uniform distribution” questions about related circuit classes: Question: Can monotone-TC0 approximate Rec-Maj-3? Question: (“approximate Ajtai-Gurevich”) Are there monotone functions in AC0 that can’t be approximated by monotone-AC0? Question: (“approximate Ajtai-Gurevich for TC0”) Are there monotone functions in TC0 that can’t be approximated by monotone-TC0?

30 An observation and a question
Theorem 1: Any depth-d monotone Maj circuit computing must have size Observation: “approximate” version of Theorem 1 does not hold already for depth-1 circuits of Maj gates, in a strong sense: Any n-variable monotone weighted threshold function is 0.99-approximated by a single Maj gate of size O(n) [DeDiakonikolasFeldmanS12]. Question: what about “d>1” version of the above? Can constant-depth monotone-Maj circuits approximate constant-depth monotone weighted threshold functions? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

31 End of Part I Questions?

32 Columbia/Charles University
Part II: Near-Optimal Small Depth Lower Bounds for Small-Distance Connectivity Joint work with Xi Chen Columbia Igor Oliveira Columbia/Charles University Li-Yang Tan TTI

33 Small-distance connectivity
The problem: STCONN(k(n)): Given an n-node graph G with two distinguished nodes s and t, does there exist an s-to-t path of length at most k(n)? At most k(n) edges on the path s t Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. (Don’t worry about whether G is directed or undirected.)

34 The lower bound question
Our question: Circuit complexity of STCONN(k(n)) – in fact, small-depth circuit complexity of STCONN(k(n)). What size is needed for a depth-d AND/OR/NOT circuit to compute STCONN(k(n))? Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. variables, one for each possible edge

35 Why ask this question? Strong motivation from study of space-bounded computation. Savitch’s algorithm for directed connectivity uses space. Any improvement gives A simple reduction shows: If there is any k=k(n)<n such that there are poly(n)-size, o(log k)-depth circuits for STCONN(k), then Savitch can be improved! Motivates study of small-depth circuits for STCONN(k). Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

36 Previously known bounds
Large gap. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Based on “repeated squaring” (Savitch)

37 Our result Theorem: For any and any any depth-
circuit for must have size Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

38 The current landscape of upper and lower bounds
Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Much smaller gap…and may not be closed anytime soon; getting here or here would imply

39 Our approach Two main ingredients:
Easy reduction that lets us get graphs out of the picture quickly and painlessly. Thanks to the reduction, it’s just a question about circuits. New lower bounds on size of depth-d circuits that compute a particular depth-d’ circuit (a sort of depth hierarchy), proved using random projections. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. d < d’

40 First ingredient: The reduction. From graphs to circuits
We’re interested in getting from node s to node t. Graphs correspond to bitstrings: presence/absense of an edge corresponds to variable being 1/0. s t Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

41 From graphs to circuits
How about OR? s t Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

42 (nothing fancy – use a fresh variable for each edge)
Easy to compose… s t Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. (nothing fancy – use a fresh variable for each edge)

43 From monotone read-once formulas to series-parallel graphs
Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

44 From monotone read-once formulas to series-parallel graphs
formula f graph Gf If f has r levels of AND gates with fanins a1,…,ar, then every shortest path in Gf has length a1…ar. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Given an assignment z, have f(z)=1 iff G(f,z) has an s-to-t path of length a1…ar.

45 Goodbye, graphs Goal: Size lower bound on depth-d circuits computing STCONN(k). Thanks to reduction, this has become Goal: Size lower bound on depth-d circuits computing a monotone read-once formula f in which (product of AND-gate fanins across all depths) = k. Let’s cook up an f for which we could try to prove such a size lower bound. Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

46 The formula we want Some requirements: “Skewed Sipser functions”:
Formula f better have depth > d (so it has a chance of being hard for circuits of depth d) Product of AND-gate fan-ins across all depths must = k “Skewed Sipser functions”: Every OR fan-in = w Every AND fan-in = u<<w d layers of ANDs (so k=ud) fan-in w fan-in u<w Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. ... ... ...

47 All that remains: a straight-up circuit lower bound
Skewed Sipser function has depth 2d and size n=(uw)d. Recall u=AND-fan-in, w=OR-fan-in, k=ud. fan-in w fan-in u<w ... ... ... To prove the desired lower bound for STCONN(k), suffices to show: Theorem: Any depth-d circuit computing skewed Sipser function must have size Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. a depth-(2d) circuit!

48 What this might remind you of: Hastad’s depth hierarchy theorem
[Hastad86] proved that depth-d circuits require exp(n1/d) size to compute the “unskewed” Sipser function of depth d+1. But there are important differences: Need size lower bound for circuits of half the depth (instead of one less) Need to handle “skewed” Sipser functions (small AND-fan-in) Indeed, if u = AND-fan-in = 2 (an interesting case!), circuits of depth (2d-1) can compute the depth-2d skewed Sipser function in fixed poly(n) size! Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

49 Hastad’s depth hierarchy theorem: proved using random restrictions
Theorem: Any depth-d circuit computing skewed Sipser function must have size a depth-(2d) circuit Hastad’s depth hierarchy theorem: proved using random restrictions Our theorem (above): proved using random projections Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

50 What’s a random projection?
Have n Boolean variables x1,…,xn. What’s a random projection? 1 xi (denoted *) Restriction: Each xi set to constant or “survives”: xi Projection: Each xi set to constant or new formal variable 1 yj xi where { y1,…,ym } are new formal variables Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Our proof: { y-variables } much smaller than { x-variables }. Distinct x-variables collide to same y-variable (depending on the structure of the skewed Sipser formula)

51 Structure of our argument
Requirements for our theorem: Skewed Sipser (depth 2d) retains structure after random projection Sequence of (d-1) projections each “peel off” two layers from skewed Sipser, but don’t “damage” it further Small AC0d circuits collapse to “simple” function under random projection Each sub-restriction collapses bottom depth-2 circuit to shallow decision tree (as usual) Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know. Key technical ingredient: random projection switching lemma

52 A peek at the random projection switching lemma
Unusual distribution over random projections Unusual (and delicate) requirements: the condition is essential (the conclusion provably does not hold for some for ) Define monotone threshold gate, monotone TC0: Yao: monotone TC0 can’t compute monotone NC1 Question: Can monotone NC1 compute monotone TC0? We (still) don’t know.

53 Conclusion First part of talk:
Small-depth circuits of low-weight monotone LTF gates (Maj gates) can’t efficiently simulate a single monotone high-weight LTF gate (careful inductive construction of “hard distributions”) Many “uniform distribution” questions for future work Second part of talk: Near-optimal lower bounds on size of small-depth circuits for “small-distance connectivity” (random projection switching lemma) What else can we do with random projections?

54 Thank you! Thank you!


Download ppt "Random projections and depth hierarchy theorems"

Similar presentations


Ads by Google