1. Asymmetric Communication Complexity And its implications on Cell Probe Complexity Slides by Elad Verbin Based on a paper of Peter Bro Miltersen, Noam Nisan, Muli Safra and Avi Wigderson

2. Purpose We want to get Lower Bounds. Best known lower bounds: Sorting is Ω(nlogn) in the comparison model Sorting is Ω(nlogn) in the comparison model Trivial lower bounds. i.e. MAX is Ω(n) Trivial lower bounds. i.e. MAX is Ω(n) What can we really do, i.e. for RAM? What can we really do, i.e. for RAM?

3. Outline Yao decided to strengthen the model – Considered the Cell Probe model. Yao decided to strengthen the model – Considered the Cell Probe model. Lower bounding Cell Probe is hard too. We strengthen even more – Communication Complexity Lower bounding Cell Probe is hard too. We strengthen even more – Communication Complexity

4. Outline We show the relationship between Cell Probe and Communication Complexity. We show the relationship between Cell Probe and Communication Complexity. We show how to get lower bounds for Communication Complexity using two techniques: We show how to get lower bounds for Communication Complexity using two techniques: 1. The Richness Technique 2. The Round Elimination Technique

5. Communication Complexity The problem : f:X  Y  {0,1} The problem : f:X  Y  {0,1} Alice gets x  X, Bob gets y  Y, their goal is to exchange messages to decide f(x,y). Alice gets x  X, Bob gets y  Y, their goal is to exchange messages to decide f(x,y). A solution is a communication protocol that can compute f(x,y) for all x,y. A solution is a communication protocol that can compute f(x,y) for all x,y. f(x,y)

6. Asymmetric Communication Complexity A Communication Protocol that computes function f in which Alice sends at most a bits and Bob sends at most b bits is called a [a,b]-protocol for f. A Communication Protocol that computes function f in which Alice sends at most a bits and Bob sends at most b bits is called a [a,b]-protocol for f. f(x,y) Pink<=aBlue<=b

7. Randomized Protocols If the protocol is allowed to flip (public) coins, and gives the correct answer with probability > 2/3 it is called a randomized protocol. If the protocol is allowed to flip (public) coins, and gives the correct answer with probability > 2/3 it is called a randomized protocol. If it always correctly identifies a 0-instance it is called a one-sided error protocol. If it always correctly identifies a 0-instance it is called a one-sided error protocol.

8. Example For any problem, there are trivial deterministic protocols: For any problem, there are trivial deterministic protocols: [log|X|,1]-protocol [log|X|,1]-protocol [1,log|Y|]-protocol. [1,log|Y|]-protocol.

9. The Problem DISJ(N,k,l) We work on the universe U={0,1,…,N-1} We work on the universe U={0,1,…,N-1} Alice gets x, a set of k elements Alice gets x, a set of k elements Bob gets y, a set of l elements Bob gets y, a set of l elements They must decide if x∩y=Ø They must decide if x∩y=Ø ∩=Ø x∩y=Ø? x y

10. A one-sided error randomized [O(k),O(l)]-protocol for DISJ(N,k,l) If we say that x and y are disjoint then we want to have complete confidence. If we say they intersect, we want to be reasonably certain. If we say that x and y are disjoint then we want to have complete confidence. If we say they intersect, we want to be reasonably certain. Flip public coins to get a sequence of random subsets of the universe: R 1, R 2, … Flip public coins to get a sequence of random subsets of the universe: R 1, R 2, …

11. R3R3 x y

12. R3R3 x y y=y∩R 3 y

13. R3R3 x y y R8R8 x=x∩R 8 AND SO ON….

14. A one-sided error randomized [O(k),O(l)]-protocol for DISJ(N,k,l) We don’t really send the index of R 8, we just send the distance from the last set (8-3=5). This means that the expected numbers of bits sent by a player is equal to the size of his set We don’t really send the index of R 8, we just send the distance from the last set (8-3=5). This means that the expected numbers of bits sent by a player is equal to the size of his set If at some point one of the sets becomes empty, then the originals were disjoint – say so. Otherwise, after a long time, say that there is an intersection. If at some point one of the sets becomes empty, then the originals were disjoint – say so. Otherwise, after a long time, say that there is an intersection.

15. A one-sided error randomized [O(k),O(l)]-protocol for DISJ(N,k,l) If x and y were indeed disjoint, the sizes of x and y decrease by a factor of 2 each round. Therefore the total communication is [O(k),O(l)]. If x and y were indeed disjoint, the sizes of x and y decrease by a factor of 2 each round. Therefore the total communication is [O(k),O(l)]. If the sets were disjoint, what is the chance that we say that there is an intersection? Very low. If the sets were disjoint, what is the chance that we say that there is an intersection? Very low.

16. Fixed-round protocols If t alternating messages are sent and each message is of size a or b it is called a [t,a,b]-protocol. If t alternating messages are sent and each message is of size a or b it is called a [t,a,b]-protocol. f(x,y) a a b b t

17. Static Data Structure Problems A static data structure problem is a function f:D  Q  R A static data structure problem is a function f:D  Q  R D – the data D – the data Q – the queries Q – the queries R – Possible answers. Typically, R={0,1} R – Possible answers. Typically, R={0,1} query Data DS

18. The Problem MEMBERSHIP(N) INPUT: a set S  [N] INPUT: a set S  [N] QUERIES: of the form “x  S?” QUERIES: of the form “x  S?” D={S  [N]}, |D|=2 N D={S  [N]}, |D|=2 N Q=[N] Q=[N] R={0,1} R={0,1} The trivial solution is optimal. The trivial solution is optimal.

19. The Problem MEMBERSHIP(N,n) INPUT: a set S  [N] of size n INPUT: a set S  [N] of size n QUERIES: of the form “x  S?” QUERIES: of the form “x  S?” D={S  [N] | |S|=n}, |D|=choose(N,n) D={S  [N] | |S|=n}, |D|=choose(N,n) Q=[N] Q=[N] R={0,1} R={0,1}

20. The Cell Probe Model Parameter w – word size Parameter w – word size s cells, each containing w bits. s cells, each containing w bits. Each query probes at most t cells to get answer Each query probes at most t cells to get answer A query is a decision tree of depth t and degree 2 w A query is a decision tree of depth t and degree 2 w

21. MEMBERSHIP(N,n) Solutions: Keep every possible answer. s=N, t=1 (better – s=N/w, t=1) Keep every possible answer. s=N, t=1 (better – s=N/w, t=1) Keep a nonredundant representation. s=log(choose(N,n)), t=log(choose(N,n)) Keep a nonredundant representation. s=log(choose(N,n)), t=log(choose(N,n))

22. MEMBERSHIP(N,n) Solutions: Keep a sorted list of all elements. s=nlog(N)/w, t=log(n)*log(N)/w Keep a sorted list of all elements. s=nlog(N)/w, t=log(n)*log(N)/w There is a randomized solution with s=(n/w) c, t=O(1), for some constant c. There is a randomized solution with s=(n/w) c, t=O(1), for some constant c.

23. What is the connection between Cell Probe and Asymmetric Communication Complexity?

24. ACC Cell Probe The communication problem related to a static data structure problem f:D  Q  {0,1} if the problem where Alice gets a query, Bob gets the data, and they should decide if this is a “yes” instance or a “no” instance The communication problem related to a static data structure problem f:D  Q  {0,1} if the problem where Alice gets a query, Bob gets the data, and they should decide if this is a “yes” instance or a “no” instance

25. MEM(N,n) Communication Problem MEM(N,n) Alice gets x  [N], Bob gets y  [N], |y|=n, they should decide if x  y. Alice gets x  [N], Bob gets y  [N], |y|=n, they should decide if x  y. Trivial protocols: [1,nlogN], [logN,1] Trivial protocols: [1,nlogN], [logN,1]

26. Lemma CP->AAC If there is a solution to the data structure problem with word size w taking s cells and with query time t, then there is a [2t,log(s),w]-protocol for the communication problem If there is a solution to the data structure problem with word size w taking s cells and with query time t, then there is a [2t,log(s),w]-protocol for the communication problem Therefore a lower bound on ACC gives us a lower bound on Cell Probe Therefore a lower bound on ACC gives us a lower bound on Cell Probe

27. Finer points of CP->AAC How is the communication complexity model stronger than the Cell Probe Model? How is the communication complexity model stronger than the Cell Probe Model? Answer: In its adaptivity Answer: In its adaptivity Which form of Cell Probe lower bounds can we get from the CP->AAC Lemma? Which form of Cell Probe lower bounds can we get from the CP->AAC Lemma? Answer: the bound on space is up to a polynomial Answer: the bound on space is up to a polynomial

28. Restricted AAC->CP If there is a [O(1),a,b]-protocol for the communication problem then the data structure problem has a solution with word size w=b, t=O(1) and s=2 O(a) If there is a [O(1),a,b]-protocol for the communication problem then the data structure problem has a solution with word size w=b, t=O(1) and s=2 O(a) Proof: The Data Structure for input y contains the message Bob should send next for every possible history of messages Alice can send, for any query. Proof: The Data Structure for input y contains the message Bob should send next for every possible history of messages Alice can send, for any query.

29. Lower Bounding The Communication Complexity

30. MEM(N,l) Communication Problem MEM(N,l) Alice gets x  [N], Bob gets y  [N], |y|=l, they should decide if x  y. Alice gets x  [N], Bob gets y  [N], |y|=l, they should decide if x  y. NONMEM(N,l) is the same problem, when Alice and Bob want to decide if x  y NONMEM(N,l) is the same problem, when Alice and Bob want to decide if x  y Trivial protocols: [1,l*logN], [logN,1] Trivial protocols: [1,l*logN], [logN,1]

31. Problem Matrix We identify a communication problem f:X×Y  {0,1} with a |X|×|Y| Matrix where M[x][y]=f(x,y). We identify a communication problem f:X×Y  {0,1} with a |X|×|Y| Matrix where M[x][y]=f(x,y). The matrix of NONMEM(N,l) has N rows and columns. Each column has N-l 1- entries The matrix of NONMEM(N,l) has N rows and columns. Each column has N-l 1- entries

32. Problem Matrix A problem (matrix) is (u,v)-rich if at least v columns contain at least u 1-entries. A problem (matrix) is (u,v)-rich if at least v columns contain at least u 1-entries. NONMEM(N,l) is (N-l, )-rich. NONMEM(N,l) is (N-l, )-rich. (4,3)-rich

33. The Richness Lemma Let f be a communication problem that: Let f be a communication problem that: 1. is (u,v)-rich 2. has a randomized one-sided error [a,b]- protocol. Then f contains a u/2 a+2 over Then f contains a u/2 a+2 over v/2 a+b+2 submatrix of 1-entries. v/2 a+b+2 submatrix of 1-entries.

34. Randomized Lower Bound for MEM(N,l) Say MEM(N,l) has a negative-one-sided error [a,b]-protocol. Let a<log(l), l<N/2. Say MEM(N,l) has a negative-one-sided error [a,b]-protocol. Let a<log(l), l<N/2. Then NONMEM(N,l) has a one-sided error [a,b]-protocol Then NONMEM(N,l) has a one-sided error [a,b]-protocol

35. NONMEM(N,l) is (N-l, )-rich NONMEM(N,l) is (N-l, )-rich Therefore it has a 1-submatrix of dimensions at least (N-l)/2 a+2 over Therefore it has a 1-submatrix of dimensions at least (N-l)/2 a+2 over /2 a+b+2 /2 a+b+2 Randomized Lower Bound for NONMEM(N,l)

36. However, if there is a 1-submatrix of dimensions r on s then s≤ However, if there is a 1-submatrix of dimensions r on s then s≤ By substituting for s and r, simplifying and bounding we get 2 a (a+b)=Ω(l) By substituting for s and r, simplifying and bounding we get 2 a (a+b)=Ω(l) Randomized Lower Bound for NONMEM(N,l)

37. Randomized [O(a),O(l/2 a )] Upper Bound for NONMEM(N,l) On the other hand, NONMEM(N,l) has a [O(a),O(l/2 a )]-protocol, for all a<log(l): On the other hand, NONMEM(N,l) has a [O(a),O(l/2 a )]-protocol, for all a<log(l): Alice sends Bob the first a indices of R’s that contain x. This allows Bob to reduce y to expected size l/2 a. Alice sends Bob the first a indices of R’s that contain x. This allows Bob to reduce y to expected size l/2 a. Then Bob sends a couple indices that contain y. Then Bob sends a couple indices that contain y. If we are not yet sure that they are disjoint, we say that they intersect. If we are not yet sure that they are disjoint, we say that they intersect.

38. Tightness for NONMEM(N,l) NONMEM(N,l) has a [O(a),O(l/2 a )]-protocol, for all a<log(l) NONMEM(N,l) has a [O(a),O(l/2 a )]-protocol, for all a<log(l) 2 a (a+b)= 2 a (a+l/2 a )= ? O(l) Therefore the last result is tight? 2 a (a+b)= 2 a (a+l/2 a )= ? O(l) Therefore the last result is tight? There are constants c,c’>0, so that for any a, There are constants c,c’>0, so that for any a, b=l/2 ca is enough. b=l/2 ca is enough. b=l/2 c’a is not enough. b=l/2 c’a is not enough.

39. The Richness Lemma Let f be a communication problem that: Let f be a communication problem that: 1. is (u,v)-rich 2. has a randomized one-sided error [a,b]- protocol. Then f contains a u/2 a+2 over Then f contains a u/2 a+2 over v/2 a+b+2 submatrix of 1-entries. v/2 a+b+2 submatrix of 1-entries.

40. Proof of the Richness Lemma First let us prove a weaker result: if f has a deterministic [a,b]-protocol then it contains a contains a u/2 a over First let us prove a weaker result: if f has a deterministic [a,b]-protocol then it contains a contains a u/2 a over v/2 a+b submatrix of 1-entries. v/2 a+b submatrix of 1-entries. We prove this by induction on a+b: We prove this by induction on a+b:

41. Proof of the Richness Lemma For a+b=0 – |X| ≥u, |Y|≥v, and f(x,y)=1 for all x,y, so this is trivial. For a+b=0 – |X| ≥u, |Y|≥v, and f(x,y)=1 for all x,y, so this is trivial. Now, if Alice send the first bit: Now, if Alice send the first bit: X 0 – inputs for which she sends 0 X 0 – inputs for which she sends 0 X 1 – inputs for which she sends 1 X 1 – inputs for which she sends 1 Let f 0, f 1 be the restrictions of f to X 0  Y, X 1  Y. Let f 0, f 1 be the restrictions of f to X 0  Y, X 1  Y.

42. Proof of the Richness Lemma At least one of them is (u/2,v/2)-rich, and both have a [a-1,b] protocol. At least one of them is (u/2,v/2)-rich, and both have a [a-1,b] protocol. By the induction it contains a (u/2)/2 a-1 over (v/2)/2 a+b-1 1-submatrix. By the induction it contains a (u/2)/2 a-1 over (v/2)/2 a+b-1 1-submatrix. In the other case, Bob send the first bit. In the other case, Bob send the first bit. Define Y 0,Y 1,f 0,f 1. At least one of them is (u,v/2)-rich, and proceed similarly. Define Y 0,Y 1,f 0,f 1. At least one of them is (u,v/2)-rich, and proceed similarly.

43. Proof of the Richness Lemma Now let us prove the general case: Now let us prove the general case: Let S be the set of u*v rich-positions in the matrix Let S be the set of u*v rich-positions in the matrix Let us look at some coin-flip sequence. Let us look at some coin-flip sequence.

44. Proof of the Richness Lemma Let X = #{1s in S} Let X = #{1s in S} E[X]>=2/3 * uv E[X]>=2/3 * uv => There exists such a sequence for which X>=2/3 * uv => There exists such a sequence for which X>=2/3 * uv Fix the sequence, to get a deterministic algorithm. This algorithm computes a function f’ that is close to f. Fix the sequence, to get a deterministic algorithm. This algorithm computes a function f’ that is close to f.

45. Proof of the Richness Lemma By a counting argument, f’ is (u/4,v/4)-rich, and so it has a 1-submatrix of the required size ( u/2 a+2 over v/2 a+b+2 ) By a counting argument, f’ is (u/4,v/4)-rich, and so it has a 1-submatrix of the required size ( u/2 a+2 over v/2 a+b+2 ) This is a 1-submatrix in f too, because the error is one-sided. This is a 1-submatrix in f too, because the error is one-sided. Q.E.D. Q.E.D.

46. A Richness Results for two-sided error Let d,e>0, and let f:X×Y  {0,1} be a communication problem with at least a d- fraction of 1s. If f has a randomized two- sided error [a,b]-protocol then f has a submatrix M of dimensions at least |X|/2 O(a) over |Y|/2 O(a+b) with at least a (1-e)-fraction of 1s. Let d,e>0, and let f:X×Y  {0,1} be a communication problem with at least a d- fraction of 1s. If f has a randomized two- sided error [a,b]-protocol then f has a submatrix M of dimensions at least |X|/2 O(a) over |Y|/2 O(a+b) with at least a (1-e)-fraction of 1s.

47. The SPAN(n) Problem In SPAN, Alice gets x  {0,1} n and Bob gets a vector subspace y  {0,1} n In SPAN, Alice gets x  {0,1} n and Bob gets a vector subspace y  {0,1} n y can be represented using a basis of k≤n vectors – O(n 2 ) bits y can be represented using a basis of k≤n vectors – O(n 2 ) bits Alice and Bob must decide if x ∈ y. Alice and Bob must decide if x ∈ y. Trivial Protocols:[n,1], [1,n 2 ] Trivial Protocols:[n,1], [1,n 2 ]

48. Lower bounds for SPAN Let’s prove that in any [a,b] randomized one-sided error protocol for SPAN, either a=Ω(n), or b=Ω(n 2 ) Let’s prove that in any [a,b] randomized one-sided error protocol for SPAN, either a=Ω(n), or b=Ω(n 2 ) We will assume that y is of dimension n/2. We will assume that y is of dimension n/2. We will prove that: We will prove that: 1. SPAN is (2 n/2,2 n 2 /4 )-rich, and 2. SPAN does not contain a 1-submatrix of dimensions 2 n/3 over 2 n 2 /12

49. SPAN is (2 n/2,2 n 2 /4 )-rich Each subspace contains exactly 2 n/2 vectors => each column contains 2 n/2 1s. Each subspace contains exactly 2 n/2 vectors => each column contains 2 n/2 1s. How many subspaces of dimension n/2 are there? How many subspaces of dimension n/2 are there? Lets choose a basis: we have 2 n -1 possibilities for the first vector, 2 n -2 for the second, 2 n -4 for the third, etc. Lets choose a basis: we have 2 n -1 possibilities for the first vector, 2 n -2 for the second, 2 n -4 for the third, etc.

50. SPAN is (2 n/2,2 n 2 /4 )-rich We chose each basis (n/2)! times We chose each basis (n/2)! times How many basis does a subspace has? How many basis does a subspace has? We have 2 n/2 -1 options to choose the first vector, 2 n/2 -2 for the second, etc. We have 2 n/2 -1 options to choose the first vector, 2 n/2 -2 for the second, etc. We again chose each basis (n/2)! times. We again chose each basis (n/2)! times. Thus, there are at least 2 n 2 /4 subspaces of dimension n/2. Thus, there are at least 2 n 2 /4 subspaces of dimension n/2.

51. SPAN does not contain a 1-submatrix of dimensions 2 n/3 over 2 n 2 /12 Lets look at a 1-submatrix with at least 2 n/3 rows. The subspace spanned by them is of dimension at least n/3. Lets look at a 1-submatrix with at least 2 n/3 rows. The subspace spanned by them is of dimension at least n/3. How many subspaces of dimension n/2 can include this entire subspace? How many subspaces of dimension n/2 can include this entire subspace? 2 n 2 /12 2 n 2 /12 And we’re done. And we’re done.

52. The Round Elimination Lemma

53. f->P m (f) f->P m (f) Let f:X×Y->{0,1} be a communication problem Let f:X×Y->{0,1} be a communication problem P m (f) is: P m (f) is: Alice gets m elements from X, x 1, …, x m Alice gets m elements from X, x 1, …, x m Bob gets 1≤i≤m, y  Y and also x 1, …, x i-1 Bob gets 1≤i≤m, y  Y and also x 1, …, x i-1 They want to compute f(x i,y) They want to compute f(x i,y) How meaningful can Alice’s first message be? How meaningful can Alice’s first message be?

54. The Round Elimination Lemma Let C=99, R=4256. Let C=99, R=4256. Say that P Ra (f) has a randomized two- sided error [t,a,b]-protocol in which Alice sends the first message. Say that P Ra (f) has a randomized two- sided error [t,a,b]-protocol in which Alice sends the first message. Then there is a randomized two-sided error [t-1,Ca,Cb]-protocol with Bob sending the first message. Then there is a randomized two-sided error [t-1,Ca,Cb]-protocol with Bob sending the first message.

55. General framework for LB proofs using the Round Elimination Lemma [t,a,b]-protocol for F(n) [t,a,b]-protocol for P m (F(n’)) (typically n’=n/m) [t-1,Ca,Cb]-protocol for F(n’) [1,Ct-1a,Ct-1b]-protocol for F(n (t-1) )

56. The problem GT(n) Alice and Bob each gets an n-bit integer. Alice and Bob each gets an n-bit integer. They want to decide if x<y. They want to decide if x<y. x<y? xy

57. The problem GT(n) Deterministic communication complexity is linear Deterministic communication complexity is linear Randomized comm. complexity with two- sided error is O(logn) (using a logarithmic number of rounds) Randomized comm. complexity with two- sided error is O(logn) (using a logarithmic number of rounds) When limited to t rounds: There is a [t,n 1/t logn,n 1/t logn]-protocol When limited to t rounds: There is a [t,n 1/t logn,n 1/t logn]-protocol

58. GT(n) does not have a [t,n 1/t C -t,n 1/t C -t ]-protocol Theorem: Let C=99. Theorem: Let C=99. There does not exist a randomized two- sided error [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n) There does not exist a randomized two- sided error [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n)

59. GT(n) does not have a [t,n 1/t C -t,n 1/t C -t ]-protocol By induction on t By induction on t Say there was a [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n). Say there was a [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n). Then there is a [t,n 1/t C -t,n 1/t C -t ]-protocol for P m (GT(n’)) for m=n 1/t, n’=n (t-1)/t Then there is a [t,n 1/t C -t,n 1/t C -t ]-protocol for P m (GT(n’)) for m=n 1/t, n’=n (t-1)/t From the round Elimination Lemma, there is a [t-1, n 1/t C -(t-1),n 1/t C -(t-1) ]-protocol for GT(n’). Contradiction. From the round Elimination Lemma, there is a [t-1, n 1/t C -(t-1),n 1/t C -(t-1) ]-protocol for GT(n’). Contradiction.

60. GT(n) does not have a [t,n 1/t C -t,n 1/t C -t ]-protocol [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n) [t,n 1/t C -t,n 1/t C -t ]-protocol for GT(n) [t,n 1/t C -t,n 1/t C -t ]-protocol for P m (GT(n’)) for m=n 1/t, n’=n (t-1)/t [t,n 1/t C -t,n 1/t C -t ]-protocol for P m (GT(n’)) for m=n 1/t, n’=n (t-1)/t Alice constructs a n-bit integer x’: She concatenates x 1 …x m Alice constructs a n-bit integer x’: She concatenates x 1 …x m Bob constructs a n-bit integer y’: He concatenates x 1 …x i-1 then y and the rest is 1s Bob constructs a n-bit integer y’: He concatenates x 1 …x i-1 then y and the rest is 1s We get x’>y’  x i >y We get x’>y’  x i >y

61. THE END