Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,

Similar presentations


Presentation on theme: "1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,"— Presentation transcript:

1 1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov, and Omri Weinstein

2 Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost 2

3 Background – information theory Shannon (1948) introduced information theory as a tool for studying the communication cost of transmission tasks. 3 communication channel Alice Bob

4 Shannon’s entropy 4 communication channel X

5 Shannon’s noiseless coding 5

6 Shannon’s entropy – cont’d communication channel X Y

7 A simple example 7 Easy and complete!

8 Communication complexity [Yao] Focus on the two party randomized setting. 8 A B X Y F(X,Y) Meanwhile, in a galaxy far far away… Shared randomness R

9 Communication complexity A B X Y F(X,Y) m 1 (X,R) m 2 (Y,m 1,R) m 3 (X,m 1,m 2,R) Communication cost = #of bits exchanged. Shared randomness R

10 Communication complexity Numerous applications/potential applications (streaming, data structures, circuits lower bounds…) Considerably more difficult to obtain lower bounds than transmission (still much easier than other models of computation). Many lower-bound techniques exists. Exact bounds?? 10

11 Communication complexity 11

12 Set disjointness and intersection

13 Information complexity 13

14 Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

15 Mutual information 15 H(A) H(B) I(A,B)

16 Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

17 Example A B X Y what Alice learns about Y + what Bob learns about X MD5(X) [128 bits] X=Y? [1 bit]

18 Information complexity 18

19 Prior-free information complexity 19

20 Connection to privacy 20

21 Information equals amortized communication 21

22 Without priors 22

23 Intersection 23

24 The two-bit AND 24

25 The optimal protocol for AND A B X  {0,1} Y  {0,1} If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached”

26 The optimal protocol for AND A B If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached” X  {0,1} Y  {0,1}

27 Analysis 27

28 The analytical view A message is just a mapping from the current prior to a distribution of posteriors (new priors). Ex: 28 Y=0Y=1 X= X= Y=0Y=1 X=02/31/3 X=100 Y=0Y=1 X=000 X= Alice sends her bit “0”: 0.6 “1”: 0.4

29 The analytical view 29 Y=0Y=1 X= X= Y=0Y=1 X= X= Y=0Y=1 X=02/91/9 X=11/21/6 Alice sends her bit w.p ½ and unif. random bit w.p ½. “0”: 0.55 “1”: 0.45

30 Analytical view – cont’d 30

31 IC of AND 31

32 *Not a real protocol 32

33 Previous numerical evidence [Ma,Ishwar’09] – numerical calculation results. 33

34 Applications: communication complexity of intersection 34

35 Applications 2: set disjointness 35

36 A hard distribution? Y=0Y=1 X=01/4 X=11/4 Very easy!

37 A hard distribution Y=0Y=1 X=01/3 X=11/3 At most one (1,1) location!

38 Communication complexity of Disjointness 38

39 Small-set Disjointness 39

40 Using information complexity Y=0Y=1 X=01-2k/nk/n X=1k/n

41 Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost Today: focused on exact bounds using IC. 41

42 Selected open problems 1

43 Interactive compression? 43

44 Interactive compression? 44

45 Selected open problems 2 45

46 External information cost A B X Y Protocol π C

47 External information complexity 47

48 48 Thank You!


Download ppt "1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,"

Similar presentations


Ads by Google