Download presentation

Presentation is loading. Please wait.

Published byIsrael Mossman Modified about 1 year ago

1
1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov, and Omri Weinstein

2
Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost 2

3
Background – information theory Shannon (1948) introduced information theory as a tool for studying the communication cost of transmission tasks. 3 communication channel Alice Bob

4
Shannon’s entropy 4 communication channel X

5
Shannon’s noiseless coding 5

6
Shannon’s entropy – cont’d communication channel X Y

7
A simple example 7 Easy and complete!

8
Communication complexity [Yao] Focus on the two party randomized setting. 8 A B X Y F(X,Y) Meanwhile, in a galaxy far far away… Shared randomness R

9
Communication complexity A B X Y F(X,Y) m 1 (X,R) m 2 (Y,m 1,R) m 3 (X,m 1,m 2,R) Communication cost = #of bits exchanged. Shared randomness R

10
Communication complexity Numerous applications/potential applications (streaming, data structures, circuits lower bounds…) Considerably more difficult to obtain lower bounds than transmission (still much easier than other models of computation). Many lower-bound techniques exists. Exact bounds?? 10

11
Communication complexity 11

12
Set disjointness and intersection

13
Information complexity 13

14
Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

15
Mutual information 15 H(A) H(B) I(A,B)

16
Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

17
Example A B X Y what Alice learns about Y + what Bob learns about X MD5(X) [128 bits] X=Y? [1 bit]

18
Information complexity 18

19
Prior-free information complexity 19

20
Connection to privacy 20

21
Information equals amortized communication 21

22
Without priors 22

23
Intersection 23

24
The two-bit AND 24

25
The optimal protocol for AND A B X {0,1} Y {0,1} If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached”

26
The optimal protocol for AND A B If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached” X {0,1} Y {0,1}

27
Analysis 27

28
The analytical view A message is just a mapping from the current prior to a distribution of posteriors (new priors). Ex: 28 Y=0Y=1 X= X= Y=0Y=1 X=02/31/3 X=100 Y=0Y=1 X=000 X= Alice sends her bit “0”: 0.6 “1”: 0.4

29
The analytical view 29 Y=0Y=1 X= X= Y=0Y=1 X= X= Y=0Y=1 X=02/91/9 X=11/21/6 Alice sends her bit w.p ½ and unif. random bit w.p ½. “0”: 0.55 “1”: 0.45

30
Analytical view – cont’d 30

31
IC of AND 31

32
*Not a real protocol 32

33
Previous numerical evidence [Ma,Ishwar’09] – numerical calculation results. 33

34
Applications: communication complexity of intersection 34

35
Applications 2: set disjointness 35

36
A hard distribution? Y=0Y=1 X=01/4 X=11/4 Very easy!

37
A hard distribution Y=0Y=1 X=01/3 X=11/3 At most one (1,1) location!

38
Communication complexity of Disjointness 38

39
Small-set Disjointness 39

40
Using information complexity Y=0Y=1 X=01-2k/nk/n X=1k/n

41
Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost Today: focused on exact bounds using IC. 41

42
Selected open problems 1

43
Interactive compression? 43

44
Interactive compression? 44

45
Selected open problems 2 45

46
External information cost A B X Y Protocol π C

47
External information complexity 47

48
48 Thank You!

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google