# 1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,

## Presentation on theme: "1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,"— Presentation transcript:

1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov, and Omri Weinstein

Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost 2

Background – information theory Shannon (1948) introduced information theory as a tool for studying the communication cost of transmission tasks. 3 communication channel Alice Bob

Shannon’s entropy 4 communication channel X

Shannon’s noiseless coding 5

Shannon’s entropy – cont’d communication channel X Y

A simple example 7 Easy and complete!

Communication complexity [Yao] Focus on the two party randomized setting. 8 A B X Y F(X,Y) Meanwhile, in a galaxy far far away… Shared randomness R

Communication complexity A B X Y F(X,Y) m 1 (X,R) m 2 (Y,m 1,R) m 3 (X,m 1,m 2,R) Communication cost = #of bits exchanged. Shared randomness R

Communication complexity Numerous applications/potential applications (streaming, data structures, circuits lower bounds…) Considerably more difficult to obtain lower bounds than transmission (still much easier than other models of computation). Many lower-bound techniques exists. Exact bounds?? 10

Communication complexity 11

Set disjointness and intersection

Information complexity 13

Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

Mutual information 15 H(A) H(B) I(A,B)

Basic definition 1: The information cost of a protocol A B X Y Protocol π what Alice learns about Y + what Bob learns about X

Example A B X Y what Alice learns about Y + what Bob learns about X MD5(X) [128 bits] X=Y? [1 bit]

Information complexity 18

Prior-free information complexity 19

Connection to privacy 20

Information equals amortized communication 21

Without priors 22

Intersection 23

The two-bit AND 24

The optimal protocol for AND A B X  {0,1} Y  {0,1} If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached”

The optimal protocol for AND A B If X=1, A=1 If X=0, A=U [0,1] If Y=1, B=1 If Y=0, B=U [0,1] 0 1 “Raise your hand when your number is reached” X  {0,1} Y  {0,1}

Analysis 27

The analytical view A message is just a mapping from the current prior to a distribution of posteriors (new priors). Ex: 28 Y=0Y=1 X=00.40.2 X=10.30.1 Y=0Y=1 X=02/31/3 X=100 Y=0Y=1 X=000 X=10.750.25 Alice sends her bit “0”: 0.6 “1”: 0.4

The analytical view 29 Y=0Y=1 X=00.40.2 X=10.30.1 Y=0Y=1 X=00.5450.273 X=10.1360.045 Y=0Y=1 X=02/91/9 X=11/21/6 Alice sends her bit w.p ½ and unif. random bit w.p ½. “0”: 0.55 “1”: 0.45

Analytical view – cont’d 30

IC of AND 31

*Not a real protocol 32

Previous numerical evidence [Ma,Ishwar’09] – numerical calculation results. 33

Applications: communication complexity of intersection 34

Applications 2: set disjointness 35

A hard distribution? 36 001101000100111101100 101001110011101011000 Y=0Y=1 X=01/4 X=11/4 Very easy!

A hard distribution 37 000101000100110101100 101000110011100010000 Y=0Y=1 X=01/3 X=11/3 At most one (1,1) location!

Communication complexity of Disjointness 38

Small-set Disjointness 39

Using information complexity Y=0Y=1 X=01-2k/nk/n X=1k/n

Overview: information complexity Information complexity :: communication complexity as Shannon’s entropy :: transmission cost Today: focused on exact bounds using IC. 41

Selected open problems 1

Interactive compression? 43

Interactive compression? 44

Selected open problems 2 45

External information cost A B X Y Protocol π C

External information complexity 47

48 Thank You!

Download ppt "1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,"

Similar presentations