of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty1 ( Deterministic ) Communication amid Uncertainty Madhu Sudan Microsoft, New England Based on joint works with: (1) Adam Kalai (MSR), Sanjeev Khanna (U.Penn), Brendan Juba (Harvard) and (2) Elad Haramaty (Technion)

of 24 Classical Communication 11/20/2012TIFR: Deterministic Communication Amid Uncertainty2

of 24 Outline Part 1: Motivation Part 1: Motivation Part 2: Formalism Part 2: Formalism Part 3: Randomized Solution Part 3: Randomized Solution Part 4: Issues with Randomized Solution Part 4: Issues with Randomized Solution Part 5: Deterministic Issues. Part 5: Deterministic Issues. 11/20/2012TIFR: Deterministic Communication Amid Uncertainty3

of 24 Motivation: Human Communication Human communication (dictated by languages, grammars) very different from Shannon setting. Human communication (dictated by languages, grammars) very different from Shannon setting. Grammar: Rules, often violated. Grammar: Rules, often violated. Dictionary: Often multiple meanings to a word. Dictionary: Often multiple meanings to a word. Redundant: But not as in any predefined way (not an error-correcting code). Redundant: But not as in any predefined way (not an error-correcting code). Theory? Theory? Information theory? Information theory? Linguistics? (Universal grammars etc.)? Linguistics? (Universal grammars etc.)? 11/20/2012TIFR: Deterministic Communication Amid Uncertainty4

of 24 Behavioral aspects of natural communication (Vast) Implicit context. (Vast) Implicit context. Sender sends increasingly long messages to receiver till receiver “gets” (the meaning of) the message. Sender sends increasingly long messages to receiver till receiver “gets” (the meaning of) the message. Where do the options come from? Where do the options come from? Sender may use feedback from receiver if available; or estimates receiver’s knowledge if not. Sender may use feedback from receiver if available; or estimates receiver’s knowledge if not. How does estimation influence message. How does estimation influence message. Language provides sequence of (increasingly) long ways to represent a message. Language provides sequence of (increasingly) long ways to represent a message. How? Why? What features are good/bad. How? Why? What features are good/bad. 11/20/2012TIFR: Deterministic Communication Amid Uncertainty5

of 24 Model: Reason to choose short messages: Compression. Reason to choose short messages: Compression. Channel is still a scarce resource; still want to use optimally. Channel is still a scarce resource; still want to use optimally. Reason to choose long messages (when short ones are available): Reducing ambiguity. Reason to choose long messages (when short ones are available): Reducing ambiguity. Sender unsure of receiver’s prior (context). Sender unsure of receiver’s prior (context). Sender wishes to ensure receiver gets the message, no matter what its prior (within reason). Sender wishes to ensure receiver gets the message, no matter what its prior (within reason). 11/20/2012TIFR: Deterministic Communication Amid Uncertainty6

of 24 Back to Problem 11/20/2012TIFR: Deterministic Communication Amid Uncertainty7

of 24 Contrast with some previous models Universal compression? Universal compression? Doesn’t apply: P,Q are not finitely specified. Doesn’t apply: P,Q are not finitely specified. Don’t have a sequence of samples from P; just one! Don’t have a sequence of samples from P; just one! K-L divergence? K-L divergence? Measures inefficiency of compressing for Q if real distribution is P. Measures inefficiency of compressing for Q if real distribution is P. But assumes encoding/decoding according to same distribution Q. But assumes encoding/decoding according to same distribution Q. Semantic Communication: Semantic Communication: Uncertainty of sender/receiver; but no special goal. Uncertainty of sender/receiver; but no special goal. 11/20/2012TIFR: Deterministic Communication Amid Uncertainty8

of 24 Closeness of distributions: 11/20/2012TIFR: Deterministic Communication Amid Uncertainty9

of 24 Dictionary = Shared Randomness? 11/20/2012TIFR: Deterministic Communication Amid Uncertainty10

of 24 Solution (variant of Arith. Coding) 11/20/2012TIFR: Deterministic Communication Amid Uncertainty11

of 24 Performance 11/20/2012TIFR: Deterministic Communication Amid Uncertainty12

of 24 Implications 11/20/2012TIFR: Deterministic Communication Amid Uncertainty13

of 24 Deterministic Compression? 11/20/2012TIFR: Deterministic Communication Amid Uncertainty14

of 24 Challenging special case 11/20/2012TIFR: Deterministic Communication Amid Uncertainty15

of 24 Model as a graph coloring problem 11/20/2012TIFR: Deterministic Communication Amid Uncertainty16X

of 24 Main Results [w. Elad Haramaty] 11/20/2012TIFR: Deterministic Communication Amid Uncertainty17

of 24 Restricted Uncertainty Graphs 11/20/2012TIFR: Deterministic Communication Amid Uncertainty18X

of 24 Homomorphisms 11/20/2012TIFR: Deterministic Communication Amid Uncertainty19

of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty20

of 24 Better upper bounds: 11/20/2012TIFR: Deterministic Communication Amid Uncertainty21

of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty22 Better upper bounds:

of 24 Future work? 11/20/2012TIFR: Deterministic Communication Amid Uncertainty23

of 24 Thank You 11/20/2012TIFR: Deterministic Communication Amid Uncertainty24