1. of 29 12/02/2013Purdue: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic Communication – Juba & S. (STOC 2008) -Goal-Oriented Communication – Goldreich, Juba & S. (JACM 2012) -Compression without a common prior … – Kalai, Khanna, Juba & S. (ICS 2011) -Efficient Semantic Communication with Compatible Beliefs – Juba & S. (ICS 2011) -Deterministic Compression with uncertain priors – Haramaty & S. (ITCS 2014)

2. of 29 Classical theory of communication Clean architecture for reliable communication. Clean architecture for reliable communication. Remarkable mathematical discoveries: Prob. Method, Entropy, (Mutual) Information Remarkable mathematical discoveries: Prob. Method, Entropy, (Mutual) Information Needs reliable encoder + decoder (two reliable computers). Needs reliable encoder + decoder (two reliable computers). 12/02/2013Purdue: Uncertainty in Communication2 Shannon (1948) AliceAlice BobBob EncoderEncoder DecoderDecoder

3. of 29 Uncertainty in Communication? Always has been a central problem: Always has been a central problem: But usually focusses on uncertainty introduced by the channel But usually focusses on uncertainty introduced by the channel Standard Solution: Standard Solution: Use error-correcting codes Use error-correcting codes Significantly: Significantly: Design Encoder/Decoder jointly Design Encoder/Decoder jointly Deploy Encoder at Sender, Decoder at Receiver Deploy Encoder at Sender, Decoder at Receiver 12/02/2013Purdue: Uncertainty in Communication3

4. of 29 New Era, New Challenges: Interacting entities not jointly designed. Interacting entities not jointly designed. Can’t design encoder+decoder jointly. Can’t design encoder+decoder jointly. Can they be build independently? Can they be build independently? Can we have a theory about such? Can we have a theory about such? Where we prove that they will work? Where we prove that they will work? Hopefully: Hopefully: YES YES And the world of practice will adopt principles. And the world of practice will adopt principles. 12/02/2013Purdue: Uncertainty in Communication4

5. of 29 Example 1 Printing in a new environment Printing in a new environment Say, you are visiting a new university. Say, you are visiting a new university. Printer is intelligent; so is your computer; Printer is intelligent; so is your computer; Can’t they figure out how to talk to each other? Can’t they figure out how to talk to each other? Problem (with current designs): Problem (with current designs): Computers need to know about the printer already to print on them. Computers need to know about the printer already to print on them. Why can’t they also figure out how future printers will work? Why can’t they also figure out how future printers will work? Uncertainty (about printers of the future). Uncertainty (about printers of the future). 12/02/2013Purdue: Uncertainty in Communication5

6. of 29 Example 2 Archiving data Archiving data Physical libraries have survived for 100s of years. Physical libraries have survived for 100s of years. Digital books have survived for five years. Digital books have survived for five years. Can we be sure they will survive for the next five hundred? Can we be sure they will survive for the next five hundred? Problem: Uncertainty of the future. Problem: Uncertainty of the future. What systems will prevail? What systems will prevail? Why aren’t software systems ever constant? Why aren’t software systems ever constant? Problem: Problem: When designing one system, it is uncertain what the other’s design is (or will be in the future)! When designing one system, it is uncertain what the other’s design is (or will be in the future)! 12/02/2013Purdue: Uncertainty in Communication6

7. of 29 Modelling uncertainty Classical Shannon Model 12/02/2013Purdue: Uncertainty in Communication7 A B Channel B2B2B2B2 AkAkAkAk A3A3A3A3 A2A2A2A2 A1A1A1A1 B1B1B1B1 B3B3B3B3 BjBjBjBj Semantic Communication Model New Class of Problems New challenges Needs more attention!

8. of 29 Nature of uncertainty 12/02/2013Purdue: Uncertainty in Communication8

9. of 29 12/02/2013Purdue: Uncertainty in Communication9 II: Compression under uncertain beliefs/priors

10. of 29 Motivation 12/02/2013Purdue: Uncertainty in Communication10

11. of 29 Role of Dictionary (/Grammar/Language) 12/02/2013Purdue: Uncertainty in Communication11

12. of 29 Context? In general complex notion … In general complex notion … What does sender know/believe What does sender know/believe What does receiver know/believe What does receiver know/believe Modifies as conversation progresses. Modifies as conversation progresses. Our abstraction: Our abstraction: Context = Probability distribution on potential “meanings”. Context = Probability distribution on potential “meanings”. Certainly part of what the context provides; and sufficient abstraction to highlight the problem. Certainly part of what the context provides; and sufficient abstraction to highlight the problem. 12/02/2013Purdue: Uncertainty in Communication12

13. of 29 The problem 12/02/2013Purdue: Uncertainty in Communication13

14. of 29 Closeness of distributions: 12/02/2013Purdue: Uncertainty in Communication14

15. of 29 Dictionary = Shared Randomness? 12/02/2013Purdue: Uncertainty in Communication15

16. of 29 Solution (variant of Arith. Coding) 12/02/2013Purdue: Uncertainty in Communication16

17. of 29 Performance 12/02/2013Purdue: Uncertainty in Communication17

18. of 29 Implications 12/02/2013Purdue: Uncertainty in Communication18

19. of 29 12/02/2013Purdue: Uncertainty in Communication19 III: Deterministic Communication Amid Uncertainty

20. of 29 A challenging special case 12/02/2013Purdue: Uncertainty in Communication20

21. of 29 Model as a graph coloring problem 12/02/2013Purdue: Uncertainty in Communication21X

22. of 29 Main Results [w. Elad Haramaty] 12/02/2013Purdue: Uncertainty in Communication22

23. of 29 Restricted Uncertainty Graphs 12/02/2013Purdue: Uncertainty in Communication23X

24. of 29 Homomorphisms 12/02/2013Purdue: Uncertainty in Communication24

25. of 29 12/02/2013Purdue: Uncertainty in Communication25

26. of 29 Better upper bounds: 12/02/2013Purdue: Uncertainty in Communication26

27. of 29 12/02/2013Purdue: Uncertainty in Communication27 Better upper bounds:

28. of 29 Future work? 12/02/2013Purdue: Uncertainty in Communication28

29. of 29 Thank You 12/02/2013Purdue: Uncertainty in Communication29