1. Some Open Problems in Publish/Subscribe Networking David S. Rosenblum Chief Technology Officer PreCache Inc.

2. Acknowledgments Alexander L. Wolf  University of Colorado at Boulder Antonio Carzaniga  University of Colorado at Boulder PreCache Engineering Team

3. Background

4. Information-Centric Internet Applications Software and Antivirus Updates Consumer Alerts Location-Based Services for Mobile Wireless Multiplayer Online Games Web Search Engines e-Business (e.g., Supply Chain Mgmt) Distributed Sensor Networks Publish/subscribe is a natural fit!

5. Publish/Subscribe Networking Publish/subscribe is traditionally implemented by centralized servers But server-based realizations do not scale to Internet-wide applications So existing networks require “faking it”  Request/response interaction  Continual subscriber polling  Enormous server farms  Dumb caching And so we must realize publish/subscribe via a distributed network of routers

6. 3 2 4 8 1 7 6 5 9 S IENA Content-Based Routing Subscription Forwarding a s1s1 s1:a s1:1 s1:2 s1:3 s1:2 s1:6 s1:3 s1:1 s1:5

7. 3 2 4 8 7 6 5 9 S IENA Content-Based Routing Subscription Merging s1:1 s1:2 s1:6 s1:3 s1:1 s2s2 b s1:3 s1:2 s1:5 s1:1 s2:5 s1:1 s2:5 s1:2 s2:8 s1:2 s2:8 s1:5 s2:b s1:5 s2:b a s 1 covers s 2 1 s1:a s2:2 s1:a s2:2

8. 3 2 4 8 1 7 6 5 9 S IENA Content-Based Routing Notification Delivery b s1:1 s2:5 s1:1 s2:5 s1:2 s1:6 s1:3 s1:1 s1:3 s1:2 s2:8 s1:2 s2:8 a s1:a s2:2 s1:a s2:2 s1:5 s2:b s1:5 s2:b n 1 matches s 1 n 1 matches s 2 n1n1

9. PreCache N ET I NJECTOR Architecture = Routing Engine= Channel Manager Internet Publisher Subscriber = Event Agent

10. PreCache N ET I NJECTOR Routing and forwarding based on S IENA  Generalize idea of subscription merging Compute single subscription covering all received subscriptions  Employ approximate matching Constant time and space complexity Log time and space with additional leakage reduction Channel services  Namespace management  Resource allocation  Load balancing, fault tolerance, authentication

11. Open Problems

12. Comments on the Problems Problems identified based on experience with N ET I NJECTOR and S IENA Many of the problems arise because of a desire for scalability Some problems are deeply technical Other problems are simply pragmatic

13. 3 2 4 8 1 7 6 5 9 Problem Wireless Mobile Devices (WMDs?) a s1:a s1:1 s1:2 s1:3 s1:2 s1:6 s1:3 s1:1 s1:5 a

14. Problem Issues with Wireless Mobile Devices Caching notifications in the network Stream reconstruction and duplicate suppression Frequency of movement versus overhead of reconfiguration Gateways for email, SMS, etc.

15. Problem Security Traditional security properties are address-based  Example: Authentication Bob wants to make sure Alice sent the message  Content-based analog Bob wants to make sure a message represents reality Pub/sub admits new kinds of vulnerabilities  Example: Denial of Service Highly generic subscription (“Price > 0”) causes flood of notifications to subscriber  How do you distinguish a malicious subscriber from a greedy subscriber? How do you do content-based routing when the content is encrypted???

16. Problem Client Connections and Firewalls Want constant connection between subscriber and edge router  Otherwise subscriber polls for notifications  Connections limits may require multiplexing Client must initiate connection to edge router in order to breach firewall And if port 80 is the only open port …  Need HTTP encapsulation of messages  May need HTML formatting of messages  Routers need to multiplex and/or demultiplex message traffic

17. Problem Approximate Matching Rationale: High-performance routing  Expect approximate matching to have better time/space complexity than exact matching Approximation must be conservative  False positives OK, false negatives not  Must still perform exact match at some point before delivery to subscriber Leakage may increase traffic  Tradeoff in computational resources We need simulation tools to explore this!

18. Problem Optimizing for Traffic Variations Can routers dynamically optimize for traffic variations? Example: The Brittany Spears Effect  All subscribers want certain notifications N 1  Few subscribers want other notifications N 2  N 1 notifications may flood network Example: The Google Effect  Certain subscribers S 1 want all notifications  Other subscribers S 2 want few notifications  S 1 subscriptions may dominate routing We need simulation tools to explore this!

19. Problem Service-Provider Deployment Difficult to convince network service providers to enhance their networks with publish/subscribe  Application demand not yet critical  Lack of standards Economic barriers govern router design  Example: 100M users, $10K/router 1000 users/router: 100K routers, $1G outlay 100 routers: $1M outlay, 1M users/router

20. Problem Peer-to-Peer Deployment Reasonable alternative to service-provider deployment  “Grass roots” generation of demand Challenges  Dynamically aligning peer topology to underlying network topology  Dynamically partitioning routing responsibilities across peers  Ensuring reliability, privacy and/or integrity of messages

21. Problem Unicast Fanout at Edge Routers Example: 100M users on 1K routers  100K users per router  10Kbyte notification >80 milliseconds over OC-192 >80 seconds over 10Mb Ethernet >4 hours over 56K modem Idea: Use publish/subscribe for “leveling”  Partition users into classes Example: Last digit of serial number  Publish once per class  Tune publication rate to available bandwidth and SLA

22. Conclusion

23. Many Internet applications naturally require publish/subscribe messaging Scalability can be achieved through publish/subscribe networking S IENA, PreCache, and others have established many fundamental results But many open problems remain to be solved