1. MIDDLEWARE SYSTEMS RESEARCH GROUP Adaptive Content-based Routing In General Overlay Topologies Guoli Li, Vinod Muthusamy Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto http://padres.msrg.toronto.edu

2. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Distributed Publish/Subscribe An acyclic overlay is sensitive to: Congestion Broker failures Benefits of a general overlay: Routing around congestion and failures Handling imbalanced workloads Publisher Subscriber Subscription Publication Advertisement Subscriber Applications Business process execution e.g., BPEL Business activity monitoring Service discovery and integration …

3. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Challenges With General Overlays Subscriptions are routed in loops Brokers receive duplicate subscriptions Subscription copies exacerbate the problem Same problem for publications 2 34 5 1 6 Adv 1 Adv 2 S X S S

4. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Agenda Content-based routing protocol for general overlays Atomic and composite subscriptions Optimal publication routing Evaluation Dynamic publication routing Adaptive composite subscription routing

5. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium TID-based Approach Each advertisement is assigned to a unique tree identifier (TID) Each subscription has a TID predicate with a variable 2 34 5 1 6 Adv 1 Adv 2 X S

6. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Subscription Routing 2 34 5 1 6 Adv 1 Adv 2 X S: [class=stock][symbol=*] [TID=$Z ] At Broker 1: Adv1: [class=stock][symbol=IBM] [TID=Adv1] Adv2: [class=stock][symbol=HP] [TID=Adv2] S matching Adv1: [class=stock][symbol=*][TID=Adv1] S matching Adv2: [class=stock][symbol=*] [TID=Adv2] S A2 S A1 S

7. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Publication Routing Each publication is assigned the TID of its matching advertisement e.g., p [class, stock][symbol,HP][TID, adv_msg_id] Publications are routed: Fixed TID routing: a publication is routed to subscribers along its advertisement tree. Dynamic publication routing: a publication may be routed to subscribers across advertisement trees.

8. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Fixed TID Routing Property No broker receives duplicate publication messages 2 34 5 1 6 Adv 1 Adv 2 Sub X P P

9. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Dynamic Publication Routing Publication’s TID is changeable Routing heuristic Util = R output / R sending Property Changing a publication’s TID while in transit will not change the set of notified subscribers. 2 34 5 1 6 Adv 1 Adv 2 Sub X P

10. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Advantages Retains the publish/subscribe client interface Speeds up subscription and publication matching Avoids duplicate subscriptions and publications Routes publications dynamically across multiple alternatives Enables routing around failures, congestion and load imbalances

11. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Composite Subscription AND S1S2 A composite subscription consists of atomic subscriptions linked by logical operators (e.g., AND, OR). Composite subscription routing Topology-based routing Adaptive routing e.g., CS= {[class=stock][symbol=YHOO][price>12]} AND {[class=stock][symbol=MSFT][price<20]}

12. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Topology-based CS Routing 2 34 5 1 6 Adv 1 Adv 2 7 8 9 CS={{S1 AND S2} ANDS3} CS S2 A2 S3 A3 S1 A1 CS’ Adv 3 CS’ ={S1 AND S2} Broker 4 and 8 are the joint point brokers

13. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Adaptive CS Routing CS’s joint points are determined according to potential publication traffic, bandwidth, latency, etc. 2 13 Adv 1 Adv 2 CS={S1 AND S2} 2 13 Adv 1 Adv 2 CS={S1 AND S2}

14. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Cost Model Routing cost of CS RC(CS)) = + + Subscription cardinality |P(S)| : The number of matching publications per unit of time. |P(S)| = |P(CS)| = |P(S l )| + |P(S r )| if op = or subscriptiondest Matching Engine Routing Table + symbol=IBM B1 symbol=HP B2 input queue output queue B1 output queue B2 Broker

15. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Adaptive CS Routing 2 34 5 1 6 Adv 1 Adv 2 7 8 9 CS={{S1 AND S2} ANDS3} CS S2 A2 S3 A3 S1 A1 CS’ Adv 3 CS’ ={S1 AND S2} CS’

16. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Evaluation Setup Overlays of 32 brokers with different connection degrees Cluster (each node:1.86GHz, 4G) and PlanetLab Workloads: Yahoo!Finance stock quote traces http://research.msrg.utoronto.ca/Padres/DataSets Metrics End to end notification delay Network traffic

17. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Dense vs. Sparser Topologies 20% 4% Note: The benefit is not proportional to the connection degree.

18. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Higher Publication Rate stabilized

19. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Publication Burst Burst

20. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium With Broker Failures 1 st failure 2 nd failure

21. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium CS Routing Traffic

22. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Conclusions Enables routing around failures, congestion and load imbalances Allows publications routing across alternative paths Improves the notification delay by 20% Enables flexible CS routing Reduces 80% publication traffic Improves the notification delay by 55% Simplifies solutions for failure recovery and load balancing

23. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Questions? A D R E S P http://padres.msrg.utoronto.ca

24. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium More Publishers

25. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Effect of Subscriber Distance DistanceFixed(ms)Dynamic(ms)Improvement 6 Hops47.20247.568-0.78% 10 Hops64.47752.89517.96% 12 Hops74.41660.59818.57% Max Diff57.65%27.39%

26. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium On PlanetLab

27. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium CS Delay

28. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Faster Matching with TIDs Subscriptions are augmented with TIDs only once at the first broker. Other brokers can route the subscription based on the TID alone. Similar argument applies to publication routing.

29. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Advertisement Routing Each advertisement forms a spanning advertisement tree Duplicated advertisements are discarded by brokers Each advertisement is assigned a unique tree identifier (TID) e.g., a [class,eq,stock]……[TID,eq,adv_msg_id] Subscription Routing Table (SRT) A set of [advertisement, last hop]

30. MIDDLEWARE SYSTEMS RESEARCH GROUP Middleware 2008 @ Leuven, Belgium Subscription Routing Each subscription has a TID predicate with a variable. e.g., s [class,eq,stock]……[TID,eq,$X] The variable is bound to the TID of a matching advertisement Publication Routing Table (PRT) A set of [subscription, {TID, last hop of subscription }]