MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Total Order in Content-based Publish/Subscribe Systems Joint work with: Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto ICDCS 2012, June 19 th 2012 Kaiwen Zhang

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 2 Outline Background: publish/subscribe Total order semantics Related work: sequencers Natural pairwise total order Our two-phase solution Performance evaluation Comparison with Group communication systems

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 3 Advertisement-Based Pub/Sub 3 broker match & forward Advertisement path Subscription path Publication path publisher name = `IBM’ price = $40 subscriber name = `IBM’ price > $20 subscriber name = `IBM’ price > $30 name = ‘IBM’

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 4 Pairwise Total Order (PTO) for Pub/Sub

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 5 Application Scenarios Little work on total order in pub/sub so far  Subscriber decoupling  Minimize overhead External communication between peers  Comparing event streams Consistency  Online games (non-commutative operations)  Monitoring (stream ordering) Fairness  Stock quote (detecting event patterns)

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 6 Drawbacks of Sequencers Sequencer networks [Lumezanu 2006] Topic-based Global knowledge Sequencing overhead applied to all pubs and subs

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 7 Per-Publisher Ordering using FIFO Links Per-Publisher TO  Maintaining order per publisher (i.e. FIFO order) Per-Publisher TO is maintained using FIFO links between overlay neighbours  Requires unique paths to each sub!

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 8 Natural Pairwise TO (N-PTO) Using FIFO links, a common broker for all publisher/subscriber pairs provides N-PTO.  Consider that broker as a “super-publisher” Per-publisher TO is then enforced from that broker on Consider P1 and P2 as internal to the publisher B0

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 9 Our Solution Uses FIFO links  N-PTO provided by a common broker Two phases:  Conflict detection Identify pairs of disjoint publisher/subscriber paths  Resolution Defer delivery only to affected subscribers Lightweight solution for content-based P/S  Uses local ads/subs broker knowledge (next-hops)

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 10 No Conflict P2 doesn't match S4 No conflict S1-S3 share same next-hop as P2 No conflicts detected at B1 S1-S3 have different next hop from P2: B3 is a common broker

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 11 Conflict Resolution S4 forwarded immediately S1 has a different next-hop from S2-3 and P2: Their paths are disjoint! Defer delivery to S1, wait for ack Conflict for S3, defer until ack Detects that P2 and S2 have different next-hops: Send ack back

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 12 Properties of the Solution System model:  Acyclic topology  FIFO links  Advertisement-based Solution is safe under crash failures  Theorem: “Pairwise total order is impossible in publish/subscribe systems under the presence of failures.” Optimizations:  Progressive delivery  Faster detection

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 13 Evaluation Implementation using PADRES in Java  Modular; requires only few changes to the core Workload  Overlapping subscriptions; stress test  High conflict rates: % Topology  24 to 96 brokers  800 subscribers, 240 publishers

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 14 Low Detection Overhead Overall, still small impact to non-conflict subs Overhead increases with less conflict

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 15 Scalability over Baseline Central sequencer Our solution No total order Throughput saturated for central sequencer

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 16 Performance over Spread (Group Communication Systems) Our solution Spread w/o ord. Outperformed after a certain size Spread w/ TO (token algorithm) Overhead due to TO

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 17 Conclusions & Future Work Lightweight solution for content-based p/s  Local broker knowledge  Contained resolution overhead Extension for cyclic topologies  Adjust routing paths Pub/sub clustering and client mobility  Maximize natural ordering

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen 18 Thank you!