Dynamic Load Balancing in Distributed Content-based Publish/Subscribe Alex K. Y. Cheung & Hans-Arno Jacobsen University of Toronto November 30 th, 2006 Middleware 2006 MIDDLEWARE SYSTEMS RESEARCH GROUP

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Overview Introduction  Background  Motivation Contributions  Load Balancing Framework  Load Estimation Experiments  Setup Conclusions  Future Work  Goals  Related Work  3 Offload Algorithms  Results

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Background on Distributed Pub/Sub subscriber stock = ‘YHOO’ price > $35.00 subscriber stock = ‘YHOO’ price > $30.00 publisher stock = ‘YHOO’ price = $40.00 publisher stock = ‘GM’ price = $20.00 broker multicast

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Motivation A distributed content-based publish/subscribe system offers: 1.Loosely coupled data sources and sinks 2.Flexible routing based on message content 3.Comprised of a distributed broker overlay network Main focus: online games, decentralized workflow execution, and real-time monitoring systems Real-world usage patterns are never evenly distributed:  Performance degradation  Instability and unavailability  Under-utilized resources: Reduced scalability Poor investment in resources

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Research Goal Develop a load balancing algorithm for distributed content-based pub/sub having these strong properties: 1.Dynamic – load balancing can be invoked anytime when overload or uneven load distribution is detected 2.Adaptive – a distinct offload algorithm designed to load balance on each type of resource 3.Heterogeneous – account for brokers with different resource capacities and subscriptions subscribing to different publication sets 4.Distributed – scalability of load balancing algorithm and the pub/sub infrastructure 5.Transparent – publishers and subscribers have no idea about load balancing

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Related Work Compared to Tersptra et al. [5], Meghdoot [2], and Opportunistic Overlays [1]:  Applied to dedicated broker pub/sub architecture  Accounts for heterogeneity  Unlike load sharing, the proposed solution is a dynamic load balancing algorithm that can distribute load in the absence of new node joins  Load estimation prevents overloading load-accepting brokers in load balancing  Balances 3 performance metrics simultaneously on a best-effort approach using adaptive offload algorithms input utilization ratio, matching delay and output utilization ratio  Load balancing is done by migrating subscribers

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Overview Introduction Contributions  Load Balancing Framework  Load Estimation  3 Offload Algorithms Experiments Conclusions

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Load Balancing Framework Pub Sub PEER (Padres Efficient Event Routing) clusters Promotes 2 levels of load balancing: 1.Local-level 2.Global-level cluster-head brokers edge-brokers

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Sub Load Balancing Framework Pub

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Sub Load Balancing Framework Local Load Balancing Global Load Balancing Pub offloading broker load-accepting broker

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Load Estimation Padres Real-time Event-to-Subscription Spectrum (PRESS)  Bit vector approach to record, compare, and estimate subscription load reduced/gained at each broker How much additional traffic does the subscription [class,=,*] introduce onto a broker already subscribed to [class,=,’SPORTS’] ?

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware [class,=,’STOCK’], [value,>,40] [class,=,’STOCK’], [value,>,45] [class,=,’SPORTS’] [class,=,*] Load Estimation with PRESS SubscriptionsPRESS bit vector 1. [class,’STOCK’], [value,60] 2. [class,’SPORTS’] 3. [class,’STOCK’], [value,99] 4. [class,’SPORTS’], [year,04] 5. [class,’STOCK’], [value,42] Publication arrival order: To answer the question: “How much additional traffic does the subscription [class,=,*] introduce onto another broker already subscribing to [class,=,’SPORTS’] ? Answer: 3/5 of the measured total incoming traffic Assuming an incoming publication rate of 5 msg/s, then the additional publication rate is: (5 msg/s) x (3/5) = 3 msg/s

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Offload Algorithms Input Offload Algorithm Load balances on the input utilization ratio: input utilization ratio = incoming pub rate / max match rate Offload subscriptions that reduce the incoming publication rate of the broker Increase matching rate by reducing the matching delay through the match offload algorithm only if the broker is overloaded input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Offload Algorithms Match Offload Algorithm Load balances on the matching delay Offload subscriptions with the lowest traffic input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Offload Algorithms Output Offload Algorithm Load balances on the output utilization ratio output utilization ratio = output bandwidth used / total output bandwidth Offload subscriptions with the highest traffic possible. Favor subscriptions that do not increase the incoming publication rate onto the load- accepting broker input queue Incoming pub/sub messages Matching Engine & Routing Tables … Load Balancer output queues Outgoing pub/sub messages PADRES broker

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Load Balancing Components DETECTOR PIE MEDIATOR PIE messages OFFLOAD ALGORITHMS InputMatchOutput LOAD ESTIMATION PRESS Mediation and migration protocols Subscribers to offload Target broker Load Balancer Establish and teardown load balancing sessions between two brokers/clusters Coordinate transparent subscriber migrations Calculates the set of subscribers to offload for balancing the performance metric in question based on load information about the subscriptions and load-accepting broker Padres Real-time Event-to- Subscription Spectrum (PRESS) uses a bit vector approach to estimate load of subscriptions Triggers load balancing if overload or uneven load distribution is detected by examining the 3 performance metrics Padres Information Exchange (PIE) for exchanging load information with other brokers

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Overview Introduction Contributions Experiments  Setup  Results Conclusions

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Experiments Integrated the load balancing algorithm into PADRES [3] with 20,000 LOC Results obtained through emulation  Bandwidth, queuing, and matching delays Dataset consists of real stock quote publications obtained from Yahoo! Finance [6]  [class,'STOCK'],[symbol,'YHOO'],[open,30.25],[high,31.75],[low,30.00],[close,31.62], [volume,203400],[date,'1-May-96'] Subscription distribution: 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[high,>,REPLACE_HIGH] 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[low,<,REPLACE_LOW] 0.20 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'],[volume,>,REPLACE_VOLUME] 0.34 = [class,eq,'STOCK'],[symbol,eq,'REPLACE_SYMBOL'] 0.05 = [class,eq,'STOCK'],[volume,>,REPLACE_VOLUME] 0.01 = [class,eq,'STOCK']

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Macro Experiment Setup – Local LB B3B2 B1B4 B0 Pub 40 unique publishers Sub 2000 subscribers, 20% sink zero traffic Broker ID CPU (MHz) Memory (MB) Bandwidth (Mbps) B B B B B

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Input Utilization Ratio new subscribers join converged 50% of the publishers increased their publication rate by 100%

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Matching Delay converged 50% of the publishers increased their publication rate by 100% new subscribers join

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Output Utilization Ratio

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Subscriber Distribution

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Client Perceived Delivery Delay

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Input Utilization Ratio Estimation Accuracy

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Press Samples on Input Utilization Ratio Estimation Accuracy

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Scalability on Number of Edge Brokers

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Scalability on Number of Clusters

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Summary of Other Experimental Results Load balancing effectiveness is independent of:  Number of subscribers  Distribution of zero-traffic subscribers  Number of brokers/clusters Load balancing on the input and output utilization ratios also balances their queuing delay counterparts Decreasing the detection threshold can reduce load difference between brokers at the expense of higher overhead Increasing the PIE publication frequency can decrease the convergence time in local load balancing at the expense of higher overhead

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Conclusions Load balancing solution for distributed content- based pub/sub that is dynamic, adaptive, distributed, transparent, and accommodates for heterogeneity Consisting of 3 key contributions:  Load balancing framework  Load estimation methodologies  3 offload algorithms Promote scalability by evenly distributing load onto the available resources

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Future Work Evaluation on a real-world test bed, such as PlanetLab [4] Support load balancing for cluster-head brokers Integrate load balancing with other on-going research projects in PADRES, such as composite subscriptions, historical data access, and failure recovery Self-tuning capability

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware References [1] Y. Chen and K. Schwan, “Opportunistic overlays: Efficient content delivery in mobile ad hoc networks,” in ACM Middleware ’05 Grenoble, [2] A. Gupta, O. D. Sahin, D. Agrawal, and A. E. Abbadi, “Meghdoot: content- based publish/subscribe over p2p networks,” in Proceedings of the 5 th ACM/IFIP/USENIX international conference on Middleware. New York, NY, USA: Springer-Verlag New York, Inc., 2004, pp [3] G. Li and H.-A. Jacobsen, “Composite subscriptions in content-based publish/subscribe systems,” in ACM Middleware ’05 Grenoble, [4] PlanetLab. [Online]. Available: [5] W. W. Terpstra, S. Behnel, L. Fiege, A. Zeidler, and A. P. Buchmann, “A peer-to-peer approach to content-based publish/subscribe,” in DEBS ’03: Proceedings of the 2 nd international workshop on distributed event-based systems. New York, NY, USA: ACM Press, 2003, pp [6] Yahoo! Finance. [Online]. Available:

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware The End Thank you

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Offload Strategies Poset [Carzaniga et al.01] [class,eq,*] [class,eq,’stock’][class,eq,’jobs’] [class,eq,’stock’], [name,eq,’cyber’] [class,eq,’stock’], [value,>,30] [class,eq,’jobs’], [type,eq,’engineering’]

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Handoff 1.Tell selected subscribers to connect to new broker 2.Subscribers issue subscription 3.Broker 2 counts down from 10 x (n-1) seconds after receiving first subscription before assuming migration is complete = subscriber [class,’SUB_CONTROL’], [command,’MIGRATE’], [IP, ], [port,1234] payload [offloadID,’e8sU27’], [population,2]

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Handoff Broker 2 updates its timer. 8.Broker 2 sends OK message to broker 1 when t = 0 or all subscribers are connected 9.Broker 1 tells all remaining migrating subscribers to unsubscribe from broker 1 after its processing delay 10.Subscribers unsubscribe from broker 1 Done! = subscriber OK unsubscribe

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Hand-off Goal:  Achieve 100% delivery ratio in migration process Solution:  Subscribers issue subscription to remote broker  Remote broker sends to original broker that it received all subscribers (guarantees subscriptions are processed by the cluster-head already)  Original broker tells the subscribers to disconnect from it after a delay: delay = (input queuing delay) + (output queuing delay) + (matching delay) To ensure publications still in waiting in the queues get delivered to the migrated subscriber

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Output Utilization Ratio Estimation Accuracy

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Overhead

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Local LB – Press Samples on Output Utilization Ratio Estimation Accuracy

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Macro Experiment Setup – Global LB B12 B11 B10 B22 B21 B20 B32 B31 B30 B42 B41 B40 Pub 40 Sub 2000

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Global LB – Input Utilization Ratio

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Global LB – Matching Delay

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Global LB – Output Utilization Ratio

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Global LB – Subscriber Distribution

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Global LB – Overhead

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Related Work Tersptra et al. [1]  Load sharing scheme for distributed hash table (DHT) based peer-to-peer (P2P) pub/sub involving a unique dissemination tree from each node Meghdoot [2]  Load sharing technique for DHT based P2P pub/sub  Strategically have new incoming nodes to share the matching or bandwidth load of existing node Opportunistic Overlays [3]  Load sharing as a secondary requirement to optimizing delivery path in a mobile P2P publish/subscribe environment

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Load Balancing Framework = Local PIE Message

MIDDLEWARE SYSTEMS RESEARCH GROUP Copyright © 2006 by Alex K. Y. Cheung Middleware Load Balancing Framework = Global PIE Message