Presentation is loading. Please wait.

Presentation is loading. Please wait.

Content-Based Routing. Asynchronous Middleware© Gero Mühl2 The Need for a Distributed Implementation  Disadvantages of a centralized notification service.

Similar presentations


Presentation on theme: "Content-Based Routing. Asynchronous Middleware© Gero Mühl2 The Need for a Distributed Implementation  Disadvantages of a centralized notification service."— Presentation transcript:

1 Content-Based Routing

2 Asynchronous Middleware© Gero Mühl2 The Need for a Distributed Implementation  Disadvantages of a centralized notification service  Single point of failure  Bottleneck wrt. bandwidth and computing power  Does not exploit locality although most interaction will be local ⇒ Neither reliable nor scalable to wide-area scenarios  Service must somehow be distributed and replicated ⇒ Multiple cooperating service instances necessary!

3 Asynchronous Middleware© Gero Mühl3 Broker Network  Service instances called brokers  Each broker manages a set of exclusive local clients to which it forwards matching notifications it processes  Notifications are stepwise forwarded through broker network  Cyclic forwarding and delivery of duplicates must be avoided!  Easy in acyclic topologies but these provide no fault tolerance  Complex in topologies with cycles B1B1 Local clients B2B2 B3B3 B4B4

4 Asynchronous Middleware© Gero Mühl4 Flooding  Each notification is delivered to all brokers  A broker forwards an incoming notification to all neighbors if it was received from a local client and to all other neighbors if it was received from a neighbor  Advantages  No routing tables necessary  Subscriptions become active immediately  Implementations can easily exploit IP-Multicast  Disadvantage  Many notifications may be processed and forwarded unnecessarily wasting computing power and network bandwidth

5 Asynchronous Middleware© Gero Mühl5 Efficiency of Flooding  Efficient if  most notifications need to be delivered to a majority of brokers or  subscriptions are highly transient  Not efficient if  filters are highly selective or  locality among subscriptions exists

6 Asynchronous Middleware© Gero Mühl6 Content-Based Routing  Each broker manages a filter-based routing tables  A routing table T B consists of a set of routing entries  A routing entry is a pair (F, D) consisting of a filter F and a destination D which can be a local client or a neighbor broker ⇒ Local and remote routing entries B1B1 B2B2 B3B3 B4B4 n n n sub(F) sub(G) (F,B2)(G,B3)(F,B2)(G,B3) (G,B4)(G,B4) Routing tables n n n n ∈ N(F) n ∈ N(G)

7 Asynchronous Middleware© Gero Mühl7 Content-Based Routing (contd.) Broker B 1 Forwarding Engine Routing Table (F, B 3 ) (G, B 4 ) (H, B 3 ) n n ∈ N(F) n ∉ N(G) n ∉ N(H) B3B3 B4B4 B2B2 1. n 2.

8 Asynchronous Middleware© Gero Mühl8 Notification Forwarding in Acyclic Topologies  Each broker forwards an incoming notification to all neighbor brokers and to local clients for which there is a matching routing entry in the routing table.  If the notification was received from a neighbor broker, it is not forwarded to that neighbor.

9 Asynchronous Middleware© Gero Mühl9 Routing Algorithms  Routing tables must be updated if subscriptions are issued or revoked  Update processes triggered  Many update processes might be in progress currently ⇒ Routing algorithm ⇒ Subscriptions become active gradually ⇒ Race conditions and latency ⇒ Fully active after corresponding update process terminated  Forwarding of unsubscriptions only done for efficiency  Delivery of non-matching notifications can be prevented locally

10 Asynchronous Middleware© Gero Mühl10 Routing Algorithms (contd.)  A correct routing algorithm ensures that 1.Each update process terminates within finite time 2.After an update process terminated, all published notifications matching this subscription are delivered to the subscribing client 3.No duplicates and no non-matching notifications are delivered

11 Asynchronous Middleware© Gero Mühl11 Routing Algorithms (contd.)  Trade-Off  Flooding notifications vs. filtering at intermediate brokers and updating of routing tables  Optimal approach heavily depends on current setting! ⇒ In dynamic environments adaptive solutions necessary

12 Asynchronous Middleware© Gero Mühl12 Simple Routing  Only assumption: Filters can be uniquely identified  Each subscription stored in every routing table  Size of routing tables grows linearly in the number of active subscriptions and the number of brokers  Each routing table affected by a new/cancelled subscription  #brokers-1update messages necessary if update information is not batched ⇒ Not scalable! B4B4 B1B1 B2B2 B3B3 (F,B3)(F,B3) (F,B1)(F,B1) 1. sub(F) 2. sub(G) (F,B3)(F,B3) G (F,B1)(G,B2)(F,B1)(G,B2) (G,B3)(G,B3) (F,B 3 ) (G,B 3 ) G G

13 Asynchronous Middleware© Gero Mühl13 Simple Routing (contd.)  New subscription flooded into the broker network  If a broker receives a subscription F from a neighbor B,  it adds an entry (F, B) to its routing table and  forwards the subscription F to all its other neighbors  Unsubscription flooded into the broker network  If a broker receives an unsubscription F from a neighbor B,  it removes the entry (F, B) from its routing table and  forwards the unsubscription F to all its other neighbors

14 Asynchronous Middleware© Gero Mühl14 Advanced Content-Based Routing  Goals  Smaller routing tables  Reduced filter forwarding overhead  Possible Solutions  Routing algorithms that exploit similarities and merge subscriptions  Integration of advertisements (announcements of producers)  Trade accuracy vs. efficiency  perfect routing  Notifications are forwarded only if a matching subscription exists in respective subnet  imperfect routing  Notifications may be forwarded without any matching subscription

15 Asynchronous Middleware© Gero Mühl15 Identity-Based Routing  Uses identity tests for routing decisions F≡G ⇔ N(F) = N(G)  Avoids  Forwarding of identical filters  Routing entries with identical filters for the same neighbor broker F≡GF≡G geometrical visualization n1n1 n2n2 B4B4 B1B1 B2B2 B3B3 (F,B3)(F,B3) (F,B1)(F,B1) sub(F) (F,B3)(F,B3) (F,B3)(F,B3) F F (F,B1)(F,B2)(F,B1)(F,B2)

16 Asynchronous Middleware© Gero Mühl16 Covering-Based Routing  Uses covering tests for routing decisions F covers G ⇔ N(F) ⊇ N(G)  Avoids  Forwarding of covered filters  Routing entries with covered filters for the same neighbor broker  Forwarding of subscriptions similar to identity-based routing  Forwarding of unsubscriptions more complex due to uncovered subscriptions F G

17 Asynchronous Middleware© Gero Mühl17 Covering-based Routing (contd.) B4B4 B1B1 B2B2 B3B3 (F,B3)(F,B3) (F,B1)(F,B1) sub(F) (G,B3)(G,B3) (F,B3)(F,B3) G G (F,B1)(G,B2)(F,B1)(G,B2) sub(G)  Processing of a new subscription F from a neighbor B  Routing entries regarding B whose filter is covered by F are dropped  F is forwarded only to those other neighbors to which no covering subscription was forwarded before which is still active F G

18 Asynchronous Middleware© Gero Mühl18 Covering-Based Routing (contd.)  Processing of an unsubscription F from a neighbor B  F is forwarded only to those other neighbors to which no covering subscription was forwarded before which is still active  Along with F, uncovered subscriptions must be forwarded B4B4 B1B1 B2B2 B3B3 (F,B3)(F,B3) (F,B1)(G,B2)(H,B2)(F,B1)(G,B2)(H,B2) sub(F) sub(G) sub(H) F G H F (F,B3)(F,B3) F G H F (G,B3)(H,B3)(G,B3)(H,B3) (G,B3)(H,B3)(G,B3)(H,B3)

19 Asynchronous Middleware© Gero Mühl19 Merging-Based Routing  Filters can be merged  perfectlyN(F) = N(G) ∪ N(H)  imperfectlyN(F) = N(G) ∪ N(H)  Merging generates new covers F G H G H F B1B1 B2B2 B3B3 1. sub(G) B4B4 (G,B3)(G,B3) (G,B1)(G,B1) H 2. sub(H) (G,B1)(H,B2)(G,B1)(H,B2) (G,B3)(G,B3) (H,B3)(H,B3) H F (F,B3)(F,B3)

20 Asynchronous Middleware© Gero Mühl20 Merging-Based Routing (contd.)  Merging-based routing is perfect iff  only perfect mergers are generated and  mergers are forwarded and cancelled in a way such that no unnecessary notifications are received later on  … otherwise it is imperfect

21 Asynchronous Middleware© Gero Mühl21 Perfect Merging-Based Routing  Possible implementation on top of covering-based routing  Brokers merge suitable routing entries (F 1, D),…, (F n, D) with the same destination D to a single entry (F, D) such that F is a perfect merger of {F 1, …, F n }  Merged entries are removed from routing table  Merger is added to routing table and forwarded like a normal subscription received from D

22 Asynchronous Middleware© Gero Mühl22 Hierarchical Routing Algorithms  One broker is labeled as root broker R  Any notification and filter is forwarded “upwards” to R  Notifications are only forwarded “downwards” if a matching filter exists  Leads to smaller routing tables but increases the load imposed on brokers on higher levels such as the root broker R B1B1 B2B2 B3B3 B4B4 B5B5 B6B6 1. sub(F) (F,B3)(F,B3) (F,B1)(F,B1) n n n n 2. pub(n) n ∈ N(F)

23 Asynchronous Middleware© Gero Mühl23 2. pub(n) n ∈ N(F) Hybrid Routing Algorithms  Combine hierarchical with peer-to- peer routing B0B0 B1B1 B2B2 B3B3 B4B4 B5B5 B6B6 B7B7 Peer-to-Peer Routing Hierarchical Routing 1. sub(F) (F,B1)(F,B1) (F,B1)(F,B1) (F,B0)(F,B0) n n n n n (F,B3)(F,B3) n

24 Asynchronous Middleware© Gero Mühl24 Advertisements  Producers specify the notifications they will publish  Advertisements are filters that are forwarded like subscriptions by the help of a second routing table  Subscriptions are only forwarded to those destinations for which there is an overlapping advertisement  Test for overlapping: N(F) ∩ N(G) ≠ ∅ G F

25 Asynchronous Middleware© Gero Mühl25 Advertisements (contd.) G F F (G,B4)(H,B5)(G,B4)(H,B5) Advertisements Routing Table B4B4 B1B1 B3B3 B5B5 F H Subscription Advertisements

26 Asynchronous Middleware© Gero Mühl26 Advertisements (contd.)  New advertisement  If B receives a new advertisement from H, B forwards those overlapping subscriptions to H which can newly be served  Cancelled advertisement  If B receives a cancelled advertisement from H, B drops those overlapping subscriptions which can no longer be serviced

27 Asynchronous Middleware© Gero Mühl27 Advertisements (contd.)  Race condition  Published notifications may be dropped if they match only a newly servable subscription that has not yet arrived!  Potential solutions  Keep notifications that match only the new advertisement until all newly servable subscriptions have arrived  corrupts FIFO producer ordering  Block a producer that publishes a notification that matches only the new advertisement until all newly servable subscriptions have arrived  introduces some synchronicity

28 Asynchronous Middleware© Gero Mühl28 Combining Advertisements with Hierarchical Routing  Advertisements are distributed using peer-to- peer routing  Subscriptions are distributed using hierarchical routing but they are only forwarded “upwards” if an overlapping advertisement exists  A flag indicates whether for an advertisements an overlapping subscription exists in an upper sub-tree R B1B1 B2B2 B3B3 B4B4 B5B5 B6B6 1. adv(F) 2. sub(G) N(F) ∩ N(G) ≠ ∅ G (B,B4)(B,B4) false

29 Asynchronous Middleware© Gero Mühl29 Topology Changes  Easy approach preserving an acyclic topology  A new broker B is connected to exactly one broker B’  B’ sends all interesting subscriptions to B  Simple Routing: all subscriptions  Identity-based Routing: identical subscriptions are suppressed  Covering-based Routing: covered subscriptions are suppressed  If advertisements are used, they are exchanged instead of subscriptions  Only leaf brokers can be removed

30 Asynchronous Middleware© Gero Mühl30 Supporting Routing Optimizations  Complexity of routing optimizations depends on the underlying data/filter model  In the most general case not computable  E.g., covering test among relational expressions is NP- complete  Scalability vs. Expressiveness ⇒ Expressiveness must be limited to a sensible level

31 Asynchronous Middleware© Gero Mühl31 Identity among Filters  Two filters F 1 and F 2 are identical, written F 1 ≡ F 2, iff they contain the same number of attribute filters and if for each attribute filter in F 1 there is an attribute filter in F 2 such that these attribute filters are identical  Example of identical filters F 1 = {x ≥ 2} ∧ {y > 5} F 2 = {x ≥ 2} ∧ {y > 5} ≡≡≡

32 Asynchronous Middleware© Gero Mühl32 Overlapping among Filters  Two filters F 1 and F 2 are disjoint iff for an attribute filter in F 1 there exists an attribute filter in F 2 that constrains the same attribute such that these attribute filters are disjoint  Two filters are overlapping iff they are not disjoint  Example for disjoint filters F 1 = {x ≥ 2} ∧ {y > 5} F 2 = {x < 1} ∧ {y = 7} ⊓⊓⊓

33 Asynchronous Middleware© Gero Mühl33 Covering among Filters  A filter F 1 covers a filter F 2, written F 1 ⊒ F 2, iff for each attribute filter in F 1 there is a covered attribute filter in F 2  Example of covering filters F 1 = {x ≥ 2} ∧ {y > 5} F 2 = {x = 4} ∧ {y = 7} ∧ {z ∈ [3,5]} ⊒⊒⊒

34 Asynchronous Middleware© Gero Mühl34 Filter Lattice  Filters are arranged in a lattice according to covering relation  Lattice is updated according to new and cancelled subscriptions  Lattice is used by covering-based routing algorithm F 1 :={x>5} F 2 :={x>8} F 3 :={x=9} F 4 :={x=6} F 5 :={x>6 ∧ y>7} F 6 :={x>7 ∧ y=9} F 7 :={x=9 ∧ y=9}

35 Asynchronous Middleware© Gero Mühl35 Merging of Filters  Two filters F 1 and F 2 can be merged perfectly iff they are identical in all but a single attribute filter  Merging Example F 1 = {x ≥ 2} ∧ {y 6} F 3 = {x ≥ 2} ∧ {y ∉ [4,6]}(Merger) ≡

36 Asynchronous Middleware© Gero Mühl36 Example: Geographic Information Systems (GIS)  F = {(Type = TrafficInformation) ∧ (Location around(Frankfurt,50km))}  G = {(Type = TrafficJam) ∧ (Length ≥ 5km?) ∧ (Location around(Darmstadt,20km))} ⇒ F covers G  H = {(Type = TrafficJam) ∧ (Location around(Frankfurt,40km))}  I = {(Type = TrafficJam) ∧ (Location around(Wiesbaden,40km))} ⇒ H and I can be merged imperfectly X X Frankfurt Darmstadt X X X Frankfurt Wiesbaden

37 Asynchronous Middleware© Gero Mühl37 Bibliography 1.A. Carzaniga. Architectures for an Event Notification Service Scalable to Wide- area Networks. PhD thesis, Politecnico di Milano, Milano, Italy, Dec G. Mühl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis, Darmstadt University of Technology, Chapter 3 3.A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Design and Evaluation of a Wide-Area Event Notification Service. ACM Transactions on Computer Systems, 19(3): , 2001.


Download ppt "Content-Based Routing. Asynchronous Middleware© Gero Mühl2 The Need for a Distributed Implementation  Disadvantages of a centralized notification service."

Similar presentations


Ads by Google