Distributed Publish/Subscribe Network Presented by: Yu-Ling Chang

Overview Introduction Basic Overview of Pub/Sub System Classification of Pub/Sub Architecture Event filtering and Event Routing SIENA Pros and Cons Conclusion and Future work References

Introduction: Motivations for Pub/Sub model Traditional Client/Server communication model (Employs RPC, message queue, shared memory etc..)  Synchronous, tightly-coupled request invocations.  Very restrictive for distributed applications, especially for WAN and mobile environments.  When nodes/links fail, system is affected. Fault Tolerance must be built in to support this. Require a more flexible and de-coupled communication style that offers anonymous and asynchronous mechanisms.

What is a Publish/Subscribe System? Distributed Pub/Sub System is a communication paradigm that allows freedom in the distributed system by the decoupling of communication entities in terms of time, space and synchronization. An event service system that is asynchronous, anonymous and loosely-coupled. Ability to quickly adapt in a dynamic environment.

Key components of Pub/Sub System Publishers : Publishers generate event data and publishes them. Subscribers : Subscribers submit their subscriptions and process the events received P/S service: It’s the mediator/broker that filters and routes events from publishers to interested subscribers.

Publish/Subscribe System

Decoupling in time, space and synchronization Provides decoupling in time, space and synchronization.

Classification of Pub/Sub Architectures Centralized Broker model  Consists of multiple publishers and multiple subscribers and centralized broker/brokers (an overlay network of brokers interacting with each other).  Subscribers/Publishers will contact 1 broker, and does not need to have knowledge about others.  E.g. CORBA event services, JMS, JEDI etc…

Classification of Pub/Sub Architectures Peer-to-Peer model  Each node can be publisher, subscriber or broker.  Subscribers subscribe to publishers directly and publishers notify subscribers directly. Therefore they must maintain knowledge of each other.  Complex in nature, mechanisms such as DHT and CHORD are employed to locate nodes in the network.  E.g. Java distributed event service

Key functions implemented by P/S middleware service Event filtering (event selection)  The process which selects the set of subscribers that have shown interest in a given event. Subscriptions are stored in memory and searched when a publisher publishes a new event. Event routing (event delivery)  The process of routing the published events from the publisher to all interested subscribers

Event Filtering (Subscription Model) Topic based VS Content based Topic based  Generally also known as topic based, group based or channel based event filtering.  Each event is published to one of these channels by its publisher.  Subscribers subscribes to a particular channel and will receive ALL events published to the subscribed channel.

Topic-based subscription  Simple process for matching an event to subscriptions. However, limited expressiveness.  Event filtering is easy, event routing is difficult (Heavy load on the network). The challenge is to multicast event effectively to subscribers.

Event Filtering- Subscription Model Topic based VS Content based Content based  More flexibility and power to subscribers, by allowing more expression in arbitrary/customized query over the contents of the event.  Event publication by a key/value attribute pair, and subscriptions specify filters using a explicit subscription language.  E.g. Notify me of all stock quotes of IBM from New York stock exchange if the price is greater than 150

Content-based Subscription  Added complexity in matching an event to subscriptions. (Implementation: Subscription arranged in a matching tree, where each node is a partial condition.  However, more precision is provided and event routing is easier

Event Routing After filtering the events, the broker/brokers must route the events to the corresponding subscribers. Can be done in the following ways:  Unicast  Multicast  Server push/ client pull

Event Routing The broker makes the decision: how to route the message to the subscriber. Several optimization schemes are available.  Profile forwarding scheme – brokers only forward the event to their neighbor broker which fulfill their subscription  Filtering the total covering of the subscription of the system – accept publisher events only if a subscriber has subscribed this event.

Example: SIENA SIENA is a wide area notification service that uses covering-based routing.  Consists of Nodes and servers (access points), Event notifications & filters, Publish/subscribe protocol + advertisements, Identities and handlers, Filtering Siena system can be configured in three types of inter- connection topologies:  Hierarchical client/server architecture  Acyclic P2P architecture  General P2P architecture

SIENA: Hierarchical Architecture Servers interact with each other in an asymmetric client-server fashion. Server is not distinguished from objects of interest or interested parties Potential overloading of server stationed at higher level of hierarchy Failure of one node in hierarchy causes all the nodes below that node to fail

Acyclic P2P architecture and General P2P architecture The acyclic P2P architecture and General P2P architecture are very similar.  Both represented by an undirected graph and allows bidirectional communication.  Scaling an issue for both. Acyclic P2P  Restriction on the configuration of connections between servers to forming acyclic graph representation  Therefore no redundant connections/ multiple paths are not allowed. (Enforcement by a cycle avoiding algorithm)  Can be difficult to maintain and not as robust as general P2P architecture. General P2P architecture  Requires less coordination among servers.  Redundancy enforces robustness of Siena system with respect to failure of single servers.  Drawback: Special algorithms must be run to choose the best path.

Siena: Routing Simplest strategy is to maintain the subscriptions at their access point and broadcast the notification throughout the network  Least efficient  Consumes lots of bandwidth Send the notification towards the event servers that have clients that are interested in that notification (possibly using shortest path)

SIENA : Routing Downstream Replication  Events are kept as one copy as long as possible and only replicated when it is as close as possible to the subscribing servers/clients.

SIENA : Routing Upstream Evaluation  Applying filters upstream, that is as close to the event publisher as possible

Advantages of Pub/Sub Highly suited for mobile applications, ubiquitous computing and distributed embedded systems Robust – Failure of publishers or subscribers does not bring down the entire system Scalability- Suited to build distributed applications consisting a large number of entities Adaptability- can be varied to suit different environments (mobile, internet game, embedded systems etc…)

Disadvantages of Pub/Sub Reliability – no strong guarantee on broker to deliver content to subscriber. After a publisher publishes the event, it assumes that all corresponding subscribers would receive it. Potential bottleneck in brokers when subscribers and publishers overload them. (Solve by load balancing techniques) Security an issue:  Encryption hard to implement when the brokers has to filter out the events according to context.  Brokers might be fooled into sending notifications to the wrong client, amplifying denial of service requests against the client.

Conclusion Distributed Pub/Sub System provides a loosely- coupled, asynchronous model which is useful in many fields of network utilization. Several areas are still open for research:  Effective routing and filtering algorithms for better performance  Scalability versus expressiveness issue  Fault tolerance  Security

References Papers:  P.Eugster, P.Felber,RGuerraoui and A.Kermarrec. The Many Faces of Publish/Subscribe. In ACM Computing Surveys, Vol. 35, No.2, June  A.Carzaniga, D.Rosenblum, A.Wolf. Design and Evaluation of a Wide-Area Event Notification Service. ACM Transactions on Computer Systems, Vol. 19, No. 3, August Wikipedia: 