1. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Brief introduction to Messaging Systems Daniel Rodrigues

2. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Summary Messaging Systems Overview –What –Why –How –Where Monitoring context in the Grid The MSG – Messaging System for Grids Fast Forward

3. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems (Grid Maps image)

4. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems What is Messaging? –Human communication process, characterized for being asynchronous –Examples: The medieval messenger; Post; Message in a bottle. Abstraction of the communication process as messages being sent from one or more producers to one or more consumers; Uhooo… that sounds complicated and off the scope :S “Messaging is a technology that enables high-speed, asynchronous, program-to-program communication with reliable delivery. Programs communicate by sending packets of data called messages to each other. Channels (...) are logical pathways that connect the programs and convey messages. Sender or producer is a program that sends a message (...). A receiver or consumer is a program that receives a message (...)” Hohpe et al, Enterprise Integration Patterns

5. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Why use messaging? –For communicating we could use: File transfer Shared Databases Remote Procedure Calls Web Services Mail CORBA –They do exist; –They have common ideas; –They share implementations; –You might be using more than one to achieve a result that suits your needs!

6. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Why use messaging? –Key ideas and benefits: Loosely coupled distributed communication; Exceptional interoperability; Asynchronous; Reliable; Configurable Persistence –Drawbacks: More complex programming model Harder to do sequenced and synchronous model Performance? (maybe FTP could do the trick)

7. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems How? (Or, “Ok, may we finally see a picture?”) Publisher Consumer How?

8. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Patterns: Message –Header Routing information Description –Body Data Ignored by the messaging system –EventMessage, CommandMessage, DocumentMessage, RequestReply –Could be SOAP, JMS, Stomp, etc.

9. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Patterns: Message Channel –Point-to-Point Snail Mail Queues –Publish-Subscribe Television/radio Broadcast Topics –DataTypeChannel, InvalidMessageChannel, DeadLetterChannel, ChannelAdapter, MessageBus, MessagingBridge

10. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Patterns: Message Endpoint –Publisher Gets data from application and creates a message. –Consumer Extracts data from a message and passes it on to the application. –SelectiveConsumer, CompetingConsumer, DurableSubscriber, MessageDispatcher, TransactionalClients, EventDrivenConsumer. –Endpoints either sends or receives messages, and are channel specific. (Ears mouth,eyes are not the same thing)

11. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Isn’t it too complex to implement all this? –Indeed. But someone has already done most of the work for you: –Commercial solutions: Tibco Rendezvous, IBM WebSphere MQ, SUN Java Message Service, Microsoft MSMQ, BEA MessageQ, SonicMQ, 29West UME/LBM. –OpenSource providers: Apache ActiveMQ, ObjectWeb JORAM, Open JMS. Each are adequate to different problems. –Integration on different platforms; –Latency concerns; –High throughputs;

12. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Where is it used? –Financial Services exchanges, brokerages, hedge funds; –Insurance Companies –Banking Industry –Telecoms Usually embedded in integrated solutions –Enterprise Backbones; WebsphereMQ example (March 2007): –10.000 customers –10 billion messages carrying US$1 quadrillion (US$ 1 000 000 000 000 000) worth of business transactions.

13. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Where is it used? –MSG! Messaging System for the Grid; Used in Monitoring; Based on an ActiveMQ backbone and python; Simple way to send records across… …and have multiple consumers consuming into an Oracle DB; SAM: metricOutput, metricDescriptions; GridFTP transfers : Gridview; Job Status, Job Details from various sources.

14. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems And now, for something completely different

15. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems Other Patterns – Message Routers Message may be routed to different channels depending on its characteristics; Simple Example: use a wild card topic! –grid.usage.transfer.*, where it will be forwarded to grid.usage.transfer. –MessageTranslators Translation at different layers (data structure, types, representation, or transport). e.g. transport protocols: TCP => HTTP => SOAP => JMS –Pipes and Filters Message may need processing in different steps. A Message goes through filtering and pipes that perform different functions (e.g, authN, authZ)

16. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems And now, for something completely different

17. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems And now, for something completely different

18. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Internet Services Messaging Systems And now, for something completely different