1. FIRMS: Federation & Implementation of Reliable Messaging Specifications for Web Services Geoffrey Fox (managerial contact) Shrideep Pallickara (technical contact) October 21 2004 Southampton

2. Staffing and Contract Update Contract still waiting negotiation of terms and conditions –e.g. is the State of Indiana part of British Empire and subject to its laws Have interviewed two good software engineers – Shrideep gave them each a one day exam

3. FIRMS and FINS Are built on NaradaBrokering Software with a different leaner deployment –Can use original deployment if need additional features Broker Publisher Subscriber Original FIRMS Service NB as Appl. Logic Handler NB as Service Handler Appl. Logic

4. Multiple protocol transport support In publish-subscribe Paradigm with different Protocols on each link Transport protocols supported include TCP, Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS. Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing Allows Highest performance transport Subscription FormatsSubscription can be Strings, Integers, XPath queries, Regular Expressions, SQL and tag=value pairs. Reliable delivery Robust and exactly-once delivery in presence of failures Ordered delivery Producer Order and Total Order over a message type. Time Ordered delivery using Grid-wide NTP based absolute time Recovery and Replay Recovery from failures and disconnects. Replay of events/messages at any time. Buffering services. Security Message-level WS-Security compatible security Message Payload options Compression and Decompression of payloads Fragmentation and Coalescing of payloads Messaging Related Compliance Java Message Service ( JMS ) 1.0.2b compliant Support for routing P2P JXTA interactions. Grid Feature SupportNaradaBrokering enhanced Grid-FTP. Bridge to Globus GT3. Web Service reliabilityPrototype implementation of WS-ReliableMessaging Current NaradaBrokering Features

5. Forthcoming Features Production implementations of WS-Eventing, WS-Notification, WS-RM and WS-Reliability. Time Differential Services: Preserve time spacing between events, that are time-stamped using high-resolution timers. Active replay support: Pause and Replay live streams. Replicated storage support for fault tolerance and resiliency to storage failures.

6. WS-Reliable Messaging Specification from IBM, and Microsoft Leverages the WS-Addressing and WS-Policy specifications. Provides support for both positive and negative acknowledgements. Provides operations for explicit creation and termination of sequences. Delivery assurance mode supported: at-least- once, at-most-once, exactly-once, and ordered delivery

7. WS-Reliability Specification from Fujitsu, Oracle, and Sun Provides support for positive acknowledgements. Provides support for a variety of message-exchange patterns. –Request/Response, One-way, Polling Delivery assurance mode supported –Unreliable, at-least-once, ordered-and-exactly-once Under consideration by the OASIS to be a standard

8. Mechanisms for Reliable Messaging I There are essentially sequence numbers on each message Unreliable transmission detected by non-arrival of a message with a particular sequence number This is “just some TCP reliability” built at application level (Service level Internet) One can either use ACK’s – Receiver (service B) positively acknowledges messages when received –Service A fully responsible for reliability Or NAK’s – Service B is partially responsible and tracks message numbers – sends a NAK if sequence number missing Service BService A M(n+1)M(n)

9. Mechanisms for Reliable Messaging II Each message has a retransmission time; messages are retransmitted if ACK’s not received in time –Uses some increasing time delay if retransmit fails Note need to be informed (eventually) that OK to throw away messages at sender; pure NAK insufficient Note this is reliability from final end-point to beginning end-point: TCP reliability is for each link and has different grain size and less flexible reliability mechanisms There are several efficiency issues –Divide messages into groups and sequence within groups –Do not ACK each message but rather sequences of messages NAK based system attractive if high latency (some mobile devices) on messaging from receiver back to sender

10. Custom Message Reliability Narada Broker Filter 1 Filter 2 WS-RM Wireless Optimized WS-RM WS-Reliability 2 second PDA reply latency! Different endpoints may well need different reliability schemes. Another reason to use application layer. Need to define easy to use “standard reliability profiles

11. Handlers and Filters in-memory Processing Process WS-RM Built-in Handlers

12. Deployment Issues for “System Services” “System Services” are ones that act before the real application logic of a service They gobble up part of the SOAP header identified by the namespace they care about and possibly part or all of the SOAP body –e.g. the XML elements in header from the WS-RM namespace They return a modified SOAP header and body to next handler in chain WS-RM Handler WS-…….. Handler Header Body e.g. ……. Could be WS-Eventing WS-Transfer ….

13. Proxy Distributed WS-RM Processing A handler is like an in memory “service” so one can build handlers that can alternatively be deployed “outside” application service and look like a WS-RM service. Comments on handlers as services paradigm? –Will capture this as a deployment memo –Handlers could be just part of application logic – more natural for WS-Eventing than WS-RM WS-RM enabled SOAP WS-RM Service Legacy Service WS-RM removed from SOAP Header

14. WSRM Implementation: Overview

15. Stable Storage This is where messages are stored until receiver indicates that message has been successfully processed –In memory, Flat Files, MySQL supported –In memory (default?) or Flat File is sufficient for WS-RM messages that do not require sophisticated search Can store messages at one or more distributed NaradaBrokering sites –Could keep messages for a long time! All “new” communication will be done using SOAP and WSDL defined ports

16. Complicated NaradaBrokering NaradaBrokering has more capabilities than OMII needs –We will make them optional to reduce code bloat NaradaBrokering could implement Handler chains for protocols and WSDL bindings not supported by AXIS –UDP plus WS-RM (or equivalent) has been shown to outperform traditional TCP over high bandwidth high latency links –Also supports parallel TCP and GridFTP

17. WS-ReliabilityWS- ReliableMessaging DefinesDefines elements and attributes in the header block of a SOAP envelope. An XML based schema for elements that are needed for reliable messaging. Related SpecificationsSOAP, WS-SecurityWS_addressing, WS- Policy, WS-Security Companies InvolvedSun, Oracle, FujitsuIBM, Microsoft, BEA Submission StatusUnder consideration by OASIS to be a standard Not submitted yet. Delivery modes supported Unreliable, at-least- once, ordered-and- exactly-once At most once, at least once, ordered and exactly-once.

18. WS-ReliabilityWS- ReliableMessaging Groups of messagesIdentified by GroupId information associated with every message in sequence. Individual messages in the group also have a SequenceNumber which increases monotonically. Grouped together using Sequence element, which has a unique identifier, and a message number which increments sequentially. Support for creation and termination of message groups NOYES OrderingIs guaranteed only within a group of messages. Is guaranteed only within a group of messages Ordering and Duplication dependence Order is always tied to Guaranteed delivery and cannot be separately specified. Order is not necessarily tied to guaranteed delivery

19. WS-ReliabilityWS-ReliableMessaging Exchange and Specification of protocol constants Through an abstract concept referred to as Agreement. The spec does not recommend one. WS-Policy. Numbering info associated with first element in a group of messages. SequenceNumber starts at 0 for the first message in a group. MessageNumber starts at 1 for the first message in a group. Defines Message- Exchange patterns Request/Response, One- way and Polling Not defined Support for negative acknowledgements NOYES. This enables receiver- initiated error corrections. Support for acknowledging range of messages YES Requesting acknowledgements The AckRequested element is REQUIRED in every message for which reliable delivery needs to be ensured. AckRequested is used to request the receiving entity to acknowledge the message received.

20. WS-ReliabilityWS-ReliableMessaging Time based expiry removerAfter : Receiver Maintains value of GroupId until the end of the sequence is received or after the expiry of a specified time. Plays a role in order/duplicate detection. ExpiryTime : Defines the expiration time associated with an individual message. Expires : Provides an indication of the expiry time for the sequence specified in the Sequence element. There is also an Inactivity timeout associated with sequences. This is specified in milliseconds. RetransmissionsTriggered after receipt of a set of acknowledgements. In the event an acknowledgement is not received, the message is retransmitted until a specified number of resend attempts have been made. Triggered by either positive or negative acknowledgments. Specification of a Retransmission Interval for a sequence. This effects every message within a sequence of messages. The interval can also be adjusted based on the exponential backoff algorithm.

21. WS-ReliabilityWS-ReliableMessaging SecurityRelies on WS-Security and assorted specifications ErrorsAre notified through SOAP faults. Are notified through SOAP faults. Fault processing is more sophisticated since one can leverage WS- Addressing’s message information headers.

22. WSRM/WSR Similarities Messages are part of a sequence/group of messages. Addresses issues pertaining to ordering and duplicate detection. Quality of service constraints are applied to groups of messages. Recommends message-level security: specifically WS-Security for secure delivery of messages. Provides framework for reporting faults/errors in processing between endpoints involved in reliable delivery Provide support for acknowledging multiple range of messages received within a group/sequence.

23. WSRM/WSR Differences WSR has no support for negative acknowledgements. Retransmissions are always initiated by the source of messages. WSRM supports negative acknowledgements. WSRM has an explicit operation for the creation of sequence and associated sequence identifier. WSR has no such operation, a new group begins when a receiver has encountered a new groupID. –disadvantage: difficult to resolve collisions in group id namespace WSRM uses WS-Policy for specifying and exchanging featured info. WSR does not advocate any specification though it alludes to an abstract concept of agreements. Order is always tied to duplication elimination in WSR. WSRM allows order and duplication detection to exist independent of each other WSR incorporates a HTTP binding for its specification. WSRM currently does not have one; though one can simply use SOAP’s HTTP binding. WSRM has explicit termination of sequences. WSR groups cannot be terminated. They are first closed and then removed.

24. Federation, Why? WSRM being supported by powerful group: IBM/Microsoft. WSR being actively considered for recommendation as a standard by OASIS It is quite possible that these specification may continue to co-exist for a while. Federation would allow end-points belonging to different specifications to communicate with each other.

25. Federation, How? Mapping of physical (XML) elements and semantics associated with these specifications. –Mapping of sequence numbering. WSRM starts numbering at 1, WSR starts at 0. –NAKs in WSRM messages will simply be ignored, since WSR does not support it. Mapping of policy elements and rules associated with where/when and the combination in which multiple policy/agreement elements may occur. Enforcing constraints on delivery. WSR provides a subset of the delivery modes available in WSRM Mapping of faults/error reporting Creation/Termination of sequence in WSRM have no equivalents in WSR. –So terminate-sequence in WSRM will trigger multiple requests/retransmission to ensure WSR has received everything. Group expiry time then needs to be updated at the WSR side.

26. FIRMS Schedule

27. FINS: Federation & Implementation of Notification Specifications for Web Services Geoffrey Fox (managerial contact) Shrideep Pallickara (technical contact)

28. WS-Eventing Delivery Modes: Push –Pull and Trap (UDP Notifications) through WS-Management Subscription Manager can be a separate Web service or part of handler bundled with Source Web Service when it gives as in OGSI special ports for each Source service

29. WS-Notification WS-BaseNotification, WS-BrokeredNotification, WS-Topics, WS-ResourceProperties and WS-ResourceLifetime

30. WS-Eventing/Notification Issues IBM (Sun..) joined Microsoft (BEA. Tibco) in new August 2004 specification WS-Eventing will presumably replace WS-BaseNotification as core of WS-BrokeredNotification We made WS-Eventing as our highest priority Eventually support WS-Eventing, WS-Notification and JMS (Java Message Service) in a federated model WS-Metadata needed to query WS-Eventing sources –Not in current OMII plans but clear how to add Note FINS will use FIRMS in all messages if desired

31. FINS Schedule

32. WS-NotificationWS-Eventing Delegated Management of subscriptions Yes. Through the subscription manager interface. Support for replay like features One can get last message to a topic. A sink can also retrieve messages issued between the pausing and resumption of a subscription. No. Subscription operations Subscribe, Pause and Resume. (There is NO exchange to unsubscribe ). Subscribe, Renew, Unsubscribe and Subscription End. Support for filters to narrow notifications YES Subscription lifetimes Defined using the WS-Resource Lifetime specification. Contained within the Subscribe and Renew exchanges.

33. WS-NotificationWS-Eventing Notification filters and topic expressions supported Topic Expressions supported: QName, “/” separated Strings, and XPath path expressions. Filter supported is XPath. User defined filters allowed Hierarchical topics and Wildcards support Yes. Supports * and // wildcards for selection of topic descendants in a topic tree. No. Topic space management Defined using WS-Topics. The topic space will also support exchanges as defined by the WS- ResourceProperties specification. No formal recommendation regarding topic management. Advertisement of supported topics Yes. The NotificationProducer interface allows inspection of available topics. No.

34. WS-NotificationWS-Eventing On demand publishing YES. This is supported through the WS- Brokered Notification specification. This allows a publisher to publish ONLY if there is a consumer interested in receipt of notifications. No. Notification messages Provides support for both a Notify message as well as “ raw ” application specific message, Does not define any special Notification message type. Application can define special ports for different notifications distinct from “real” messages Suggested Security WS-Security and assorted specifications. WS-Security & assorted specifications.

35. WS-Eventing and WS-Notification WS-NotificationWS-Eventing Related Specifications SOAP, WS-Addressing, WS- BaseNotification, WS-Brokered Notification, WS-Topics, WS-Resource Properties and WS-ResourceLifetime SOAP, WS- Addressing Support for loosely coupled notifications. (Producers need not know consumers) Yes. The intermediary called Notification Broker and the exchanges that need to be supported are defined in the WS-Brokered Notification specification. No. Delivery modes supported Push Pull and Trap (UDP transmissions) defined in WS- Management