1. The Narada Event Brokering System: Overview and Extensions
PDPTA Las Vegas June
PTLIU Laboratory for Community Grids
Geoffrey Fox, Shrideep Pallickara
Computer Science, Informatics, Physics
Indiana University, Bloomington IN
12/3/2018
uri="

2. Implications of Web Services
A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL)
In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever
Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but
The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python
Web Services separate the meaning of a port (message) interface from its implementation
Web Services communicate with messages
12/3/2018
uri="

3. Raw Resources Clients Raw Data Raw Data Render to XML Display Format
(Virtual) XML Data Interface
Web Service (WS) WS WS WS
etc.
XML WS to WS Interfaces WS
(Virtual) XML Knowledge (User) Interface
Render to XML Display Format
(Virtual) XML Rendering Interface
Clients
12/3/2018
uri="

4. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
XML Skin
XML Skin
Message
Or Event
Based
Inter Connection
Soft ware
Resource
Soft ware
Resource
Data
base
e-Science/Grid/P2P Networks are XML Specified Resources connected by XML specified messages Implementation of resource and connection may or may not be XML
12/3/2018
uri="

5. Different Web Service Organizations
Everything is a resource implemented as a Web Service, whether it be:
back end supercomputers and a petabyte data
Microsoft PowerPoint and this file
Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport …..
Gives a Peer-to-Peer Grid
Here we discuss Event or Message Service linking web services together
12/3/2018
uri="

6. Classic Grid Architecture
Resources
Database
Database
Content Access
Composition
Middle Tier Brokers Service Providers
Netsolve
Security
Collaboration
Computing
Middle Tier becomes Web Services
Clients
Users and Devices
12/3/2018
uri="

7. Peer to Peer Grid JXTA Integrate P2P and Grid/WS JXTA
Database
Database
JXTA
Web Service Interfaces
Event/ Message Brokers
Integrate P2P
and Grid/WS
Peer to Peer Grid
Web Service Interfaces
JXTA
A democratic organization
Peer to Peer Grid
12/3/2018
uri="

8. Role of Event/Message Brokers
We will use events and messages interchangeably
An event is a time stamped message
Our systems are built from clients, servers and “event brokers”
These are logical functions – a given computer can have one or more of these functions
In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers
Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services
There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications
NaradaBrokering is designed to provide these functionalities
MPI provided such functionalities for all parallel computing
12/3/2018
uri="

9. NaradaBrokering implements an Event Web Service
(Virtual) Queue
Web Service 2
Destination Source Matching
Filter
Routing
workflow
WSDL Ports
Broker
Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor
Workflow implements message process
Routing illustrated by JXTA
Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism
12/3/2018
uri="

10. Features of Event Service I
MPI nowadays aims at a microsecond latency
The Event Web Service aims at a millisecond latency
Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites)
Different performance/functionality trade-off
Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S
Synchronous systems: B acts as a real-time router/filterer
Messages can be archived and software multicast
Asynchronous systems: B acts as an XML database and workflow engine
Subscription is in each case, roughly equivalent to a database query
12/3/2018
uri="

11. Features of Event Web Service II
In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism
All Web Services are specified in XML but can be implemented quite differently
Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP
There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints
One can sometimes compile message streams specified in GXOS to MPI or to local method call
Event Service must support dynamic heterogeneous protocols
12/3/2018
uri="

12. Features of Event Web Service III
The event web service is naturally implemented as a dynamic distributed network
Required for fault tolerance and performance
A new classroom joins my online lecture
A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students
Company X sets up a firewall
The event service sets up brokers either side of firewall to optimize transport through the firewall
Note all message based applications use same message service
Web services imply ALL applications are (possibly virtual) message based
12/3/2018
uri="

13. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Broker Network
(P2P) Community
For message/events service
Broker
Broker
(P2P) Community
Resource
Broker
Broker
Broker
Data
base
(P2P) Community
Software multicast
Broker
(P2P) Community
12/3/2018
uri="

14. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
System Structure I
Systems are a dynamic mix of structured and unstructured entities
P2P systems like JXTA support unstructured systems realized by opportunistic messaging “broadcast locally” over a certain “network distance”
Java Message Service JMS supports structured systems where clients (message endpoints) link to one of a known set of “central servers”
Event system must support
Advertise capability – Publish
Advertise need – Subscribe both for type and form of messages
Transport designated messages/events
12/3/2018
uri="

15. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
System Structure II
One could think that the world is a well defined structure of unstructured systems
Unstructured dynamic systems are P2P (JXTA) Peer Groups
Peer Groups could be cluster of students in a class for distance learning or cluster of Grid (OGSA) Web services generated to support running a job
But maybe it is a set of structured communities with unstructured connection
NaradaBrokering needs to support both models and those in between
Currently has JMS mode, JXTA mode and Native (most powerful) mode
P2P usually thought of as a set of “unruly dangerous clients” but can equally well be used securely as a middleware interaction mode between web services
12/3/2018
uri="

16. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Database
Database
MP Group
Grid Middleware
Grid Middleware
Grid
Middleware
MP G r o u p
MP G r o u p
MP Group
MP=Middleware Peer
12/3/2018
uri="

17. Community Grids Laboratory Activities I
Core NaradaBrokering Event Service
Operation in JMS (see IC paper) or JXTA mode to demonstrate integration of central and peer-to-peer mode
Focus is Performance and Capabilities (see later)
Garnet synchronous collaboration environment used for distance education and seminars
Built first on commercial JMS but ported to Narada – shows that one can afford to use message service in synchronous application sharing
Interface of Garnet to PDA with message size filtering and optimized HHMS message service (IC Paper)
This filtering also needed for slow clients – mix of dial-ups and Internet2 clients in a collaboration
Event system supports (XML) client profiles
12/3/2018
uri="

18. Single Server P2P Illusion
Traditional Collaboration Architecture e.g. commercial WebEx
Data
base
Collaboration Server
12/3/2018
uri="

19. NaradaBrokering Performance Results
12/3/2018
uri="

20. NaradaBrokering and JMS
Low Rate; Small Messages
12/3/2018
uri="

21. NaradaBrokering and JXTA
Comparing Pure JXTA, Narada-JXTA and Direct P2P
There is a bug in JXTA and this was only just fixed
Narada-JXTA provides JXTA guaranteed long distance delivery
Small Payload
Larger Payload
12/3/2018
uri="

22. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Batik Viewer on PC
PowerPoint can be converted to SVG via Illustrator or Web export
SVG Sharing PC to PDA
12/3/2018
uri="

23. PDA Collaboration Event Filter
GMS = JMS or Narada
12/3/2018
uri="

24. Community Grids Laboratory Activities II
Use of JMS (Narada) to support asynchronous collaboration including early GXOS Schema (IKE Paper)
XML based News Groups and Web Site management
Integrated with Apache Slide and Jetspeed portals
Audio-Video Conferencing as a Web service (IC Paper)
H323 and SIP as Web services using XML Session Schema
NaradaBrokering support of UDP
Computing Portals as Web services (CIC Paper)
NaradaBrokering could support events (status, performance, job flow) linking operational job to control servers and researchers
12/3/2018
uri="

25. NaradaBrokering Futures
Higher Performance – reduce minimum transit time to around one millisecond
Substantial operational testing
Security – allow Grid (Kerberos/PKI) security mechanisms
Support of more protocols with dynamic switching as in JXTA – SOAP, RMI, RTP/UDP
Integration of simple XML database model using JXTA Search to manage distributed archives
More formal specification of “native mode” and dynamic instantiation of brokers
General Collaborative Web services
12/3/2018
uri="

26. Collaborative Web Service Access
Intercept and multicast messages produced by Web Service
Web Service Interceptor Providing General Services
Collaboration as a Web Service
Collaborative Web Service
Set Collaboration and Message Mode
Master Client
Web Service has a port on which collaborative modes set Web Service can be “front-end” (in middle tier) to complex back-end object
Event (Message) Service
Client
Client
12/3/2018
uri="

27. Collaborative Replicated Web Services
Intercept and multicast messages SENT to Web Service
Web Service Interceptor Providing General Services
Set Collaboration Mode
Web Service
Master
Object Display
Object Viewer
Object
Web Service
Object Viewer
Object Display
Event (Message) Service
Web Service
Object Viewer
Object Display
12/3/2018
uri="