A Scalable Virtual Registry Service for jGMA Matthew Grove CCGRID WIP May 2005

1 Outline Introduction and motivation, The jGMA architecture, The Virtual Registry (VR), An implementation of the VR using IRC, Future work, Summary.

2 Introduction jGMA is a pure Java reference implementation of the GGF's Grid Monitoring Architecture, initially designed to be a “standards-based” messaging layer to bind together the global layer of a project called GridRM, jGMA requirements: –Compliant to the GMA specification, –Supports both non-blocking and blocking events, –Work locally over a LAN or a wide-area, such as the Internet, –Fast, and have a minimal impact on its hosts, –Scalable, –An integrated registry service, –Able to work through firewalls.

3 Design Philosophy Reuse existing software components, if possible, rather than reinvent functionality, Small and well defined API, Minimal number of other installation dependencies, Simple to install, configure and use, Provide a pluggable infrastructure with a choice to extend core functionality with a further more-sophisticated component.

4 jGMA Implementation jGMA has the following components: –A producer and consumer (shared code-base), –A Mediator which provides WAN access to producers and consumers and hooks for a distributed Virtual Registry, and debugging/monitoring. jGMA features: –Written in Java, so takes advantage of usual features and libraries, –Small API (<15 calls), –Released as single “jar” of currently 28Kbytes, –Limited dependencies: Java.

5 jGMA Architecture

6 The Mediator The Mediator replaces the PC and Registry servlets.

7 The jGMA Library and API A choice local registries, two have been implemented: Java data structure and MySQL.

8 Virtual Registry Requirements To complete the implementation jGMA requires a registry component which must: – Be scalable, – Store sufficient information to be GMA compliant, – Be secure, and prevent unauthorised access to the data, – Need minimal configuration, – Ideally have no single point of failure, – Be robust and tolerant to failures, – Efficient query routing between VRs.

9 Virtual Registry Services Core Services: –Boot-strapping: Joining the jGMA network with minimal hardwiring, –Communications: Efficient routing of queries between VR peers, –Caching: Keeping a temporary local copy of some information to reduce the amount of communications between peers. The API is designed to allow different implementations of these core services to be mixed and matched.

10 IRC VR Service The IRC VR service is an attempt at meeting the criteria of the jGMA VR by building a service which uses the mature Internet Relay Chat protocol, IRC networks have servers connected via a graph topology (attempting to efficiently route messages and provide fault tolerance), some networks manage thousands of users. Fault tolerant boot-strapping via DNS, Various degrees of security can be provided by combinations of: –Leveraging functionality built into the IRC protocol (locking the channel, private messages), –Using features of the IRC daemons such as encrypted routing for IRC messages, –Running your own IRC daemons rather than using a public network.

11 IRC VR Service Continued Each Mediator has an “IRC Bot” which connects to a chosen IRC network: –The Bot joins a channel and announces itself to other Bots, –The Bot publishes search queries to the IRC channel, –Other Bots receive the queries and pass them onto the Registry backend, –The Bots reply to the queries via a private message over IRC. All messages are routed by the IRC network.

12 IRC VR Schematic

13 IRC VR Issues / Questions Currently IRC channels are used to partition the VR by project, There needs to be other ways to reduce the impact of a ‘flood query’ on the network: –What are the limits on individual channels, is there a need to have multiple channels and route messages between them? Using IRC requires an extra port be accessible to the mediator (out going port 6667).

14 Current Status LAN (socket) and WAN (HTTP) layers complete, The initial implementation of the IRC VR is complete and functional, Demo applications and benchmarking has thoroughly tested the jGMA software, Permanent on-line demo:

15 Webcam Browser Demo

16 Future Work Short term: – Implement a generic caching service for the VR, – Write a simple P2P VR, which can be used in place of the IRC VR, – Add hooks for debugging, – Benchmark jGMA against Naradabroker. Longer term: – Provide a updated binary release of jGMA, – Develop an application or library that uses jGMA.

17 Summary jGMA is functional despite a few missing features, Current work is focusing on the implementation of the VR, The design of the jGMA architecture is complete.

18 University of Portsmouth, Members of the DSG: –Mark Baker, –Hong Ong, –Garry Smith, –Mr Boe-akis, –Aamir Shafi, –Helen Xaing. Acknowledgements

19 Project Web page: – The DSG Web page: – Links