A Scalable Virtual Registry Service for jGMA Matthew Grove DSG Seminar 3 rd May 2005

1 Outline Introduction, Changes made to the jGMA design from last time (November 2004), The architecture of the Virtual Registry (VR), An implementation of the VR using IRC, Benchmarking jGMA, Summary.

2 Introduction jGMA is a pure Java reference implementation of the GGF's Grid Monitoring Architecture, The jGMA design philosophy is to reuse existing implementations rather than reinvent them, To complete the implementation jGMA requires a registry component which must: – Be scalable, – Store sufficient information to be GMA compliant, – Be secure, and prevent unauthorised access to the data, – Need minimal configuration, – Ideally have no single point of failure, – Be robust and tolerant of network failure, – Efficient query routing between VRs.

3 Old jGMA Architecture

4 Architecture Changes We combined the old Registry and PC Servlets into one service called the Mediator, The Mediator provides hooks for a distributed Virtual Registry, The client-side architecture has remained untouched.

5 New jGMA Architecture

6 Implementation Changes Moved from Tomcat[4|5] to Jetty, Changed the internal API to cleanly encapsulate external communications code, Rewrote all code to make better use of features of Java 1.5 including the new concurrency package.

7 The Mediator The Mediator replaces the PC Servlet and the Registry Servlet.

8 The jGMA Library and API A choice of two local registries have been implemented: Simple (volatile) and MySQL. The Mediator did not require changes to the client code.

9 Virtual Registry Services Bootstrapping: Joining the jGMA network with minimal hardwiring, Communications: Efficient routing of queries between VR peers, Caching: Keeping a temporary local copy of some information to reduce the amount of communications between peers. The API is designed to allow different implementations of these core services to be mixed and matched.

10 IRC VR Service It is an attempt at meeting the criteria of the jGMA VR by building a service which uses the Internet Relay Chat protocol. IRC networks have servers connected via a graph topology, some networks manage thousands of users. Simultaneous connections yesterday on some major networks: –Efnet – 89,817 –UnderNet - 75,191 –Freenode – 21,953

11 IRC VR Service Continued –The Bot joins a channel and announces itself to other Bots, –The Bot publishes search queries to the IRC channel, –Other Bots receive the queries and pass them onto the Registry backend, –The Bots reply to the queries via a private message over IRC. Each Mediator has an “IRC Bot” which connects to a chosen IRC network (the IRC networks maintain a DNS pool providing fault tolerant bootstrapping). All messages are routed by the IRC network.

12 IRC VR Walk-Through

13 IRC VR Strengths Established distributed network (IRC was ‘born’ in 1988 and the RFC was published in 1993), Efficient / scalable routing of messages, Establishing client liveliness is handled by the IRC protocol, Fault tolerant bootstrapping via DNS, Various degrees of security can be provided by combinations of: –Leveraging functionality built into the IRC protocol (locking the channel, private messages), –Using features of the IRC daemons such as encrypted routing for IRC messages, –Running your own IRC daemons rather than using a public network.

14 IRC VR Implementation Done: –Bots join the IRC network, –Bots can post and read jGMA queries to channels, –Bots can receive query responses via IRC private messages. TODO: –The IRC Bots do not interact with the Mediator yet, –Bots need to be able to join a locked channel, –There is no caching (this is a generic VR service, not just for the IRC VR).

15 IRC VR Issues / Questions What are the limits on individual channels, is there a need to have multiple channels and route messages between them? Using IRC requires an extra port be accessible to the mediator (out going port 6667).

16 Future Work Short term: –Complete the IRC VR, –Implement a generic caching service for the VR, –Write a simple P2P VR which can be used in place of the IRC VR. –Benchmark jGMA against Naradabroker. Longer term: –Do another binary release of jGMA, –Develop an application or library on top of jGMA.

17 Summary Current work is focusing on the implementation of IRC VR, The design of the jGMA architecture is hopefully complete.

18 Project Web page: – The DSG Web page: – Links