EGEE is a project funded by the European Union under contract IST R-GMA: Production Services for Information and Monitoring in the Grid John Walk / RAL UK e-Science AHM2004, Nottingham

R-GMA, UK e-Science AHM, 1 September What’s the talk about? Overview of R-GMA Practical experience with R-GMA Building a production-quality Web Services version of R-GMA

R-GMA, UK e-Science AHM, 1 September Who are we? The system (R-GMA)  Grid Information and Monitoring System created in the European DataGrid (EDG) project  Now being taken forward as part of the production-quality middleware being developed in the EGEE project The people (JRA1-UK)  9 Full-time at Rutherford Appleton Laboratory (CCLRC)  1 Full-time at Queen Mary, University of London  Contributions from Heriot-Watt University (Edinburgh) and Trinity College Dublin We can be ed at: 

R-GMA, UK e-Science AHM, 1 September Relational Grid Monitoring Architecture Producer Service Registry Service Consumer Service API Mediator Consumer application Producer application Publish Data Send Query Receive Data Register Locate Query Data An implementation of the GGF Grid Monitoring Architecture (GMA).

R-GMA, UK e-Science AHM, 1 September Relational Grid Monitoring Architecture Producer Service Registry Service Consumer Service API Mediator Schema Service Consumer application Producer application Publish Tuples Send Query Receive Tuples Register Locate Query Tuples SQL “CREATE TABLE” SQL “INSERT” SQL “SELECT” An implementation of the GGF Grid Monitoring Architecture (GMA) All data modelled as tables: a single schema gives the impression of one (loosely coupled) virtual database for VO

R-GMA, UK e-Science AHM, 1 September Secondary Producers Used to republish data  To co-locate information to speed up queries  To reduce network traffic  To use different producer properties PP SP PP – Primary Producer SP – Secondary Producer

R-GMA, UK e-Science AHM, 1 September A unified approach to information and monitoring Provides information on both:  The Grid itself Mainly for the middleware packages, but the user may also query it to understand the status of the Grid  Grid applications For users Flexible infrastructure  All data has a timestamp enabling its use for monitoring  Dynamic addition and deletion of information producers  Dynamic creation and deletion of tables in schema  Able to cope with failures in a distributed environment

R-GMA, UK e-Science AHM, 1 September Resilience Soft state registration protects services and registry from defunct producers and consumers Registry and schema are replicated to avoid single point of failure (and improve scalability) All services tolerate failure of remote service or resource by seeking an alternative, trying again later, or failing gracefully if these are not possible

R-GMA, UK e-Science AHM, 1 September Grid Application: CMS Job Monitoring

R-GMA, UK e-Science AHM, 1 September BOSS Job wrapper P Monte Carlo Simulation Job BOSS Job wrapper P Monte Carlo Simulation Job R-GMA in CMS C BOSS Database R-GMA Primary Producer C R-GMA Consumer P In testing… Target: 2000-job batch Exceeded in 2004

R-GMA, UK e-Science AHM, 1 September Grid Middleware: Network Monitoring

R-GMA, UK e-Science AHM, 1 September R-GMA in EDG WP7 Network Monitoring SP PP PingER IPerf UDPmon GridFTP Log Network Monitoring NodesStorage Elements Network Managers Resource Brokers Replica Managers R-GMA Primary Producer SP R-GMA Secondary Producer P TCP throughput ICMP packet loss Transfer times Network metrics include… MySQL Database

R-GMA, UK e-Science AHM, 1 September Enabling Grids for E-science in Europe “EGEE will integrate current national, regional and thematic Grid efforts to create a seamless European Grid infrastructure for the support of the European Research Area” ( R-GMA is the Information Services component of the gLite middleware being developed for EGEE Main efforts are:  Producing production-quality software  Migrating to Web Services  Improving portability Duration of project is two years, with first release of software due after first year (March 2005)

R-GMA, UK e-Science AHM, 1 September Main achievements so far (JRA1-UK) Specification complete Design complete WSDL produced for user interfaces Prototype of new API available (Java, C, C++, Python), as a wrapper around the old one (feedback welcome) Registry replication completed; outline design for schema replication produced

R-GMA, UK e-Science AHM, 1 September Next steps Complete move to WS-I compliant Web Services Introduce multi-VO support Complete security design work Complete the schema replication Port to Win32 (EGEE secondary platform) Watch developments elsewhere and seek to inter-work (e.g. contribution to OGSA; collaboration with L&B work)

R-GMA, UK e-Science AHM, 1 September R-GMA Web Services API available for Java, C, C++ and Python Users may by-pass API if they wish, but API is the easiest way to use R-GMA services Registry and Schema also implemented as Web Services, but not accessed directly by users, except for administration User Application R-GMA API SOAP STUB AXIS SERVLET Primary Producer Service Secondary Producer Service On-demand Producer Service Consumer Service SOAP/HTTP Messages SOAP/HTTP Messages And SOAP Faults Tomcat Servlet Container Created from WSDL using gSOAP or Axis

R-GMA, UK e-Science AHM, 1 September R-GMA Security Many issues are EGEE-wide and are being addressed by JRA3 (Security) User requirements vary widely from HEP (would probably do without it) to BioMed (very stringent confidentiality requirements) Will allow installations to turn security off where performance is paramount Some R-GMA users require access to data to be controlled down to sub-table level. Proposal is to use parameterized views, e.g. SELECT * from Job where Owner=[DN] : DN=[DN];

R-GMA, UK e-Science AHM, 1 September Further Information JRA1-UK web site   Thanks to the EU and our national funding agencies for their support of this work