The EU DataGrid – Information and Monitoring Services The European DataGrid Project Team

The EDG Information and Monitoring Services Tutorial – n o 2 Overview of Talk u Requirements of a gird information and monitoring service u Globus Metacomputing Directory Service (MDS) l OpenLDAP, a hierarchical database l EDG GRIS/GIIS hierarchy l EDG information providers l Searching the information system u Relational Grid Monitoring Architecture (R-GMA) l A relational implementation of the Global Grid Forums GMA

The EDG Information and Monitoring Services Tutorial – n o 3 Requirements of a Grid Information & Monitoring Service u Offer a flexible infrastructure that provides information on both the Grid and Grid aplications u Must be able to cope with nodes in a distributed environment u Dynamic addition and deletion of nodes should be supported u Security system that controls the access to information to a fine level of granularity u It must be able to work well on an unreliable Wide Area Network u The system must allow new types of information to be used.

Globus MDS Metacomputing Directory Service

The EDG Information and Monitoring Services Tutorial – n o 5 LDAP attributes u EDG currently uses Globus MDS which is built on OpenLDAP u A schema describes the attributes and the types of the attributes associated with data objects u Example - some attributes of SiteInfo: l siteName: RALDEV l sysAdminContact: l userSupportContact: l siteSecurityContact: l dataGridVersion: 1.2 l InstallationDate: Z

The EDG Information and Monitoring Services Tutorial – n o 6 LDAP hierarchy u Lightweight Directory Assess Protocol (LDAP) offers a hierarchical view of information u The objects are arranged in a Directory Information Tree (DIT) u One or more attributes represent the Relative Distinguished Name (RDN) u An object is identified by its Distinguished name (DN) l This is its RDN with the Distinguished name of its parent

The EDG Information and Monitoring Services Tutorial – n o 7 RDNs and DNs u RDN l SE s seId=dev02.hepgrid.clrc. ac.uk l Status s in=status l Protocols s seProtocol=gridftp s seProtocol=rfio s seProtocol=file u DN l Site s Mds-Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid l SE s seId=dev02.hepgrid.clrc.ac.uk,Mds- Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid l Status s in=status,seId=dev02.hepgrid.clrc.a c.uk,Mds-Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid l Protocols s seProtocol=gridftp, seId=dev02.hepgrid.clrc.ac.uk,Mds- Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid s seProtocol=rfio, seId=dev02.hepgrid.clrc.ac.uk,Mds- Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid s seProtocol=file, seId=dev02.hepgrid.clrc.ac.uk,Mds- Vo-name=ral-dev,Mds-Vo- name=uk,o=Grid status supported protocols SE site

The EDG Information and Monitoring Services Tutorial – n o 8 MDS GRISs & GIISs u Information providers are scripts, which when invoked by the LDAP server, make the desired information available. l Information is cached by the server to improve performance l LDAP was designed for use with relatively static data, not rapidly changing data u Within MDS, the EDG information providers are invoked by a local LDAP server, the Grid Resource Information Server (GRIS) u “Aggregate directories”, Grid Information Index Servers (GIIS), are used to group resources u The GRISs use soft state registration to register with one or more GIISs u The GIIS can then act as a single point of contact for a number of resources l A GIIS may represent a site, country, virtual organization, etc. u In turn a GIIS may register with another GIIS

The EDG Information and Monitoring Services Tutorial – n o 9 EDG GRIS/GIIS Hierarchy u Information providers publish information to a local LDAP server known as a Grid Resource Information Server (GRIS) u Each country has a GIIS to which all of the site GIISs register u There is a top level datagrid GIIS to which all of the country GIISs register u Each Site has a Grid Information Index Server (GIIS) which acts as a single point of contact for all of the sites resources. The GRISs register with their site GIIS

The EDG Information and Monitoring Services Tutorial – n o 10 EDG Information Providers u The EDG have produced information providers: l Site information l The Computing Element l The Storage Element l Network Monitoring u All of the EDG data objects are dynamic, they have a time stamp and a time to live (used by the cache mechanism) associated with them

The EDG Information and Monitoring Services Tutorial – n o 11 EDG Information Providers & the Directory Information Tree u Note that there are 2 hierarchies: l The GIIS/GRIS structure l The DIT network information between this and other sites CE storage elements that are close (not necessarily at the same site) status supported protocols file statistics SE site information site

The EDG Information and Monitoring Services Tutorial – n o 12 Siteinfo in=siteinfo,Mds-Vo-name=ral-dev,Mds-Vo-name=uk,o=Grid objectClass: SiteInfo objectClass: DataGridTop objectClass: DynamicObject siteName: RALDEV sysAdminContact: userSupportContact: siteSecurityContact: dataGridVersion: 1.2 installationDate: Z

The EDG Information and Monitoring Services Tutorial – n o 13 Storage Element seId=dev02.hepgrid.clrc.ac.uk,Mds-Vo-name=ral-dev,Mds-Vo-name=uk,o=Grid objectClass: StorageElement objectClass: DataGridTop objectClass: DynamicObject SEId: dev02.hepgrid.clrc.ac.uk CloseCE: dev01.hepgrid.clrc.ac.uk:2119/jobmanager-pbs-M SEtypearchitecture: disk SEsize: SEResourceContactString: SEvo: wpsix, :/flatfiles/05/wpsix

The EDG Information and Monitoring Services Tutorial – n o 14 Map Centre u An easy way to browse the data available in the Information and Monitoring Service is to use Map Centre u

The EDG Information and Monitoring Services Tutorial – n o 15 u Queries can be posed to the current Information and Monitoring Service using LDAP search commands: Querying the Information & Monitoring Service $ldapsearch\ -x\ -H ldap://lxshare0225.cern.ch:2135\ -b 'Mds-Vo-name=datagrid,o=grid\ 'objectclass=StorageElment‘\ seId SEsize \ -s base|one|sub “simple” authentication uniform resource identifier base distinguished name for search filter attributes to be returned scope of the search specifying just the base object, one-level or the complete subtree

The EDG Information and Monitoring Services Tutorial – n o 16 Querying the GRIS/GIIS Hierarchy u Mds-Vo-name=siteB,o=grid l This will look at all the data from siteB Mds-Vo-name =siteA Mds-Vo-name =siteD Mds-Vo-name =siteC Mds-Vo-name =siteB Mds-Vo-name =countryA Mds-Vo-name =countryB Mds-Vo-name =datagrid u Mds-Vo-name=countryA,o=grid l This will look at all the data from countryA u Mds-Vo-name=datagrid,o=grid l This will look at all the data u Mds-Vo-name=siteB,Mds-Vo- name=countryA,o=grid l This will look at all the data from siteB u Mds-Vo-name=siteB, Mds-Vo- name=countryA,Mds-Vo- name=datagrid,o=grid l This will look at all the data from siteB

R-GMA Relational - Grid Monitoring Architecture

The EDG Information and Monitoring Services Tutorial – n o 18 R-GMA u LDAP does not allow queries over different objects l I.e. you can only query based on attributes of an object (no “Joins”) u MDS is not designed for applications to publish their own data l It has relatively static descriptions of the data being published – the schema. u R-GMA is a relational implementation of the Grid Monitoring Architecture (GMA) of the GGF l The relational model is very flexible and allows complex queries which make use of information in multiple objects l R-GMA provides a means for anyone to publish any information on the Grid – can also do the job of the current MDS l It is highly dynamic – with new Producers of information being noticed by existing Consumers

The EDG Information and Monitoring Services Tutorial – n o 19 R-GMA The Consumer Producer Model u Use the Grid Monitoring Architecture from Global Grid Forum u A relational implementation u Applied to both information and monitoring u Creates impression that you have one RDBMS per Virtual Organization Producer Consumer Registry Command flow Information flow

The EDG Information and Monitoring Services Tutorial – n o 20 Relational Approach u Not a general distributed RDBMS system, but a way to use the relational model in a distributed environment. u Producers announce:SQL “CREATE TABLE” publish:SQL “INSERT” u Consumerscollect:SQL “SELECT” u The mediator is a component within the Consumer which locates one or more Producers and combines the information as necessary

The EDG Information and Monitoring Services Tutorial – n o 21 R-GMA Browser u Information in R-GMA can easily be browsed via the browser servlet. u u The browser shows the schema, what producers are registered and allows simple queries to be done.

The EDG Information and Monitoring Services Tutorial – n o 22 Summary u The current information system is Globus MDS l This uses LDAP as its underlying data interface l The are a number of EDG information providers u The new Information and Monitoring Service is R-GMA l Early version in 1.2 l It can support complex SQL queries l It can return a single result set or stream data l Grid applications can publish data

The EDG Information and Monitoring Services Tutorial – n o 23 The End u Information and Monitoring Services l