1. E-infrastructure shared between Europe and Latin America gLite Information System(s) Manuel Rubio del Solar CETA-CIEMAT EELA Tutorial, Mérida, 07-10.11.2006

2. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 This presentation is based on LCG and gLite manuals and previous EELA tutorials

3. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Outline Grid Information Systems Overview gLite Information Systems GLUE Schema –Grid elements LCG Information System –Hierarchy –LDAP model R-GMA –Overview –Server Architecture –Consumer-Producer model –User interfaces References

4. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Grid Information Systems Overview Collect information of grid resources –Discovering new added resources –Monitoring load and health status Publish these information –Periodically updated –Well know data model Used by –Users searching a concrete resource –WMS allocating and managing jobs –Other monitoring services

5. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 gLite Information Systems Basic data model. –Grid Laboratory Uniform Environment (GLUE) Schema. Two architectures in glite3 –LCG Information System  BDII over Globus MDS.  OpenLDAP interface.  Preserves LCG-2 compatibility. –Relational Grid Monitoring Architecture (R-GMA)  Based on the GMA standard from the Grid Global Forum  Information in SQL relational databases  Web Services.  Future replacement of LCG IS.

6. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 GLUE Schema

7. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 GLUE Schema A schema of objects and attributes describing Grid resources and its relationships. –Originally a EU-DataTAG and US-iVDGL coordinated effort. –Current participants: EGEE, OSG, Globus and NorduGrid. –A way to describe Grid info  Statically and dynamically supplied  Hierarchically represented  Independently of the framework (LDAP, XML, SQL…) Actual release (1.2) is mapped into –LDAP –XML –Old ClassAd (used by Condor Matchmaking)

8. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Site Element

9. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Cluster Element

10. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Computing Element

11. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 LCG Information System

12. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 LCG Information System Levels Resource level: Grid Resource Information Server (GRIS) –One GRIS on top of each CE, SE, RB, MyProxy (no WNs). –Sensors and scripts get status of concrete resources statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs) Site level: Grid Index Information Server (GIIS) –Compiles all the information of the different GRISes in a site. –LCG-2 recommends using a BDII instead of a GIIS  Improves robustness and stability.  Called the site BDII. Top level: Berkeley DB Information Index (BDII) –Keeps all Grid information about the VOs (generally only one). –Stores information from local BDIIs or GIISes in its database. –Only queries sites that are included in a configuration file.

13. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 LCG Information System Hierarchy

14. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 A LDAP Model Way of collecting info –Pull model (higher level servers periodically query lower level servers) –All servers are based on LDAP  Inherit hierarchical structure (tree-like)  LDAP Data Information Format (LDIF) Users get info with –Generic applications  ldapsearch (MDS: 2135 and BDII:2170 ports)  Graphical (BDII web; LDAP GUIs)  Always can get information about specific resources (maybe more up-to-date) by querying directly the site BDIIs, GIISes or GRISes. –Querying VO info with lcg-infosites or lcg-info

15. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 LDAP Browser showing BDII Info

16. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Relational-GMA

17. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 R-GMA Overview Added from EDG Project Based on the GMA standard from the GGF Information in SQL relational databases (a DB per VO) Query syntax is a SQL subset Simple consumer-producer model Web Services oriented CLI and Web user interface Allows self-logging applications Future replacement of LCG IS

18. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 RGMA Overview R-GMA offers a global view of the VO information –In one large relational DB: virtual database. –Registry stores localization tuples (database rows) published by producers:  Standard Tables: CE state in GLUE Schema (by R-GMA-GIN)  Applications specific tables (e.g. self-logging with Log4j)  Access by SQL queries through a WS interface. –Always updated  Consumers get information directly from the resource  Can be cached by a secondary producer from single o multiple producers to improve access.

19. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 R-GMA Server Architecture

20. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 R-GMA Server Description R-GMA Servers –Server (one or several) –Schema Server (There is one common global schema server for the grid) –Registry Server (one or several) –Browser (shows grid information via web browser) R-GMA Clients –Generic Client (generic set of APIs for different languages and CLIs) –Servicetool (used by all services to publish its existence and status) –Site Publisher  Client to publish the existence of a site.  Each site must have one R-GMA site publisher. –GadgetIN (GIN)  Client to extract information from MDS and to republish it to R-GMA.  used by the CE. –Data archiver (flexible archiver)  Client to make the data that is coming from the R-GMA site-publisher, servicetool and GIN constantly available.  By default the GLUE tables and service tables are archived, however this can be configured.

21. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 R-GMA Servers Per Node Type –The gLite R-GMA Server should deployed the first as part of a gLite grid, because all services require it to publish service information. Deployment moduleContainsUsed / included by R-GMA ServerR-GMA server R-GMA registry server R-GMA schema server R-GMA browser R-GMA site publisher R-GMA archiver R-GMA servicetool MON-box R-GMA clientRGMA client APIsUI, WN, SD R-GMA servicetool R-GMA server,VOMS, LB, WMS, CE, DC, FTS, FTA, Hydra, I/O- Server,Torque Server R-GMA GINR-GMA GadgetINCE

22. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Registry ProducerConsumer Store Location Look up Location Execute or Stream data nameIDbirthGroup SELECT * FROM people WHERE group=‘HR’ Tom41977-08-20HR R-GMA Producer- Consumer model The Producer stores its location (URL) in the Registry. The Consumer looks up producer URLs in the Registry. The Consumer contacts the Producer to get all the data. Data passes directly from the producer to the consumer: (does not through the registry) The user or any user application does not need to know the registry; this will be handled directly by the consumers and producers.

23. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Consumer Producer 1 Registry TableName Value 1Value2 Value 3Value 4 TableName Value 1Value 2 TableNameURL 1 TableNameURL 2 The Consumer will get all the URLs that could satisfy the query. The Consumer will connect to all the Producers. Producers that can satisfy the query will send the tuples to the Consumer. The Consumer will merge these tuples to form one result set. Producer 2 TableName Value 3Value 4 Multiple Producers

24. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 CPULoad (Producer 3) CHCERNATLAS1.619055611022002 CHCERNCDF0.619055511022002 CPULoad (Producer 1) UKRALCDF0.319055711022002 UKRALATLAS1.619055611022002 CPULoad (Producer 2) UKGLACDF0.419055811022002 UKGLAALICE0.519055611022002 CPULoad (Consumer) CountrySiteFacilityLoadTimestamp UKRALCDF0.319055711022002 UKRALATLAS1.619055611022002 UKGLACDF0.419055811022002 UKGLAALICE0.519055611022002 CHCERNALICE0.919055611022002 CHCERNCDF0.619055511022002 Select * from CPULoad

25. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Service URIVOtypeemailContactsite gppse01aliceSEsysad@rl.ac.ukRAL gppse01atlasSEsysad@rl.ac.ukRAL gppse02cmsSEsysad@rl.ac.ukRAL lxshare0404aliceSEsysad@cern.chCERN lxshare0404atlasSEsysad@cern.chCERN ServiceStatus URIVOtypeupstatus gppse01aliceSEySE is running gppse01atlasSEySE is running gppse02cmsSEnSE ERROR 101 lxshare0404aliceSEySE is running lxshare0404atlasSEySE is running Result Set (Consumer) URIemailContact gppse02sysad@rl.ac.uk SELECT Service.URI Service.emailContact FROM Service S, ServiceStatus SS WHERE (S.URI= SS.URI and SS.up=‘n’) Joins

26. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 R-GMA user interfaces R-GMA CLI Web interface

27. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 References GLUE Schema –http://glueschema.forge.cnaf.infn.it/ LCG-2 User Guide –https://edms.cern.ch/file/454439//LCG-2-UserGuide.html gLite 3.0 User Guide –https://edms.cern.ch/file/722398/1.1/gLite-3-UserGuide.pdf R-GMA home page –http://www.r-gma.org/ R-GMA in EGEE –http://hepunx.rl.ac.uk/egee/jra1-uk/

28. E-infrastructure shared between Europe and Latin America EELA Tutorial, Mérida, 14-16.11.2006 Questions…