1. gLite Information System
Riccardo Bruno INFN, Sez. CT
02/07/2007
First Internaional Grid School for Industrial Applications

2. Outline Grid Information Systems Overview gLite Information Systems
GLUE Schema
Grid elements
LCG Information System
Hierarchy
LDAP model
R-GMA
Overview
Server Architecture
Consumer-Producer model
User interfaces
References
Acitrezza, First Internaional Grid School for Industrial Applications,

3. Grid Information Systems Overview
Collect information of grid resources
Discovering new added resources
Monitoring load and health status
Publish these information
Periodically updated
Well know data model
Used by
Users searching a concrete resource
WMS allocating and managing jobs
Other monitoring services
Acitrezza, First Internaional Grid School for Industrial Applications,

4. gLite Information Systems
Basic data model.
Grid Laboratory Uniform Environment (GLUE) Schema.
Two architectures in glite3
LCG Information System (BDII)
BDII over Globus MDS (Monitoring and Discovery System).
OpenLDAP interface.
Preserves LCG-2 compatibility.
Relational Grid Monitoring Architecture (R-GMA)
Based on the GMA (Grid Monitoring Architecture) standard from the Grid Global Forum
Information in SQL relational databases
Web Services.
Future replacement of LCG IS.
Acitrezza, First Internaional Grid School for Industrial Applications,

5. GLUE Schema
Acitrezza, First Internaional Grid School for Industrial Applications,

6. GLUE Schema
A schema of objects and attributes describing Grid resources and its relationships.
Originally a EU-DataTAG and US-iVDGL coordinated effort.
Current participants: EGEE, OSG, Globus and NorduGrid.
A way to describe Grid info
Statically and dynamically supplied
Hierarchically represented
Independently of the framework (LDAP, XML, SQL…)
Actual release (1.2) is mapped into
LDAP
XML
Old ClassAd (used by Condor Matchmaking)
Acitrezza, First Internaional Grid School for Industrial Applications,

7. Site Element
Acitrezza, First Internaional Grid School for Industrial Applications,

8. Cluster Element
Acitrezza, First Internaional Grid School for Industrial Applications,

9. Computing Element
Acitrezza, First Internaional Grid School for Industrial Applications,

10. LCG Information System
Acitrezza, First Internaional Grid School for Industrial Applications,

11. LCG Information System Levels
Resource level: Grid Resource Information Server (GRIS)
One GRIS on top of each CE, SE, RB, MyProxy (no WNs).
Sensors and scripts get status of concrete resources statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs)
Site level: Grid Index Information Server (GIIS)
Compiles all the information of the different GRISes in a site.
LCG-2 recommends using a BDII instead of a GIIS
Improves robustness and stability.
Called the site BDII.
Top level: Berkeley DB Information Index (BDII)
Keeps all Grid information about the VOs (generally only one).
Stores information from local BDIIs or GIISes in its database.
Only queries sites that are included in a configuration file.
Acitrezza, First Internaional Grid School for Industrial Applications,

12. LCG Information System Hierarchy
Acitrezza, First Internaional Grid School for Industrial Applications,

13. LDAP Model Way of collecting info Users get info with
Pull model (higher level servers periodically query lower level servers)
All servers are based on LDAP
Inherit hierarchical structure (tree-like)
LDAP Data Information Format (LDIF)
Users get info with
Generic applications
ldapsearch (MDS(GRIS/GIIS): 2135 and BDII:2170 ports)
Graphical (BDII web; LDAP GUIs)
Always can get information about specific resources (maybe more up-to-date) by querying directly the site BDIIs, GIISes or GRISes.
Querying VO info with lcg-infosites or lcg-info
Acitrezza, First Internaional Grid School for Industrial Applications,

14. LDAP Browser showing BDII Info
Acitrezza, First Internaional Grid School for Industrial Applications,

15. Relational-GMA
Acitrezza, First Internaional Grid School for Industrial Applications,

16. R-GMA Overview Added from EDG Project
Based on the GMA standard from the GGF
Information in SQL relational databases (a DB per VO)
Query syntax is a SQL subset
Simple consumer-producer model
Web Services oriented
CLI and Web user interface
Allows self-logging applications
Future replacement of LCG IS
Acitrezza, First Internaional Grid School for Industrial Applications,

17. RGMA Overview R-GMA offers a global view of the VO information
In one large relational DB: virtual database.
Registry stores localization tuples (database rows) published by producers:
Standard Tables: CE state in GLUE Schema (by R-GMA-GIN)
Applications specific tables (e.g. self-logging with Log4j)
Access by SQL queries through a WS interface.
Always updated
Consumers get information directly from the resource
Can be cached by a secondary producer from single o multiple producers to improve access.
Acitrezza, First Internaional Grid School for Industrial Applications,

18. R-GMA Server Architecture
R-GMA Servers
Server (one or several)
Schema Server (There is one common global schema server for the grid)
Registry Server (one or several)
Browser (shows grid information via web browser)
Acitrezza, First Internaional Grid School for Industrial Applications,

19. R-GMA Server Description
R-GMA Clients
Generic Client (generic set of APIs for different languages and CLIs)
Servicetool (used by all services to publish its existence and status)
Site Publisher
Client to publish the existence of a site.
Each site must have one R-GMA site publisher.
GadgetIN (GIN)
Client to extract information from MDS and to republish it to R-GMA.
used by the CE.
Data archiver (flexible archiver)
Client to make the data that is coming from the R-GMA site-publisher, servicetool and GIN constantly available.
By default the GLUE tables and service tables are archived, however this can be configured.
Acitrezza, First Internaional Grid School for Industrial Applications,

20. R-GMA Servers Per Node Type
The gLite R-GMA Server should deployed the first as part of a gLite grid, because all services require it to publish service information.
Deployment module
Contains
Used / included by
R-GMA Server
R-GMA server
R-GMA registry server
R-GMA schema server
R-GMA browser
R-GMA site publisher
R-GMA archiver
R-GMA servicetool
MON-box
R-GMA client
RGMA client APIs
UI, WN, SD
R-GMA server,VOMS, LB, WMS, CE, DC, FTS, FTA, Hydra, I/O-Server,Torque Server
R-GMA GIN
R-GMA GadgetIN CE
Acitrezza, First Internaional Grid School for Industrial Applications,

21. GMA Producer- Consumer model
The Producer stores its location (URL) in the Registry.
The Consumer looks up producer URLs in the Registry.
The Consumer contacts the Producer to get all the data.
Data passes directly from the producer to the consumer: (does not through the registry)
The user or any user application does not need to know the registry; this will be handled directly by the consumers and producers.
Registry
Producer
Consumer
Store Location
Look up Location
Execute or Stream data
Acitrezza, First Internaional Grid School for Industrial Applications,

22. Multiple Producers
The Consumer will get all the URLs that could satisfy the query.
The Consumer will connect to all the Producers.
Producers that can satisfy the query will send the tuples to the Consumer.
The Consumer will merge these tuples to form one result set.
Registry
TableName
URL 1
URL 2
Producer 1
Producer 2
TableName
Value 1
Value 2
TableName
Value 3
Value 4
Consumer
TableName
Value 1
Value2
Value 3
Value 4
Acitrezza, First Internaional Grid School for Industrial Applications,

23. Select * from CPULoad
CPULoad (Consumer)
Country
Site
Facility
Load
Timestamp UK
RAL
CDF
0.3
ATLAS
1.6
GLA
0.4
ALICE
0.5 CH
CERN
0.9
0.6
CPULoad (Producer 2) UK
GLA
CDF
0.4
ALICE
0.5
CPULoad (Producer 1) UK
RAL
CDF
0.3
ATLAS
1.6
CPULoad (Producer 3) CH
CERN
ATLAS
1.6
CDF
0.6
Acitrezza, First Internaional Grid School for Industrial Applications,

24. Joins
Service
URI VO
type
Contact
site
gppse01
alice SE
RAL
atlas
gppse02
cms
lxshare0404
CERN
ServiceStatus
URI VO
type up
status
gppse01
alice SE y
SE is running
atlas
gppse02
cms n
SE ERROR 101
lxshare0404
Result Set (Consumer)
URI
Contact
gppse02
SELECT Service.URI Service. Contact FROM Service S, ServiceStatus SS
WHERE (S.URI= SS.URI and SS.up=‘n’)
Acitrezza, First Internaional Grid School for Industrial Applications,

25. R-GMA Concepts Producers Primary producer
The producer creates and stores locally the informatiion
Secondary producer
The producer stores locally the information coming from primary producers
On Demand
The producer returns the information only under the request of the consumer
Consumer
Continuous As soon as new data becomes available it is broadcast to all interested parties
Latest Corresponds to the intuitive idea of “current information”
History
Returns time sequenced data
LATEST RETENTION PERIOD (LRP) and HISTORY RETENTION PERIOD (RTP)
Allow producers to periodically purge old tuples, and to give a precise meaning to the “current state”.
Acitrezza, First Internaional Grid School for Industrial Applications,

26. R-GMA user interfaces R-GMA CLI Web interface
Acitrezza, First Internaional Grid School for Industrial Applications,

27. References GLUE Schema LCG-2 User Guide gLite 3.0 User Guide
LCG-2 User Guide
gLite 3.0 User Guide
R-GMA home page
R-GMA in EGEE
Acitrezza, First Internaional Grid School for Industrial Applications,

28. Questions…
Acitrezza, First Internaional Grid School for Industrial Applications,

29. Hands On (Wiki pages) Browsing Information System (BDII/RGMA)
Advanced Queries to the MDS/BDII
Application Monitoring with RGMA
Acitrezza, First Internaional Grid School for Industrial Applications,