1. The Information System in gLite
Valeria Ardizzone INFN Catania II Corso di Formazione INFN su apsetti pratici dell’integrazione di applicazioni in GRID ICTP (Trieste),

2. Outline Overview of the LCG Information System
Differents between Top and Site BDII
Generic Information Provider
BDII update process

3. Information System What is?
System to collect information on the state of resources
Why?
To discover resources of the grid and their nature
To have useful data that helps who is in charge of managing the workload to do it more efficiently.
To check for health status of resources.
How?
Monitoring state of resources locally and publishing right information on the information system.
Adopting a data model that MUST be well known to all components that want to access monitored information
Using different approaches that we are going to investigate in next slides

4. Design of Information Systems
About Measures
Measures SHOULD be sensitive to the aim the users want to achieve.
Measures SHOULD be enough accurate to be considered valid.
Rate of taking measures MUST be adequate to be used.
About the gathering of Information
How and when collected info should be published?
Where should collected info be stored?
How long should this info be maintained in the storage?
Querying the Information System
Where should queries be sent to have a response?
What syntax and protocols have to be adopted to make queries?
What is the adopted data model to describe resources?
Security
Who is allowed to execute queries against the IS and what type of queries is he allowed to do?
Management of user rights and credentials.

5. Adopted Information Systems
The BDII (Berkley DB Information Index)
has been adopted in LCG middleware as the Information System provider.
It is an evolution of the Globus Meta Directory System (MDS)
LCG-2 actually adopts BDII as Information System.
It is based on Lightweight Directory Access Protocol (LDAP) servers.
The Relational Grid Monitoring Architecture (R-GMA)
Is an implementation of the Grid Monitoring Architecture (GMA) standardized by the Global Grid Forum (GGF)
It is a relational implementation of the GMA
It is strongly Web Services Oriented
It will be adopted by next releases of the gLite middleware

6. LCG Information System

7. LCG Information System
LCG adopts a combination of solutions
Globus MDS
At the lowest level of the information system
To discover and monitor resources and publish information
Grid Information Security (GSI) credentials
Caching
BDII
At the highest level of the system
Because MDS had some troubles in terms of scalability
Used by the Resource Broker for the matchmaking process
Can be configured by each VO
Queries underlying systems periodically (2 minutes)
Hierarchical system
Information is collected on the leaves of a hierarchical tree and travels towards the root
Clients can query the hierarchical tree at every level
The higher the level against which queries are made, the older is the obtained information

8. Collecting Information
Gathering of information at different levels
Lower level: Grid Resource Information Server (GRIS)
Collects information on the state of a given resource
One GRIS on top of each resource
A set of scripts and sensor that try to extract useful info on the resource
Medium level: Grid Index Information Server (GIIS)
Collects information on resources of a given site
One GIIS for each site
Higher level: BDII
Collects information on resources of a given VO
One BDII for each VO (suggested solution)
Way of collecting info
Pull model (higher level servers periodically query lower level servers)
LDAP query model

9. Berkley Database Information Index (BDII)
The Berkley Database Information Index (BDII)
Developed within the context of LCG project
Solves problems of instability of the MDS occurring when the number of sites grows too much
Stays on top of GIIS sites
One for each VO
Centralized system
Three levels of hierarchy
Accessed by the Workload Management System
Way of working
One GRIS for each resource
One GIIS for each site collecting info from below GRIS systems
One BDII for a given VO collecting information from below GIIS systems
Two LDAP servers, one for write access and one for read access
Every two minutes a cron-job runs a script and collects info from a list of GIIS sites
The list of GIIS is placed in the configuration file of the BDII

10. Berkley Database Information Index (BDII)
Every node (except UI,BDII and Wns) has a GRIS service in order to publish its informations
A node in every site collects all site GRISes and publishes them using a site BDII, by default installed on the CE
The top BDII collects all site BDIIs
User can run a set of commands to query the top BDII.

11. GLUE Schema

12. GLUE Schema Grid Laboratory Uniform Environment (GLUE) Schema
It is a data model to describe in a meaningful way information on grid resources (static and dynamic info)
As result of a collaboration between the EU-DataTAG and iVDGL projects
EGEE, NorduGrid, LCG and Grid3/OSG contributed to the definition of the schema
XML Schema
Now, GLUE Schema is being mapped to an XML representation

13. Site Element

14. Cluster Element

15. Computing Element

16. LCG Information System Hierarchy
BDII-A
BDII-B
BDII-C
Site BDII
Site BDII
Site BDII SE
Local GRIS SE
Local GRIS CE
Local GRIS SE
Local GRIS CE
Local GRIS SE
Local GRIS RB
Local GRIS
Site 1
Site 2
Site 3

17. Top BDII vs Site BDII Site BDII By default it is installed on the CE
It collects all site GRISes* (for example SE,RB,LFC,etc..)
The name of the service is bdii
Top BDII
It collects all site BDIIs** (CE1,CE CEn)
It gives to the RB all needed informations to match and dispatch user's jobs
It can run in the same machine where the RB is running (it's more fast in answer)

18. Generic Information Provider (GIP)
The Generic Information Provider, GIP, is a highly configurable information provider that makes a separation between static and dynamic information. It can be used to produce any kind of information for use with LDAP based grid information systems.
An information provider in its simplest form is a script that prints an LDIF file to standard out. The LDIF file should conform to the schema used in the GIP.
The problem is with dynamic information. Usually there are only a few attributes that have to be found dynamically. The idea with the GIP is to use dynamic plug-ins to obtain these values and use a common framework for everything else.

19. SiteInfo Schema objectClass: DataGridTop objectClass: SiteInfo
in: information
siteName: Name of site
sysAdminContact: Contact details for local system administrator
userSupportContact: Contact details for local user support
siteSecurityContact: Contact details for site security
dataGridVersion: Overall DataGrid software version
installationDate: Date of installation
cpuResourceDescription: Description of available CPU resources
diskResourceDescription: Description of available disk resources
supportedFileSystem: Supported file systems AFS, NFS, local
batchSystem: Available batch system, PBS, LSF, BQS, Condor, fork
etc massStorageDescription: Description of available mass storage
experimentalSoftware: Availablity of software, ATLAS, CMS, ESA etc…

20. SiteInfo Information Provider
dn: in=host.invalid/siteinfo,Mds-Vo-name=local,o=grid
objectClass: SiteInfo
objectClass: DataGridTop
objectClass: DynamicObject
siteName: SITE-NAME
sysAdminContact:
userSupportContact:
siteSecurityContact:
dataGridVersion: LCG-2_2_0
installationDate: Z

21. BDII Update Process (1)
The BDII consists of two or more standard LDAP databases that are populated by an update process.
The update process obtains LDIF (LDAP Data Interchange Format ) file from either doing an ldapsearch on LDAP URLs or by running a local script that generates LDIF. The LDIF is then inserted into the LDAP database.
Options exist to update the the list of LDAP URLs from a web page and to use an LDIF file from a web page to modify the data before inserting it into the database.
The daemon script will start the update process is bdii-update.

22. lcg-info 25

23. Praticals: lcginfo -h/--help: to see the help
–list-attrs: print the list of the possible attributes
--list-ce: lists the CEs which satisfy a query, or all the CEs if no query is given.
--list-se: lists the SEs which satisfy a query, or all the SEs if no query is given.
--bdii: allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error.
--vo: restricts the output to CEs or SEs where the given VO is authorized.

24. lcg-infosites 27

25. Praticals: lcg-infosites
-h/--help: help option
--vo: VO name (mandatory)
--is: it's possible to specify a not default Top BDII
Some options:
se: The names of the SEs supported by the user's VO
ce: The information relative to number of CPUs, running jobs,etc.
rb: Names of the Rbs available for each VO
sitenames: Names of the LCG sites
tag: The names of the tags relative to the software installed in site is printed together with the corresponding CE
closeSE: The names of the CEs where the user's VO is allowed to run together with their corresponding closest SEs are provided

26. References Esercitazione LCG-2 User Guide GLUE Schema
s#Information_System_BDII
LCG-2 User Guide
GLUE Schema