1. Information System in gLite
Valeria Ardizzone
Istituto Nazionale di Fisica Nucleare
Corso introduttivo al Grid Computing
Catania,

2. Outline What is the Information System Data Model: the GLUE Schema
Overview
Core entities
OpenLDAP server introduction LCG
Information Service Architecture
Top BDII and Site BDII
Information upgrade process
Grid Computing Course, Physics Dept, Univ. of Catania

3. Information System What is? Why? How?
System to collect information on the state of resources
Why?
To discover resources of the grid and their nature
To have useful data that helps who is in charge of managing the workload to do it more efficiently.
To check for health status of resources.
How?
Monitoring state of resources locally and publishing right information on the information system.
Adopting a data model that MUST be well known to all components that want to access monitored information
Using different approaches that we are going to investigate in next slides
Grid Computing Course, Physics Dept, Univ. of Catania

4. Design of Information Systems
About Measures
Measures SHOULD be sensitive to the aim the users want to achieve.
Measures SHOULD be enough accurate to be considered valid.
Rate of taking measures MUST be adequate to be used.
About the gathering of Information
How and when collected info should be published?
Where should collected info be stored?
How long should this info be maintained in the storage?
Querying the Information System
Where should queries be sent to have a response?
What syntax and protocols have to be adopted to make queries?
What is the adopted data model to describe resources?
Security
Who is allowed to execute queries against the IS and what type of queries is he allowed to do?
Management of user rights and credentials.
Grid Computing Course, Physics Dept, Univ. of Catania

5. Adopted Information Systems
The BDII (Berkley DB Information Index)
has been adopted in LCG middleware as the Information System provider.
It is an evolution of the Globus Meta Directory System (MDS)
LCG-2 actually adopts BDII as Information System.
It is based on Lightweight Directory Access Protocol (LDAP) servers.
The Relational Grid Monitoring Architecture (R-GMA)
Is an implementation of the Grid Monitoring Architecture (GMA) standardized by the Global Grid Forum (GGF)
It is a relational implementation of the GMA
It is strongly Web Services Oriented
It will be adopted by next releases of the gLite middleware
Grid Computing Course, Physics Dept, Univ. of Catania

6. The Data Model: GLUE Schema
Grid Computing Course, Physics Dept, Univ. of Catania

7. GLUE: overview GLUE: Grid Laboratory Uniform Environment
It’s an information model that describe all those resources that partecipate in the Grid system and that are requested to be discoverable and monitored
The same information can be retrieved from different GISs relying on different technology (e.g. R-GMA)
Grid Computing Course, Physics Dept, Univ. of Catania

8. Site Element
Grid Computing Course, Physics Dept, Univ. of Catania 8

9. GLUE: site GlueSiteUniqueID: TRIGRID-INFN-CATANIA
GlueSiteName: TRIGRID-INFN-CATANIA
GlueSiteDescription: LCG Site
GlueSiteUserSupportContact: mailto:
GlueSiteSysAdminContact: mailto:
GlueSiteSecurityContact: mailto:
GlueSiteLocation: Catania, Italy
GlueSiteLatitude:
GlueSiteLongitude:
GlueSiteWeb:
GlueSiteOtherInfo: TIER 1
GlueSiteOtherInfo: Trigrid Team
Grid Computing Course, Physics Dept, Univ. of Catania

10. GLUE: service GlueServiceUniqueID: infn-rb-01.ct.trigrid.it:7772
GlueServiceName: INFN-CATANIA-rb
GlueServiceType: ResourceBroker
GlueServiceVersion: 1.2.0
GlueServiceEndpoint: infn-rb-01.ct.trigrid.it:7772
GlueServiceURI: unset
GlueServiceAccessPointURL: not_used
GlueServiceStatus: OK
GlueServiceStatusInfo: No Problems
GlueServiceWSDL: unset
GlueServiceSemantics: unset
GlueServiceStartTime: T00:00:00Z
GlueServiceOwner: trigrid
GlueServiceOwner: cometa
GlueServiceOwner: inaf
GlueServiceOwner: alice
GlueServiceAccessControlRule: trigrid
GlueServiceAccessControlRule: cometa
GlueServiceAccessControlRule: inaf
GlueServiceAccessControlRule: alice
GlueForeignKey: GlueSiteUniqueID=INFN-CATANIA
Grid Computing Course, Physics Dept, Univ. of Catania

11. Cluster Element Grid Computing Course, Physics Dept, Univ. of Catania11

12. GLUE: cluster and subcluster
GlueClusterName: infn-ce-01.ct.trigrid.it
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-short
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-long
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-infinite
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cert
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-inaf
GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-alice
[..]
GlueSubClusterPhysicalCPUs: 4
GlueSubClusterLogicalCPUs: 4
GlueSubClusterTmpDir: /tmp
GlueSubClusterWNTmpDir: /tmp
Grid Computing Course, Physics Dept, Univ. of Catania

13. GLUE: Host GlueHostApplicationSoftwareRunTimeEnvironment: GLITE-3_0_0
GlueHostApplicationSoftwareRunTimeEnvironment: INFN-CATANIA
GlueHostApplicationSoftwareRunTimeEnvironment: MPICH
[..]
GlueHostArchitectureSMPSize: 4
GlueHostBenchmarkSF00: 1937
GlueHostBenchmarkSI00: 1483
GlueHostMainMemoryRAMSize: 4096
GlueHostMainMemoryVirtualSize: 8192
GlueHostNetworkAdapterInboundIP: TRUE
GlueHostNetworkAdapterOutboundIP: TRUE
GlueHostOperatingSystemName: Scientific Linux CERN
GlueHostOperatingSystemRelease: 3.0.6
GlueHostOperatingSystemVersion: SLC
GlueHostProcessorClockSpeed: 2392
GlueHostProcessorModel: Dual Core Opteron 280
GlueHostProcessorVendor: AMD
Grid Computing Course, Physics Dept, Univ. of Catania

14. Computing Element
Grid Computing Course, Physics Dept, Univ. of Catania 14

15. GLUE: Host Grid Computing Course, Physics Dept, Univ. of Catania
GlueCEName: cometa
GlueCEUniqueID: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa
GlueCEInfoGatekeeperPort: 2119
GlueCEInfoHostName: infn-ce-01.ct.trigrid.it
GlueCEInfoLRMSType: lsf
GlueCEInfoLRMSVersion: 6.1
GlueCEInfoTotalCPUs: 98
GlueCEInfoJobManager: lcglsf
GlueCEInfoContactString: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa
GlueCEInfoApplicationDir: /opt/exp_soft
GlueCEInfoDataDir: unset
GlueCEInfoDefaultSE: infn-se-01.ct.trigrid.it
GlueCEStateEstimatedResponseTime: 61713
GlueCEStateFreeCPUs: 26
GlueCEStateRunningJobs: 70
GlueCEStateStatus: Production
GlueCEStateTotalJobs: 70
GlueCEStateWaitingJobs: 0
GlueCEStateWorstResponseTime:
GlueCEStateFreeJobSlots: 26
GlueCEPolicyMaxCPUTime: 2880
GlueCEPolicyMaxRunningJobs: 98
GlueCEPolicyMaxTotalJobs: 0
GlueCEPolicyMaxWallClockTime: 2880
GlueCEPolicyPriority: -10
GlueCEPolicyAssignedJobSlots: 98
GlueCEAccessControlBaseRule: VO:cometa
Grid Computing Course, Physics Dept, Univ. of Catania

16. Computing Element Storage Element Storage Area Access protocols
Information about the
service
(like Name,Port,URL)
Storage Area
Contains info of available and used disk space,file policies,
access rules,etc.
Access protocols
Contains info about
the protocols used
to transfer files
Grid Computing Course, Physics Dept, Univ. of Catania 16

17. GLUE: Storage Element GlueSEUniqueID: infn-se-01.ct.trigrid.it
GlueSEName: TRIGRID-INFN-CATANIA:srm_v1
GlueSEPort: 2811
GlueSESizeTotal: 16350
GlueSESizeFree: 16350
GlueSEArchitecture: multidisk
GlueInformationServiceURL: ldap://infn-se-01.ct.trigrid.it:2135/mds-vo-name=lo
cal,o=grid
Grid Computing Course, Physics Dept, Univ. of Catania

18. GLUE: Storage Area GlueSARoot: cometa:/dpm/ct.trigrid.it/home/cometa
GlueSAPath: /dpm/ct.trigrid.it/home/cometa
GlueSAType: permanent
GlueSALocalID: cometa
GlueSAPolicyMaxFileSize: 10000
GlueSAPolicyMinFileSize: 1
GlueSAPolicyMaxData: 100
GlueSAPolicyMaxNumFiles: 10
GlueSAPolicyMaxPinDuration: 10
GlueSAPolicyQuota: 0
GlueSAPolicyFileLifeTime: permanent
GlueSAStateAvailableSpace:
GlueSAStateUsedSpace: 0
GlueSAAccessControlBaseRule: cometa
Grid Computing Course, Physics Dept, Univ. of Catania

19. GLUE: Access Protocols
GlueSEAccessProtocolLocalID: gsiftp
GlueSEAccessProtocolType: gsiftp
GlueSEAccessProtocolEndpoint: gsiftp://infn-se-01.ct.trigrid.it
GlueSEAccessProtocolCapability: file transfer
GlueSEAccessProtocolVersion: 1.0.0
GlueSEAccessProtocolPort: 2811
GlueSEAccessProtocolSupportedSecurity: GSI
GlueSEAccessProtocolLocalID: rfio
GlueSEAccessProtocolType: rfio
GlueSEAccessProtocolEndpoint: httpg://infn-se-01.ct.trigrid.it
GlueSEAccessProtocolCapability: byte access
GlueSEAccessProtocolPort: 5001
GlueSEAccessProtocolSupportedSecurity: RFIO
Grid Computing Course, Physics Dept, Univ. of Catania

20. LCG Information System
Grid Computing Course, Physics Dept, Univ. of Catania

21. LCG Information System
LCG adopts a combination of solutions
Globus MDS
At the lowest level of the information system
To discover and monitor resources and publish information
Grid Information Security (GSI) credentials
Caching
BDII
At the highest level of the system
Because MDS had some troubles in terms of scalability
Used by the Resource Broker for the matchmaking process
Can be configured by each VO
Queries underlying systems periodically (2 minutes)
Hierarchical system
Information is collected on the leaves of a hierarchical tree and travels towards the root
Clients can query the hierarchical tree at every level
The higher the level against which queries are made, the older is the obtained information
Grid Computing Course, Physics Dept, Univ. of Catania

22. Collecting Information
Gathering of information at different levels
Lower level: Grid Resource Information Server (GRIS)
Collects information on the state of a given resource
One GRIS on top of each resource
A set of scripts and sensor that try to extract useful info on the resource
Medium level: Grid Index Information Server (GIIS)
Collects information on resources of a given site
One GIIS for each site
Higher level: BDII
Collects information on resources of a given VO
One BDII for each VO (suggested solution)
Way of collecting info
Pull model (higher level servers periodically query lower level servers)
LDAP query model
Grid Computing Course, Physics Dept, Univ. of Catania

23. BDII overview The Berkley Database Information Index (BDII)
Developed within the context of LCG project
Solves problems of instability of the MDS occurring when the number of sites grows too much
Stays on top of GIIS sites
One for each VO
Centralized system
Three levels of hierarchy
Accessed by the Workload Management System
Way of working
One GRIS for each resource
One GIIS for each site collecting info from below GRIS systems
One BDII for a given VO collecting information from below GIIS systems
Two LDAP servers, one for write access and one for read access
Every two minutes a cron-job runs a script and collects info from a list of GIIS sites
The list of GIIS is placed in the configuration file of the BDII
Grid Computing Course, Physics Dept, Univ. of Catania

24. LCG Information System Hierarchy
Top BDII RB
Site BDII A
SE GRIS
CE GRIS
Site A
Site BDII C
SE GRIS
CE GRIS
Site C
Site BDII B
SE GRIS
CE GRIS
Site B
Grid Computing Course, Physics Dept, Univ. of Catania

25. BDII overview
Every node (except UI,BDII and Wns) has a GRIS service in order to publish its informations
A node in every site collects all site GRISes and publishes them using a site BDII, by default installed on the CE
The top BDII collects all site BDIIs
User can run a set of commands to query the top BDII.
Grid Computing Course, Physics Dept, Univ. of Catania

26. Top BDII vs Site BDII Site BDII By default it is installed on the CE
It collects all site GRISes* (for example SE,RB,LFC,etc..)
The name of the service is bdii
Top BDII
It collects all site BDIIs** (CE1,CE CEn)
It gives to the RB all needed informations to match and dispatch user's jobs
It can run in the same machine where the RB is running (it's more fast in answer)
*GRIS=Grid Resource Information Service
**BDII=Berkely Database Infomatin Index
Grid Computing Course, Physics Dept, Univ. of Catania

27. LCG Information System Hierarchy
BDII-A
BDII-B
BDII-C
Site BDII
Site BDII
Site BDII SE
Local GRIS SE
Local GRIS CE
Local GRIS SE
Local GRIS CE
Local GRIS SE
Local GRIS RB
Local GRIS
Site 1
Site 2
Site 3
Grid Computing Course, Physics Dept, Univ. of Catania

28. Generic Information Provider
The Generic Information Provider, GIP, is a highly configurable information provider that makes a separation between static and dynamic information. It can be used to produce any kind of information for use with LDAP based grid information systems.
An information provider in its simplest form is a script that prints an LDIF file to standard out. The LDIF file should conform to the schema used in the GIP.
The problem is with dynamic information. Usually there are only a few attributes that have to be found dynamically. The idea with the GIP is to use dynamic plug-ins to obtain these values and use a common framework for everything else.
Grid Computing Course, Physics Dept, Univ. of Catania

29. GIP configuration file
The GIP configuration file contains all the configuration parameters for the GIP.
temp_dir
The directory to use as the GIPs cache.
plugin_dir
The directory where to put links to the dynamic plugins.
static_dir
The directory where to put inks to the static ldif files.
provider_dir
The directory where to put links to the information providers.
freshness
How long to use the cache before running the dynamic plugins again.
cache_ttl
How long the cache is valid.
response
How long the GIP will wait for dynamic plugins before continuing.
timeout
Timeout for the dynamic plugins in seconds.
Grid Computing Course, Physics Dept, Univ. of Catania

30. GIP
The GIP script reads in all the ldif files from the static_dir and run all the providers in the plugin_dir. It will then run any dynamic plug-ins found in the plugin_dir to obtain the dynamic values. It will print the LDIF to standard out but use the any dynamic values found from the plugins.
Plugins and providers should not be put directly into the directory, insted a symbolic link or wrapper script should be used.
Grid Computing Course, Physics Dept, Univ. of Catania

31. SiteInfo Information Provider
dn: in=host.invalid/siteinfo,Mds-Vo-name=local,o=grid
objectClass: SiteInfo
objectClass: DataGridTop
objectClass: DynamicObject
siteName: SITE-NAME
sysAdminContact:
userSupportContact:
siteSecurityContact:
dataGridVersion: LCG-2_2_0
installationDate: Z
Grid Computing Course, Physics Dept, Univ. of Catania

32. SiteInfo Schema objectClass: DataGridTop objectClass: SiteInfo
in: information
siteName: Name of site
sysAdminContact: Contact details for local system administrator
userSupportContact: Contact details for local user support
siteSecurityContact: Contact details for site security
dataGridVersion: Overall DataGrid software version
installationDate: Date of installation
cpuResourceDescription: Description of available CPU resources
diskResourceDescription: Description of available disk resources
supportedFileSystem: Supported file systems AFS, NFS, local
batchSystem: Available batch system, PBS, LSF, BQS, Condor, fork
etc massStorageDescription: Description of available mass storage
experimentalSoftware: Availablity of software, ATLAS, CMS, ESA etc…
Grid Computing Course, Physics Dept, Univ. of Catania

33. BDII Update Process (1)
The BDII consists of two or more standard LDAP databases that are populated by an update process.
The update process obtains LDIF (LDAP Data Interchange Format ) file from either doing an ldapsearch on LDAP URLs or by running a local script that generates LDIF. The LDIF is then inserted into the LDAP database.
Options exist to update the the list of LDAP URLs from a web page and to use an LDIF file from a web page to modify the data before inserting it into the database.
The daemon script will start the update process is bdii-update.
Grid Computing Course, Physics Dept, Univ. of Catania

34. BDII Update Process (2)
The update process does the following in an endless loop:
Update configuration file that contains the LDAP URLs from the web.
Fork off processes to obtain the LDIF from the URLs.
Write the LDIF files in the temporary directory.
Try to obtain an LDIF file that will modify (filter) the data.
Modify the data read from the temporary files.
Kill the oldest database instance along with any stale clients.
Clear the corresponding database.
Add the fresh data to a new database.
Start the new database.
Forward traffic from the BDII read port to the new database.
Sleep for a while.
Grid Computing Course, Physics Dept, Univ. of Catania

35. lcg-info
Grid Computing Course, Physics Dept, Univ. of Catania 36

36. Praticals: lcginfo -h/--help: to see the help
–list-attrs: print the list of the possible attributes
--list-ce: lists the CEs which satisfy a query, or all the CEs if no query is given.
--list-se: lists the SEs which satisfy a query, or all the SEs if no query is given.
--bdii: allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error.
--vo: restricts the output to CEs or SEs where the given VO is authorized.
Grid Computing Course, Physics Dept, Univ. of Catania

37. lcg-infosites
Grid Computing Course, Physics Dept, Univ. of Catania 38

38. Praticals: lcg-infosites
-h/--help: help option
--vo: VO name (mandatory)
--is: it's possible to specify a not default Top BDII
Some options:
se: The names of the SEs supported by the user's VO
ce: The information relative to number of CPUs, running jobs,etc.
rb: Names of the Rbs available for each VO
sitenames: Names of the LCG sites
tag: The names of the tags relative to the software installed in site is printed together with the corresponding CE
closeSE: The names of the CEs where the user's VO is allowed to run together with their corresponding closest SEs are provided
Grid Computing Course, Physics Dept, Univ. of Catania

39. References Esercitazione LCG-2 User Guide GLUE Schema
LCG-2 User Guide
GLUE Schema
Grid Computing Course, Physics Dept, Univ. of Catania