1. gLite Grid Services Riccardo Bruno (riccardo.bruno@ct.infn.it)
INFN Dept. Catania
Joint EPIKH/EUMEDGRID Support event in Algiers
Africa 3 – Site Admin Tutorial + Remote site inst.
Algiers,

2. What gLite does? What gLite does not What gLite does
Somehow my application just run faster
My application can run as long as it needs
Users can access to any resource
Users can rely of a huge amount of software, libraries
What gLite does
Provides sharing of resources (CPUs, Storage, Sensors …)
Allows the creation of virtual organizations (People,Groups)
Applications will run faster only if properly developed and best if thought for Grid environments (Trivial parallelization, MPIs)
Provides access to computational/storage/other resources accordingly to defined: Policies and Access rights

3. Scenario Usage scenario
Many users of different organizations geographically ditributed (Virtual Oranizations VOs) requesting high computational and storage capacities, collaborating each other

4. Scenario Usage scenario
Many users of different organizations geographically ditributed (Virtual Oranizations VOs) requesting high computational and storage capacities, collaborating each other
Many computational resources (computing power and storage) belongs to different institutions but transparently accessible

5. gLite Grid gLite Grid System aims to: Integrate Virtualize Manage
RESOURCEs and SERVICEs across different VOs
The gLite middleware is the set of software packages able to do this

6. Definitions Computational Resource Job Job Submission
The machine where users can execute programs (called jobs), can store or access to stored data in a transparent fashion; no matter where they are located.
Job
An executable or a batch script that the users want to execute on the Worker Node
Job Submission
Is the action when the best available computational resource is assigned by the system to the user in order to execute the job.

7. Overview INTERNET The user joins to a VO
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

8. Overview INTERNET The user joins to a VO
Each VO shares grid resources to other VOs accordingly to several policies.
INTERNET
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

9. Overview INTERNET The user joins to a VO
Each VO shares grid resources to other Vos accordingly to several policies.
The Grid middleware allow to use and share:
“Computing Elements (CE)”
“Storage Element (SE)”
INTERNET
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

10. Overview INTERNET The users join to a VO
Each VO shares grid resources to other Vos accordingly to several policies.
The Grid middleware allow to use and share:
“Computing Elements (CE)”
“Storage Element (SE)”
Plus Additional services to empower the capabilities of the Grid
INTERNET
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

11. Overview COLLABORATION INTERNET The users join to a VO
Each VO shares grid resources to other Vos accordingly to several policies.
The Grid middleware allow to use and share:
“Computing Elements (CE)”
“Storage Element (SE)”
Plus Additional servicees to empower the capabilities of the Grid
Result:
COLLABORATION
INTERNET
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

12. Grid Requirements Heterogeneous (OSes, Devs, Apps.)
VO Resource Sharing (Management, Security and Accounting)
Resource Utilisation (Reservation, Metering, Monitoring and Logging)
Job Execution (VO access, QoS, LCM, WFM, SLA)
Data Services (Integration, Provisioning, Cataloguing, Metadata)
Security (Authentication, Authorisation and Auditing)
Administrative Costs (Provisioning, Deployment and Configuration)
Scalability
Availability (Disaster Recovery, Fault Management)
Specific Requirements: (EGEE: HEP, BioMed)
QoS – quality of service
LCM – Local Credential Maping
WFM – Workflow management system
SLA – Service Layer Architecture

13. gLite - Lightweight Middleware for Grid Computing
gLite evolution
gLite - Lightweight Middleware for Grid Computing
GTK2
Web services based
gLite-2
gLite-1
LCG-2
LCG-1
gLite-3
GTK4

14. gLite Main components services
UI: User Interface
WMS: Workload management system
LB: Logging and bookkeeping service
VOMS: Virtual Organization Management service
BDII: Information system and MON, (going to be deprecated)
CE: Computing element (LCG/gLite)
WN: Worker nodes
SE: Storage element
LFC: File catalog
AMGA: Metadata catalog

15. Components
The User Interface (UI) is the user entry point normally considered the user workstation. It is normally considered as a WMS component.
The Workload Management System (WMS) its a set of services having the responsability to find the best available computing element where submit user’ jobs in a transparent fashion
The Logging and bookkeeping service (LB), keep track of user job execution in terms of statuses: Ready, Scheduled, Waiting, Running, Done
The Computing element (CE) is the computational resource, the entry point to a cluster or PCs handled by a job queue management system; in particular: TORQUE, PBS, LSF, CONDOR
The Worker Nodes are the machines where jobs are really executed and managed by the CE’ queue management system
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

16. Components
The Information System and Monitoring (IS and MON now deprecated), maintain data related to available grid resources and their health statuses.
The Virtual Organization Management service (VOMS) , is the way gLite improves the management of authentication and authorization to the Grid resources. The VOMS allows to their own members to define different access rights to VO’ resources
The Storage element (SE) and the File catalogue (LFC), allow to manage Grid files and offer a mechanism to locate them easily for users and jobs.
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

17. Job life cycle
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

18. Information system & Monitoring Job Workload Management System
gLite services
gLite services can be grouped in 5 main high level set of services
Grid Access
Security
Information system & Monitoring
Job Workload Management System
Data Management

19. gLite – Grid access Two possibilities: APIs or CLI
Built on top of them there exist GridPortals and GUIs
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

20. gLite Security User authentication is based on X.509 (PKI)‏
Authorized Certification Authorities (CA) can generate user and service certificates who identify univocally people or Grid services in the whole Grid
Each Grid service may support or not certificates coming from different CAs
To reduce the vulnerabilities the identification of users in to the grid is done through the use of proxy certificates. Proxyes are singed ‘copies’ of the original user certificate, having a limited lifetime.
The use of Proxy certificates allows the following:
Delegation: Any grid service can operates on the user behalf making signed copies of the original proxy. (Single Sign On)
Add additional info (Add VO specific information provided by VOMS)‏
Store a long term proxy on a secure server (MyProxy)
Renewal (A Proxy close to the expiration time can be automatically renewed)‏

21. gLite Security: AutH/AhtZ
Authentication
The user receive a certificate from a CA (PKI third party)
He connects to the UI via SSH
He Creates the proxy (single sign on) – All grid services will use this proxy to identify the user.
Authorization
The user has to subscribe to a VO (VOMS)
The VO establishes the user rights
In any Grid service it will be verified if the user belongs to the VO and assigns the proper access rights to the user
A special configuration file named the “gridmapfile”, maintains the correspondency between grid users and resource users (unix pool accounts)

22. VOMS Virtual Organization Membership Service
Manages many Virtual Organizations (VOs).
Multiple user roles can be defined inside each VO
Extends the X509 schema
Extensions are Digitally Signed
Service maintenance provided by a web front-end
Support MyProxy (stored proxies)
Allow the access rights by VO or by Role
Each Grid site associates to each VO member or role a a local authentication (unix pool accounts)‏.
Allows to implement fine grained security policies to grid resources
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

23. VOMS
MyProxy
Stores a long term proxy certificates to allow the automatic proxy renewal mechanism
Allow to execute jobs requesting a computation time larger that the normal proxy liferime (normally 12 hrs)
The WMS is the responsable of the proxy renewal
Users should not use long lived proxy directly
Allow the user to access grid resources without carrying out the public and private keys.
Proxy Delegation
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

24. Information Systema and Monitoring
Berkeley Database Information Index (BDII)
The information hierarchically stored via tree modeling (The LDAP implementation of GLUE)
GRIS Stores information at resource level
Site BDII/GIIS(deprecated) Stores information at site level
BDII Stores information at VO level
VO Level
Site Level
Resource Level
To change: View -> Header and Footer

25. Two core components of WMS
Workload Management
WMS set of middleware components responsible of distribution and management of jobs across Grid resources.
Two core components of WMS
WM: accepts and satisfy requests for job management. (Matchmaking) is the process of assigning the best available resource.
Logging & Bookeeping : keeps track of job execution in term of events: (Submitted, Running, Done,...)

26. Computing Element
Service that represents the computing resource that is responsible of to manage the queue of jobs to execute
The CE may be used by a Generic Client: an end-user interacting directly with the Computing Element, or
by the Workload Manager, which submits a given job to an appropriate CE found by the matchmaking process.
Two job submission models :
PUSH (Eager Scheduling) (jobs pushed to CE),
PULL (Lazy Scheduling) (jobs coming from WMS when CE has free slots)

27. Computing Element: Architecture
CEA Computing Element Acceptance
JC Job Controller
MON Monitoring
LRMS Local Resource Management System
EELA-2 Tutorial in Cuenca, Ecuador. Universidad de Cuenca, 02/02/ /02/2010

28. Data are stored on Disk Pool Servers or Mass Storage Systems
Storage Element
SE Services are at least:
Storage back-end (Drivers and Hardware)
Storage Resource Manager (SRM) Interface
(Interface to manage the specific storage solution: dpm, rfio, …)
Transfer service (Protocols: GridFTP(gsiftp), glubus-url-copy, …)
Native POSIX like file I/O API (GFAL)
Auxiliary Accounting and Logging services (accounting: SAGE)
Data are stored on Disk Pool Servers or Mass Storage Systems

29. LFC File Catalog OS like file access metaphor. LFN (Logical file name)
GUID (Grid unique identifier)
SimLinks
SURL (Site URL)
TURL (Transfer URL)

30. AMGA – Arda Metadata Grid Application
Metadata is data about data
On the Grid: information about files
Describe files
Locate files based on their contents through queries
But also makes DB access a simple task on the Grid
Many Grid applications need structured data
Many applications require only simple schemas
Can be modelled as metadata
Main advantage: better integration with the Grid environment
Metadata Service is a Grid component
Grid security
Hide DB heterogeneity

31. Grid Services and their interactions
Grid Access
User Iinterface
Info system
Security
MyProxy (Normal, Long term)
VOMS
Job submission
WMS
Computing Element
Worker Node
Data management
Catalogs
Storage elements
BDII

32. Questions …