1. IST-2006-026409 www.eu-eela.org E-infrastructure shared between Europe and Latin America Overview of gLite Middleware Pedro Henrique Rausch Bello Instituto de Física - UFRJ Third EELA Tutorial for users and managers Rio de Janeiro, 26.06.2006

2. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 2 This presentation is based on previous work from: –Claudio Grandi –Roberto Barbera –Grupo Grid, ULA

3. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 3 Overview of the gLite Middleware The gLite Grid services follow a Service Oriented Architecture –facilitate interoperability among Grid services –allow easier compliance with upcoming standards Architecture is not bound to specific implementations –services are expected to work together –services can be deployed and used independently The gLite service decomposition has been largely influenced by the work performed in the LCG project

4. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 4 Overview of the gLite MiddlewareReplicaCatalogue UI JDL Logging & Book-keeping ResourceBroker Job Submission Service StorageElement ComputeElement InformationService Job Status DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish grid-proxy-init Expanded JDL SE & CE info

5. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 5 gLite process Process controlled by the Technical Coordination Group (TCG) Task Forces with developers, applications, testers and deployment experts After gLite 3.0 adopt a continuous release process –No more big-bang releases with fixed deadlines for all –Develop components as requested by users and sites –Deploy or upgrade as soon as testing is satisfactory Major releases synchronized with large scale activities of VOs (SCs) –Next major release foreseen for the autumn

6. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 6 SA3 Testing & Certification Functional Tests Testbed Deployment gLite Software Process JRA1 Development Software Error Fixing SA3 Integration Deployment Packages Integration Tests Installation Guide, Release Notes, etc SA1 Pre- Production Scalability Tests Pre-Production Deployment Fail Pass SA1 Production Infrastructure Release Problem Serious problem Directives

7. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 7 gLite status and plans In the following the current status and the plans for each component are given For all components the main priorities for the developers are: –support on the production infrastructure (GGUS, 2 nd line support) –bug-fixing –improve robustness and usability (efficiency, error reporting,...) –support for SL(C)4 and for x86-64 and IA64 –addressing requests for functionality improvements from users, site administrators, etc... (through the TCG) –Task Forces together with applications and site experts In parallel, and with the help of EUChinaGrid: –support for IPv6

8. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 8 gLite Architecture gLite Services

9. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 9 Middleware structure Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed to help the users building their computing infrastructure but should not be mandatory Foundation Grid Middleware will be deployed on the EGEE infrastructure –Must be complete and robust –Should allow interoperation with other major grid infrastructures –Should not assume the use of Higher-Level Grid Services

10. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 10 Foundations: Security Security Infrastructure is paramount in a grid environment, where site managers have to delegate user trust to a third party Consists of three main objectives: –Authentication (AuthN): WHO a user is –Authorization (AuthZ): WHAT a user is allowed to do –Auditing: Post-mortem analysis of security related events Authorization Security Services Authentication Auditing

11. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 11 Security – AuthN and AuthZ Authentication: Identify entities (users, systems and services) when establishing context for message exchange (Who are you?). Authorization: Allows or denies access to services, based on policies.

12. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 12 Security – Authentication (AuthN) Authentication based on X.509 PKI infrastructure –Certificate Authorities (CA) issue (long lived) certificates identifying individuals (much like a passport)  Commonly used in web browsers to authenticate to sites –Trust between CAs and sites is established (offline) –In order to reduce vulnerability, on the Grid user identification is done by using (short lived) proxies of their certificates Proxies can –Be delegated to a service such that it can act on the user’s behalf –Include additional attributes (like VO information via the VO Membership Service VOMS) –Be stored in an external proxy store (MyProxy) –Be renewed (in case they are about to expire)

13. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 13 Security – Authorization (AuthZ) Bare certificates are not enough for defining user capabilities on the Grid Users belong to VO’s, to groups inside a VO and may have special roles VOMS provides a way to add attributes to a certificate proxy: –mutual authentication of client and server –VOMS produces a signed Attribute Certificate (AC) –the client produces a new proxy that contains the attributes The attributes are used to provide the user with additional capabilities according to the VO policies. VOMS is in gLite 3.0 Query Authentication Request Auth DB C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS AC

14. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 14 Security – Authorization (AuthZ) VOMS –Manages VO Membership –Provides support for Groups and Roles –Support for MySQL and Oracle DB backend Deployed for some time (even before gLite 3.0) on the Production Infrastructure –Interfaced with OSG’s VOMSRS

15. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 15 Foundations: Information Systems Now the user can access the Grid. But how does she finds information about resources available? Information Systems are used to: – gather information about grid resources – gather job information Information & Monitoring Services Information & Monitoring Job Monitoring Service Discovery

16. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 16 Information Systems – GIP and BDII Generic Information Provider (GIP) –Provides LDIF information about a grid service in accordance to the GLUE Schema BDII: Information system in gLite 3.0 (by LCG) –LDAP database that is updated by a process –More than one DBs is used separate read and write –A port forwarder is used internally to select the correct DB Freedom of choice portal: VOs can white- or black-list resources so that BDII DBs are updated accordingly Sites failing Site Functional Tests may also be excluded 2171 LDAP 2172 LDAP 2173 LDAP 2170 Port Fwd Update DB & Modify DB 2170 Port Fwd Swap DBs GIP Provider Config File LDIF File Plugin Cache

17. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 17 Information Systems – RGMA and SD R-GMA: provides a uniform method to access and publish distributed information and monitoring data –Used for job and infrastructure monitoring in gLite 3.0 –Working to add authorization Service Discovery: –Provides a standard set of methods for locating Grid services –Currently supports R-GMA, BDII and XML files as backends –Will add local cache of information –Used by some DM and WMS components in gLite 3.0

18. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 18 Grid foundation: Computing Element LCG-CE: based on GT2 GRAM –To be replaced when other CEs prove reliability gLite-CE: based on GSI enabled Condor-C –Supported by Condor. More efficient. Uses BLAH (see below) –Deployed for the first time on the PS in gLite 3.0 CREAM: new lightweight web service CE –Not in gLite 3 release. Will need exposure to users on dedicated system. –WSDL interface –Will support bulk submission of jobs from WMS and optimization of input/output file transfer. Uses BLAH –Plan is to have a CE with both Condor-C and CREAM interfaces

19. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 19 Grid foundation: Computing Element BLAH: interfaces the CE and the local batch system –May handle arbitrary information passing from CE to LRMS  patches to support this and logging for accounting being added now –Used by gLite-CE and CREAM CEMon: Web service to publish status of a computing resource to clients –Supports synchronous queries and asynchronous notification –Uses the same information (GIP) used by BDII –In gLite 3 CEMon will be available to the users but the baseline is that the WMS queries the bdII

20. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 20 Grid foundation: Accounting APEL: Uses R-GMA to propagate and display job accounting information for infrastructure monitoring –Reads LRMS log files provided by LCG-CE and BLAH –Preparing an update for gLite 3.0 to use the files form BLAH DGAS: Collects, stores and transfers accounting data. Compliant with privacy requirements –Reads LRMS log files provided by LCG-CE and BLAH. –Stores information in a site database (HLR) and optionally in a central HLR. Access granted to user, site and VO administrators –Not yet certified in gLite 3.0. Deployment plan:  certify and activate local sensors and site HLR in parallel with APEL  replace APEL sensors with DGAS (DGAS2APEL)  certify and activate central HLR. Test scalability to the PS scale

21. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 21 Grid foundation: Storage Element Storage Element –Common interface: SRMv1,migrating to SRMv2 –Various implementation from LCG and other external projects  disk-based: DPM, dCache; tape-based: Castor, dCache –Support for ACLs in DPM, in future in Castor and dCache  After the summer: synchronization of ACLs between SEs –Common rfio library for Castor and DPM being added Posix-like file access: –Grid File Access Layer (GFAL) by LCG  Support for ACL in the SRM layer (currently in DPM only)  Support for SRMv2 being added now. In the summer add thread safety and interface to the information system –gLite I/O  Support for ACLs from the file catalog and interfaced to Hydra for data encryption  Not certified in gLite 3.0. To be dismissed when all functionalities are in GFAL.

22. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 22 High Level Services: Catalogues File Catalogs –LFC from LCG  This month: interface to POOL  In the summer: LFC replication and backup –Fireman  Not certified in gLite 3.0. To be dismissed when all functionalities are in LFC Hydra: stores keys for data encryption –Being interfaced to GFAL (finish by July) –Currently only one instance, but in future there will be 3 instances: at least 2 need to be available for decryption –Not yet certified in gLite 3.0. Certification will start soon AMGA Metadata Catalog: generic metadata catalogue –Joint JRA1-NA4 (ARDA) development. Used mainly by Biomed –Not yet certified in gLite 3.0. Certification will start soon

23. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 23 Client High Level Services: Data Encryption Application MDM Client Library Grid Catalogs Metadata Catalog (AMGA) Medical Imager Encryption Keystore (Hydra) File Catalog (Fireman) SRM DICOM MDM Trigger GridFTP gLite I/O Trigger: Retrieve DICOM files from imager. Register file in Fireman gLite EDS client: Generate encryption keys and store them in Hydra Register Metadata in AMGA Client Library: Lookup file through Metadata (AMGA) Use gLite EDS client: Retrieve file through gLite I/O Retrieve encryption Key from Hydra Decrypt data Serve it up to the application Prototype for medical data management (SRM/DICOM)

24. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 24 High Level Services: File transfer FTS: Reliable, scalable and customizable file transfer –Manages transfers through channels  mono-directional network pipes between two sites –Web service interface –Automatic discovery of services –Support for different user and administrative roles –Adding support for pre-staging and new proxy renewal schema –In the medium term add support for SRMv2, delegation, VOMS-aware proxy renewal

25. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 25 High Level Services: Workload manag. WMS helps the user accessing computing resources –Resource brokering, management of job input/output,... LCG-RB: GT2 + Condor-G –To be replaced when the gLite WMS proves reliability gLite WMS: Web service (WMProxy) + Condor-G –Management of complex workflows (DAGs) and compound jobs  bulk submission and shared input sandboxes  support for input files on different servers (scattered sandboxes) –Support for shallow resubmission of jobs –Job File Perusal: file peeking during job execution –Supports collection of information from CEMon, BDII, R-GMA and from DLI and StorageIndex data management interfaces –Support for parallel jobs (MPI) when the home dir is not shared –Deployed for the first time on the PS with gLite 3.0

26. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 26 High Level Services: Workflows Direct Acyclic Graph (DAG) is a set of jobs where the input, output, or execution of one or more jobs depends on one or more other jobs A Collection is a group of jobs with no dependencies –basically a collection of JDL’s A Parametric job is a job having one or more attributes in the JDL that vary their values according to parameters Using compound jobs it is possible to have one shot submission of a (possibly very large, up to thousands) group of jobs –Submission time reduction  Single call to WMProxy server  Single Authentication and Authorization process  Sharing of files between jobs –Availability of both a single Job Id to manage the group as a whole and an Id for each single job in the group nodeE nodeC nodeA nodeD nodeB

27. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 27 High Level Services: WMS plans Foreseen updates: –Short term: migration to GLUE 1.2 (see job priorities), bug fixing (shallow resubmission of compound jobs) –Medium term: new Condor (increase efficiency), fixes to increase performance, job prologue/epilogue, short jobs (SDJ), improve interactive access to jobs (e.g. top, ls), UI round-robin –Long term: bulk match making, high availability RB, ICE (interface to CREAM for job submission)

28. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 28 High Level Services: Job Information Logging and Bookkeeping service –Tracks jobs during their lifetime (in terms of events) –LBProxy for fast access –L&B API and CLI to query jobs –Support for “CE reputability ranking“: maintains recent statistics of job failures at CE’s and feeds back to WMS to aid planning Job Provenance: stores long term job information –Supports job rerun –If deployed will also help unloading the L&B –Not yet certified in gLite 3.0. Certification will start when requested by the TCG

29. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 29 High Level Services: Job Priorities GPBOX: Interface to define, store and propagate fine- grained VO policies –Based on VOMS groups and roles –Enforcement of policies at sites: sites may accept/reject policies –Not yet certified in gLite 3.0. Certification will start when requested by the TCG. Current plans: test job prioritization without GPBOX: 1.Mapping of VOMS groups to batch system shares (via GIDs?) 2.Two queues (long/short) for ATLAS & CMS 3.Publish info on the share in the CE GLUE 1.2 schema (VOView)  The gLite WMS is being modified to support GLUE 1.2  Testing with GPBOX if the “Service Class” is published 4.WMS match-making depending on submitter VOMS certificate  But no ranking of resources based on priority offered yet 5.Settings are not dynamic (via e-mail or CE updates)

30. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 30 Summary gLite 3 being deployed on the production infrastructure –Includes all of the well known middleware from LCG 2.7.0 New components from gLite 1.5 being deployed for the first time on the Production Infrastructure –Address requirements in terms of functionality and scalability –Components deployed for the first time need extensive testing! Developed according to a well defined process –Controlled by the EGEE Technical Coordination Group Development is continuing to provide increased robustness, usability and functionality

31. IST-2006-026409 E-infrastructure shared between Europe and Latin America www.eu-eela.org Rio de Janeiro, 3 rd EELA Tutorial, 26.06.2006 31 Questions?