1. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA www.eu-eela.org E-infrastructure shared between Europe and Latin America gLite Overview Grupo Grid Universidad de Los Andes Mérida, 24-29 April 2006

2. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 2 This presentation is based in previews presentations from: –Riccardo Bruno, –Salvatore Scifo –Mike Mineter

3. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 3 Introduction

4. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 4 Middleware Grid Middleware – Layer between user applications and grid resources.

5. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 5 Grid Concepts VOs:Individuals and/or Institutions having direct access to resources. User Interface Grid services

6. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 6 Introduction Grid Sistems & Applications aim is to: –Integrate –Virtualise –Manage Resources and services across different VOs.

7. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 7 Grid Requirements Heterogeneous VO Resource Sharing Resource Utilization Job Execution Data Services Security Administrative Costs Scalability Availability Specific Requeriments

8. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 8 Multi-VOs Users join VOs Virtual organisation contributes resources & negotiates access Grid middleware runs on each resource –“Storage elements” –“Compute elements” Additional services (both people and grid middleware) enable the grid Effect: collaboration INTERNET

9. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 9 Building on GSI Build on Grid Security Infrastructure to create services that include: –Job submission: run a job on a remote computer –Information services: So I know which computer to use –File transfer: so large data files can be transferred –Replica management: so I can have multiple versions of a file “close” to the computers where I want to run jobs Production grids are (currently) based on the Globus Toolkit release 2 Globus Alliance: http://www.globus.org/

10. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 10 Convergence of Web Services and Grids Grid prototypes web developments Web services “big Science” research INTERNET World-wide web High throughput- computing Massively parallel computing High-end computing Open Grid Services Architecture

11. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 11 gLite

12. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 12 gLite gLite is the next generation middleware for grid computing. Born from the collaborative efforts from academic and industrial research centers as part of the EGEE Project. The gLite Grid services follow a Service Oriented Architecture –facilitate interoperability among Grid services –allow easier compliance with upcoming standards Architecture is not bound to specific implementations –services are expected to work together –services can be deployed and used independently The gLite service decomposition has been largely influenced by the work performed in the LCG project

13. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 13 gLite - Middleware Many VOs need sharing of resources through services –Accessing –Allocating –Monitoring –Accounting gLite – Lightweight Middleware for Grid Computing

14. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 14 gLite – Service Decomposition 5 High level services + CLI & API

15. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 15 gLite – Security Services

16. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 16 gLite – Security Services Authentication Identify entities (users, systems and services) when establishing context for message exchange (Who are you?). Aim - Provide a Credential having a universal value that works for many purposes across many infrastructures, communities, VOs and projects. gLite uses the PKI (X.509) infrastructure using CAs as thrusted third parties. MyProxy (http://grid.ncsa.uiuc.edu/myproxy/) Trust domain: The set of all EGEE CAs is our Trust Domain. Revocation: Identities must be revocated timely Credential Storage: Local or delegated credential (Services or Users) Privacy Preservation: Use of personal data

17. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 17 gLite – Security Services Authorization Allows or denies access to services, based on policies. Agent: The user interacts with a centralized Authorization Server Push*: Authorization Services issue Tokens. Pull: The resource asks to the Authorization Services. Authorization Sources: Attribute Authority (AA): User Set of Attibutes. (VOMS) Policy Assertions: Third party policies. (CAS)

18. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 18 gLite – Security Services Auditing, Delegation, Sandboxing Auditing - Monitoring and Post-Mortem analysis of security related events. In computational grids It goes hand by hand with the accounting. Who did what? Where and when? In case of accounting: For how long? For how much? Delegation: The need of delegate privileges to other entities is done by Proxy Certificates. This is the most widely adopted mechanism by Grid communities. (Also: Single Sign-On, Dynamic entity identification). Sandboxing - Grid applications need the isolation of assigned resources in a transparent fashion by Security services: AuthN and AuthZ. (Virtualisation).

19. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 19 gLite – Grid Access Two possibilities: APIs and CLI. The use of web-services allows the automatic generation of APIs (error prone, lack of tools).

20. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 20 gLite – Information and Monitoring Services Information services are vital low level component of Grids.

21. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 21 gLite – Information and Monitoring Services Basic info and monitoring services (RGMA) Information is provided by a Publish and Consume mechanism. Appearance of a single federated database to query through the SQL. Each VO has a VDB. Schema - Contains tables (GLUE) Registry – List of available sources of information (Mediation) Producers – Source of information (Primary, Secondary, On-demand) Consumers – Make queries against tables (Continuous, Latest, History)

22. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 22 gLite – Information and Monitoring Services Job Monitoring, Service Discovery, Network performance Monitoring Job Monitoring – Java logging service, log4j, Apache/Chainsaw (for other languages). Service Discovery – Locates suitable services to both users and services (Library!). Network Performance Monitoring – Many network monitoring frameworks. Aim: perform a standard interface to those frameworks.

23. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 23 gLite – Job Management Services

24. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 24 gLite – Job Management Services Accounting Accumulates information about the resource usage done by users or groups of users (VOs). Information on Grid Services/Resources needs sensors (Resource Metering, Metering Abstraction Layer, Usage Records). Records are collected by the Accounting System (Queries: Users, Groups, Resource) Grid services should register themselves with a pricing service when accounting for billing purposes.

25. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 25 gLite – Job Management Services Computing Element Service that represent the computing resource that is responsible of the job management: (submission, control, etc.) CEs refer to a set or a cluster of computational resources (WN) managed by LRMS, to dispatch jobs matching users requests. Two job submission models (accordingly to user requests and site policies) : PUSH(jobs pushed to CE queue), PULL(jobs coming from WMS when CE queue is empty) CE responsible to collect accounting information.

26. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 26 Computing Element (CE) Works in push or pull mode Site policy enforcement Exploit new Globus GK and Condor-C (close interaction with Globus and Condor team) CEA … Computing Element Acceptance JC … Job Controller MON … Monitoring LRMS … Local Resource Management System

27. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 27 gLite – Job Management Services Workload Management WMS set of middleware components responsible of distribution and management of jobs across Grid resources. Two core components of WMS: WM: accept and satisfy requests for job management. Matchmaking is the process of assigning the best available resource. L&B: keep track of job execution in term of events: (Submitted, Running, Done,...)

28. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 28 gLite – Job Management Services Job Provenance, Package Manager Job Provenance (JP) - Keeps track of submitted jobs for long periods (months, years). Package Manager – Helper service to automate: installing, configuring, updating and removing of software components. (RPM, dpkg/APT, Portage, …)

29. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 29 gLite – Data Services

30. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 30 gLite – Data Services Storage Element Needed Service are at least: Storage back-end (Drivers and Hardware) SRM Interface (Storage Specific) Transfer service (GridFTP) Native POSIX like file I/O API (gLite-I/O) Auxiliary Accounting and Logging services

31. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 31 gLite – Data Services Catalogs OS like file access metaphor. LFN (Logical file name) GUID (Grid unique identifier) SimLinks SURL (Site URL) TURL (Transfer URL) Catalogs: Authorization Base Metadata Base Metadata Schema Replica Catalog File Catalog File Authorization Metadata Combined Catalog Storage Index gLite - (FireMAN)

32. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 32 gLite – Data Services Data Movement Data Scheduler (DS) Keep track of user/service transfer requests File Transfer/Placement Sercice (FTS/FPS) Transfer Queue (Table) Transfer Agent (Network)

33. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 33 gLite – Helper Services Configuration and Instrumentation Service – Query service state. Agreement Service – Implements a communication protocol for the SLAs. Bandwidth Allocation & Reservation service (BAR) – Controlling, Balancing and Manage Network flows.

34. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 34 Components Site: –Computing Element (CE)  Gateway to local computing resources (cluster de worker nodes) –Worker Nodes (WN) –Storage Element (SE)  Gateway to local storage (disk, tape)  A gridftp server, and SRM Interface, IO server –User Interfaces (UI)  User’s access point to the grid  Client programs using some/all grid services.

35. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 35 Components Grid – or VO- wide –Security  Virtual Organization Server (VOMS)  MyProxy server (Proxy) –Information System –Job handling  Workload Management System (WMS)  Logging & Bookkeping (LB) –Data management  File catalog (FiReMan)  File Transfer Service (FTS)  File Placement Service (FPS)

36. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 36 VOMS Virtual Organization Membership Service –Multiple VOs –Multiple roles in VO  Compatible X509 extensions  Signed by VOMS server –Web admin interface –Supports MyProxy –Resources providers grant access to VOs or roles –Sites map VO members/roles to local auth mechanism (unix users accounts)  Allows for local policy

37. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 37 MyProxy –Allows longer lived jobs / increases security  WMS renews proxy  Users should not produce long lives proxies –Allows for secure user mobility  Users does not need to copy globus-keys around

38. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 38ReplicaCatalogue UI JDL Logging & Book-keeping ResourceBroker Job Submission Service StorageElement ComputeElement InformationService Job Status DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish grid-proxy-init Expanded JDL SE & CE info

39. E-infrastructure shared between Europe and Latin America Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA 39 Questions