1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org Grid Services for Resource Reservation and Allocation Tiziana Ferrari Istituto Nazionale di Fisica Nucleare, INFN CNAF Terena Networking Conference, Poznan, June 6 2005

2. Enabling Grids for E-sciencE INFSO-RI-508833 Outline Use cases The Grid Resource Allocation and Agreement Protocol The EGEE gLite Agreement Service: –Architecture –Components Workload Management System instrumentation Implementation: the space reservation Agreement Service prototype Future work

3. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Computing (2) “A computational Grid is a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities” Ian Foster, Carl Kesselman Purpose of Grid Computing: flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”

4. Enabling Grids for E-sciencE INFSO-RI-508833 Use Cases NETWORK: Data replication:  to optimize performance of a data transfer session (that otherwise would compete with other streams and would be subject to variable throughput)  to support file transfer with deadline (to synchronize job execution with input file transfer) COMPUTING:  to reserve computing resources (e.g. worker nodes, CPU cycles) in presence of a large number of other competing jobs STORAGE:  to guarantee that a sufficient amount of space is present in a Storage Element to save the output data  the Storage Element is close to the Computing Element on which computing resources have been reserved

5. Enabling Grids for E-sciencE INFSO-RI-508833 Use cases The Grid Resource Allocation and Agreement Protocol The gLite Agreement Service: –Architecture –Components Workload Management System instrumentation Implementation: the space reservation Agreement Service prototype Future work

6. Enabling Grids for E-sciencE INFSO-RI-508833 WS-Agreement defines a language and a protocol for –Advertising the capabilities of providers –Checking compliance to pre-defined templates –Creating agreements based on creational offers Agreement Layer: –Provides a web service based interface –represents and monitors agreements with respect to provisioning of services implemented in the service layer Service Layer (out of the GRAAP scope): –is an resource-specific layer of a provided service –The interface to this layer is domain-specific: may or may not be exposed as a web service interface Grid Resource Allocation and Agreement Protocol – GGF GRAAP 2/2

7. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Resource Allocation and Agreement Protocol – GGF GRAAP 1/2 WS-Agreement specification Conceptual Layered Service Model [2] gLite Agreement Service allocation and reservation service provider

8. Enabling Grids for E-sciencE INFSO-RI-508833 Agreement: XML structure Agreement Terms Service Description Terms Guarantee Terms Context Name Name: identificator Context: participants’ names, lifetime, links to other agreements related to this (co-allocation) Terms: Service Description Terms: - provide information needed to instantiate and identify a service to which this agreement pertains - describes the functionality that will be delivered under an agreement Guaranteed Terms: specify the service level that the parties are agreeing to Terms are usage-domain independent.

9. Enabling Grids for E-sciencE INFSO-RI-508833 Use cases The Grid Resource Allocation and Agreement Protocol The gLite Agreement Service: –Architecture –Components Workload Management System instrumentation Implementation: the space reservation Agreement Service prototype Future work

10. Enabling Grids for E-sciencE INFSO-RI-508833 Agreement InitiatorAgreement Initiators Agreement Offer Storage/Computing/ Network Agreement Service Reservation and Allocation Service Providers The gLite Agreement Service 2/2

11. Enabling Grids for E-sciencE INFSO-RI-508833 The gLite Agreement Service 1/2 Agreement offer –the “description” of the requested reservation Agreement Initiator –WMS Reservation Manager (reservation manager) –Data Scheduler –User Agreement Service –Checks offer compliance to one of theinternal templates –Can interact with several types of resource reservation service providers (network – JRA4 bandwidth advance reservation, storage – StoRM, computing) Resource reservation Service Provider  Enforces the reservation on a specific resource instance --> Full architecture described in [3]

12. Enabling Grids for E-sciencE INFSO-RI-508833 Agreement Service deployment Reservation Manager (simple and co-allocation) Agreement Offer_1 Logging & Bookkeeping Agreement Service_1 Monitor_1 Reservation usage Sensor_1 Service Provider_1 Workload Manager System Agreement Offer Agreement Service_n-1Agreement Service_n Agreement Offer_n-1 Agreement Offer_n Resource Monitor_1 Reservation usage Sensor_1 Service Provider_1 Resource

13. Enabling Grids for E-sciencE INFSO-RI-508833 One Service Provider instance per “resource” instance (e.g. CE and SE). manages resource availability information over time through a local “slot table” enforces resource reservation through resource-specific interfaces/for example: –Network:  the set of agents responsible of configuring switches/routers on a given path –Storage:  srmReserveSpace / srmReleaseSpace Monitoring of resource usage Component 1. Service Provider

14. Enabling Grids for E-sciencE INFSO-RI-508833 Example: the JRA4 network Service Provider “ The Network Element should be seen as a set of services rather than a single component in the architecture. It provides user-oriented network services to the Higher Layer Middleware by abstracting the physical network services. Specifically it will allow reservation of network bandwidth and the definition of the desidered network QoS. This may involve mapping the the higher level service requests into network specific service requests.” [1] Network Agreement Service Network Service Provider

15. Enabling Grids for E-sciencE INFSO-RI-508833 Component 2. Agreement Service Functionality: –Interacts with one or more service providers –Translates high-level service description terms (from initiator) to low level service-specific terms –Advertises the Service Provider capabilities through agreement templates. The template is an XML document that describes the contract skeleton. –Handles the agreement negotiation (agreement offer attributes from the initiator are tuned during the negotiation phase) –Provides information about: s tatus of the agreement negotiation process and attributes of a specific agreement instance Benefits: 1.It hides the complexity of the service providers’ interfaces from the agreement initiator (the client). 2.It exposes a single operation which can be used independently of the nature of the agreement type (createAgreement). 3.Scalability: new Service Providers can be easily integrated by advertising the corresponding templates (clients can be notified when new templates are added). 4.No need of a-priori knowledge about services available to users.

16. Enabling Grids for E-sciencE INFSO-RI-508833 Component 3. Agreement Initiator: the WMS Reservation Manager Reservation Manager: –a proposed component of the Workload Management System (WMS) [4] which handles:  Simple agreements (to start with)  Compound agreements (next in development roadmap) Why integrated in the WMS? –Agreement offer submission: similar to job submission –Agreement status: information from Logging and Bookkeeping –Agreement Service discovery: a list of Agreement Services is tried (more powerful than submission to a single Agreement Service)  based on matchmaking library  takes advantage of the WMS internal cache –Periodic re-submission in case of failure

17. Enabling Grids for E-sciencE INFSO-RI-508833 Use cases The Grid Resource Allocation and Agreement Protocol The gLite Agreement Service: –Architecture –Components Workload Management System instrumentation Implementation: the space reservation Agreement Service prototype Future work

18. Enabling Grids for E-sciencE INFSO-RI-508833 WMS instrumentation User Interface Agreement Offer Task queue Matc h Make r Reservation Manager Cache (ISM) Cache update r Logging and Bookkeeping Reserv. Helper Agr.offer + Resource list + Agr. Service EPR list Web Service interface Reserv. Cleaner

19. Enabling Grids for E-sciencE INFSO-RI-508833 WMS Agreement Service Discovery Problem: given an agreement offer including 1.Resorce requirements 2.Resource preferences (optional) 3.Service functionality find Agreement Services that support service functionality (3) and can (indirectly) do reservation on resource instances satisfying (1) and (2) Matchmaking finds a list of resources: –on which user is authorised to do reservation and –satisfying resource requirements (1), preferences (2), and supporting (3) find Agreement Services: –authorized to talk to the resources in the list identified above –Supporting the corresponding allocation and reservation service provider interfaces

20. Enabling Grids for E-sciencE INFSO-RI-508833 Use cases The Grid Resource Allocation and Agreement Protocol The gLite Agreement Service: –Architecture –Components Workload Management System instrumentation Implementation: the storage space reservation Agreement Service prototype Future work

21. Enabling Grids for E-sciencE INFSO-RI-508833 Implementation 2/2 Agreement service: –based on WS-Agreement XML Schema Definitions (GGF, GRAAP WG) –... but operations exposed are different (no Web Services Resource Framework - WSRF) –gSOAP 2.7.0 –agreement service tested with StoRM (SRM v 2.1)

22. Enabling Grids for E-sciencE INFSO-RI-508833 Use cases The Grid Resource Allocation and Agreement Protocol The gLite Agreement Service: –Architecture –Components Extension of the Workload Management System Implementation: the storage space reservation Agreement Service prototype Future work

23. Enabling Grids for E-sciencE INFSO-RI-508833 Future work Extension of the prototype's functionality set –Improvement of error handling –Authorization –Study of deployment model –Implementation of new operations –Full integration with WMS –Agreement monitoring through LB Performance testing Support of more Service Providers: –Network –computing

24. Enabling Grids for E-sciencE INFSO-RI-508833 References [1] Specification of interfaces for bandwidth reservation service, EGEE deliverable DJRA4.1, May 2005 (http://edms.cern.ch/document/501154)http://edms.cern.ch/document/501154 [2] The Grid Resource Allocation and Agreement Protocol Working Group; Global Grid Forum (https://forge.gridforum.org/projects/graap-wg). [3] gLite Allocation and Reservation Architecture, EGEE JRA1 technical report (http://edms.cern.ch/document/508055)http://edms.cern.ch/document/508055 [4] EGEE Middleware Architecture; EGEE Deliverable JRA1.1(http://edms.cern.ch/document/476451)