Cracow Grid Workshop, October 15-17, 2007 Polish Grid Polish NGI Contribution to EGI Resource Provisioning Function Automatized Direct Communication Tomasz Szepieniec ACK CYFRONET AGH, Kraków, Poland Polish NGI, PL-GRID

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid  5 key resources centers – each owns and fully manages its resources  NGI is built on assumption that resource planning:  is open and known for NGI partners,  is done by each partner in respect to its resources,  would be matter of NGI-operator coordination in case of problems.  'Operating Center' is planned to be small  In each center grid load is mixed with local load. Resources Prov. in Polish NGI = My Motivation

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Two Aspects of Resource Provisioning  Aspect 1: Motivation  the formal frame that makes resources centers interested to provide resources  references: business model, commitments, SLAs, resource owning model  Aspect 2: Organization  includes procedures and communication tools  crucial for robustness of the resources provisioning process  YES, it is possible to discuss this without fixing first aspect – it should result with more flexible solution

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid List of Requirements (collected from discussions, but still tentative)‏ 1.A user (client) who is able to pay for resources should collect resources or receive answer that resources are not available in a relatively short time (few days)‏ 2.All kinds of resources must be allocated adequately (processors, storage, services) with a care to include all steps required to prepare resources for using by the client (e.g. set-up planning, validation process). 3.Both open and specified in time computational projects are possible. Additionally, a VO or a RC may want to change configuration of resources. 4.Users must be free to arrange the resources at their own, so any agreement done directly should be visible in the system. 5.Support for (some) new VOs should be provided in a way to enable both quick start-up and growth without necessity of starting from scratch.

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Additional Requirements 6.The procedures should be efficient and scalable 7.A management latency should be minimized 8.The process of allocating resources should be traceable for users, resource providers and for other actors involved in the process. 9.Procedures should be flexible to cover diversity on: requirements for formal agreements (SLA contracts, MoU, nothing), resource owning models, etc. 10.Cost of operations on higher levels (especially EGI but also NGI) should be minimized (it means more work should be put to RCs).

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid What we have now (EGEE II)‏ From EGEE Procedures: Once a VO registration has been approved by the OAG, the ROC managers as representatives of the RCs in their federation, will find a RC willing to host the VO's VOMS server and authorise the VO on a very restricted set of resources. [..]. The OAG will work with you in parallel to arrange more permanent resources [..] To obtain more resources, the VO manager will have to interact with the OAG and the ROCs. This is a centralized and heavy control schema. It does not meet most of the requirements.

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Direct Communication  General idea:  To decrease management latency, we should simplify communications patterns (EGI to NGIs, NGIs to RCs and response collections)‏  So, let's make VOs right to communicate directly their needs to selected/all RCs and let's make RCs right to answer directly.

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Automatization in Direct Communication  Problem1: NGIs and EGI lost control  Solution1: Provide a tool (web portal) for communication between VOs and RCs, in which communication would be visible (optionally be waiting for approval) to NGIs and EGI.  Problem2: It does not scale - large VOs would receive hundred messages  Solution2: The tool could group and summarize them automatically. There are methods to cover complexity.

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Additional Profits of Automatisation  We can trace all stages of 'contract' life in the tool  VOs could accept selectively resource that they want to use, not blocking the rest  Both VOs and RCs would have possibility to do resource planning in the tool  Different formal requirements (eg. formal MoU) could be covered in ACCEPTED state  Solution could be adapted to different roles of EGI Operator and NGI operators.  Special treatment for different types of resources could be implemented  Some centrally-managed pool of resources (as for 'new VO' in EGEEIII) could work as well.

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Automatized Direct Communication – Implementation  Problem3: Implementation of such tool would be time- and money- consuming  Solution3: We made a first prototype of such tool, named 'Bazaar'  feature list includes: project and contract management, negotiation process tracking, notification for VO and RC managers,  the aim was to simplify resources allocation in CE ROC,  tool is integrated with EGEE CIC Portal and GOC DB,  currently Bazaar v1.0 is tested before enabling in CIC Portal (for CE ROC use)‏

EGI DS WP3 Meeting, CERN, Jan 29-31, Polish Grid Summary  Organizational aspect of resource provisioning is important. It is better to consider it now.  Automatized Direct Communication would improve robustness of resource planning  Well suited collaboration tool would  make the process efficient and scalable  cover differences and changes in business models, NGIs roles, etc.