Development of a billing framework for D-Grid Prof. Dr. Gabriele von Voigt, Dr. Claus-Peter Rückemann, Wolfgang Müller Cracow Grid Workshop 2006 (CGW2006) 15 th -18 th October 2006

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 2 Overview 1.The D-Grid Initiative 2.Motivation 3.Scientific background 4.Existing billing concepts and solutions 5.Resulting concept of the D-Grid billing framework 6.Summary and future developments

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 3 The D-Grid Initiative - Overview User and application interfaces Available D­Grid resources Grid service functions Basic services Distributed data archives Distributed data archives Globus accounting and billing LCG/gLite Community Monitoring Data/ Software Distributed compute resources Distributed compute resources Security Data interfaces GAT API Data management scheduling and workflow mgmt. GridSphere Network infrastructure UNICORE D­Grid services AstroGrid-D C3-Grid HEP-Grid InGrid MediGRID TextGrid DGI – Integration Project of D-Grid

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 4 The D-Grid Initiative - German Core Grid sites SiteResourceAmount FZJ/ZAMIBM Supercomputer with 8,5 TFlops STK data robot system with 2,8 PByte 32 CPUs 300 TByte FZK/IWR8 nodes Opteron 2x2.2 GHz 8 processors of a system NEC SX-5 1 p630 with 4 processors 1 SX-6i to do tests 2 nodes Opteron 2x2.2 GHz to do tests 100% 50% LRZSGI high performance system with 20 TFlop/s Intel IA32 and IA 64 Cluster, IBM p690, SunFire 805% 5% MPI/RZGIBM supercomputer with 4,5 TFlops, PC cluster with 2 TFlops Data robot system with 8 PByte 32 CPUs 400 TByte PC²Cluster of 400 Xeon 64 Bit processors, high performance visualization and FPGAs 10% RWTH/RZ2 SunFire 6900 with 24 UltraSPARC IV each100% TU-Dresden/ZIHSGI O2K(56 proc)/O3K(192 proc.) : T3E (64 proc): PC cluster with 30 processors, end off 2005: new system with 1000 proc. 10% 20% 2% Uni-H/RRZNPC-Cluster mit 64 CPUsassoc. Uni-KAPC-Poolassoc. FHG/ITWMassoc. Hannover Bremerhaven Oldenburg Göttingen Potsdam Oberpfaffenhofen Freiburg Heidelberg Darmstadt Marburg Dortmund Wuppertal Köln Siegen Sankt Augustin München Hamburg Berlin Dresden Stuttgart Karlsruhe Jülich

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 5 Motivation Even scientific environments aim at sustainability and refinancing resource provision (especially after the funding period). For the owners of grid resources it is essential to: generate incentives for the constant provision of grid services to meet the needs of grid resource consumers, enabling them to quest offers of resources of other providers when their own capacities are insufficient, allowing them to maximize their resource utilization by offering a competitive service access price in order to attract consumers, and as a consequence, reducing their total costs of ownership (TCO). The users (resource consumers) have the option of choosing the providers that best meet their requirements or between rental or self-procurement of computing resources.

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 6 Scientific background Elements of the billing procedure 1. step: receiving the usage data provided by the accounting 2. step: pricing of the single entity of the accounting data based on the contract between resource provider and resource user (rating, tariff management, account management, contract management) 3. step: determination of the overall price for the resource usage and presenting the invoice to the resource user (bill presentment)

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 7 Existing billing concepts and solutions „Data Grid Accounting System“ (DGAS) decentralised banking structure checking the creditability of the user by his bank account before job submission the resource provider can apply various pricing metrics (such as wall clock time, CPU time, memory, storage, network traffic) each resource provider can choose the preferable metric according to his pricing policy  uses only grid credits as a virtual currency, provides no interface to common automated payment mechanisms (such as Paypal or Netcash)  uses only gLite as a middleware

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 8 Existing billing concepts and solutions „SweGrid Accounting System“ (SGAS) decentralised banking structure checking the creditability of the user by his bank account before job submission creating holds on the bank account while job processing in order to avoid overspending of grid credits implemented within SweGrid and running within a production environment  uses only grid credits as a virtual currency, provides no interface to common automated payment mechanisms  uses only Globus toolkit as a middleware  only one nearly homogenous resource type is used within the SweGrid  only one metric (wall clock time per node) is used for pricing

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 9 Existing billing concepts and solutions „GridBank/GASA” framework in the context of the Australian GRIDBUS project checking the creditability of the user by his bank account before job submission creating holds on the bank account while job processing in order to avoid overspending of grid credits the resource provider can apply various pricing metrics (such as wall clock time, CPU time, memory, storage, network traffic) uses virtual and real currencies open interface to existing payment handling systems enables different payment strategies (such as pay before use, pay as you go, post-payment)  centralised grid bank server  Globus toolkit as a middleware  developed within the scope of the project but no production phase yet

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 10 Resulting concept of the D-Grid billing framework Inquiry among the D-Grid resource provider and resource user (D-Grid communities and the German Core Grid Sites) The major features are: payment after job submission but guaranteeing sufficient user’s funds before job submission necessary security from job submission up to payment between user, resource broker, resource provider and clearing institution various mechanisms of price building for resource providers ability to handle various currencies existence of clearing institutions (bank service, account management) installation of a procedure for the settlement of disputes and conflicts independent of the utilized middleware special emphasis on virtual currency for spending funded budgets

Regionales Rechenzentrum für Niedersachsen Gabriele von Voigt,Claus-Peter Rückemann,Wolfgang Müller | CGW | Folie 11 Summary and future developments D-Grid needs an extended interpretation of billing up to the full payment process and has to take into account the German federal funding policy. A comprehensive billing solution for the mutual provision and usage of grid resources will be developed and implemented. A transparent and comprehensible billing framework for grid computing is going to be established. The aim is to supply a multiplicity of users, both from the scientific range and from the economy. Services will be offered by a multiplicity of providers. The project is funded by Thank you for your attention.