Legacy code support for commercial production Grids G.Terstyanszky, T. Kiss, T. Delaitre, S. Winter School of Informatics, University of Westminster P. Kacsuk, G Kecskemeti MTA SZTAKI Laboratory of Parallel and Distributed Systems, Hungary Extending GEMLCA to fulfil requirements of business grids

Presentation Outline 1.GEMLCA (Grid Execution Management for Legacy Code Applications) concept 2.GEMLCA requirements towards commercial production Grids 3.GEMLCA Extensions Dynamic User Management Resource & Service Monitoring Accounting and Charging 4.Conclusion and future work

GEMLCA Concept Compute Servers Resource manager deploys: LC GEMLCA GEMLCA resource Client 1: to register legacy code as a Grid service Client 2: to invoke legacy code Grid service Send custom input Invoke legacy code Produce result Return result

The GEMLCA-view of a legacy code Any code that correspond to the following model can be exposed as Grid service by GEMLCA: Legacy code Input data (files, command line params, env. vars) Output data (files) Call shared libraries Query databases Communicate over the network Perform computation...

What s behind the GEMLCA resource … Client GEMLCA Service Compute Servers Local jobmanager e.g. Condor/Fork GEMLCA Process Legacy Code Job Job submission and file transfer services Grid middleware layer Grid Service Client Consequences: 1.Not only the legacy code and the GEMLCA service, but also a local job manager and a Grid middleware layer must be installed on the hosting system! 2.The aim of the code registration process is to tell GEMLCA how to submit the legacy code to the grid middleware layer

3 rd generation Grids: (OGSA: GT4, gLite) Legacy applications Grid Site 1 Grid Site 2... The GEMLCA P-GRADE Portal GEMLCA P-GRADE Portal Server Desktop 1 Desktop N Web browser LC 1 LC 2 Result Input

GEMLCA on the UK NGS The NGS P-GRADE GEMLCA Portal Runs as a service available for all NGS users Portal Website:

Requirements – –dynamic user management users accounts created automatically grid-mapfile modified automatically – –monitoring of GEMLCA & Grid resources information about previous and current state of Grid resources & services - accounting/charging of resource/service usage resource and service usage should be metered and charged accordingly GEMLCA and business Grids

Dynamic User Management need for dynamic user management dynamic and scalable management of several hundred or thousand users without manual interactions dynamic user management solutions identity mapping (i.e. mapping Grid certificates) to local accounts providing local accounts on demand user management implementation Local Credential Mapping Service (LCMAPS) to map credentials to local account Workspace Management Service (WMS) to allocate workspace (or local account) to users

Dynamic User Management Service Workspace Management Service P-GRADE Portal Server Desktop 1 Web browser Desktop n Web browser abstract workflow Legacy applications + default input Grid Site account pool Identity Mapping (LCMAPS) returning a workspace

Grid Monitoring Service Need for Monitoring checking availability of resources/services generating historical availability data classifying availability of resources/services & making this data available for brokers Monitoring scenarios MDS initiated regular monitoring user initiated monitoring Monitoring Implementation probes are implemented by scripts and their outputs are displayed in a monitoring portlet

Grid resource probes: network connectivity of a remote host access to a remote MyProxy server file transfer using a remote host GridFTP server Globus job submission (WS-GRAM) availability of the local information system (MDS service) local job manager (Condor, PBS, SGE etc.) GEMLCA services probes: GLCAdmin, GLCList GLCProcess Grid Monitoring Service

Grid Monitoring Service Architecture Grid site with legacy code resources MDS4 GMT (GEMLCA Monitoring Toolkit) Production Grid (e.g. EGEE, NGS) Run GMT probes and collect data 3 rd party service providers User Portal Map execution

On-demand execution as a prototype Grid Monitoring Service in the P-GRADE portal Automatic execution Already works as a service on the NGS P-GRADE portal

GMT classifier Grid site with legacy code resources MDS4 GMT (GEMLCA Monitoring Toolkit) Grid broker (e.g. LCG broker) Production Grid (e.g. EGEE, NGS) Run GMT probes and collect data Historical test data Store historical data Prediction based on current and historical data Resource availability 3 rd party service providers User Portal Current work - Resource testing and availability prediction Submit workflow Map execution

Grid Accounting & Charging Service Need for accounting/charging automatic resource/service usage accounting managing multiple charging policies & implementations creating a database to store both accounting & charging data Accounting/charging architecture create a Grid Bank to manage all transactions related to accounting & charging Grid Accounting & Charging service (GAC) based on the GGFs RUR specification

resource datauser data host name / IP address Host name/IP address certificate name host type local job ID wall clock time + price user CPU time + pricejob data system CPU time + pricejob ID main memory + priceapplication name secondary storage+ pricejob start date I/O channels + price job end date Resource Usage Record (RUR) user certificate name resource certificate name job ID resource business model resource usage date resource usage time amount to be charged Resource Charge Record (RCR) Grid Accounting & Charging Service

P-GRADE Portal Server Desktop 1 Web browser Desktop n Web browser Broker Accounting Service Abstract workflow Legacy applications + default input Grid Site 1 Legacy applications + default input Grid Site m collecting usage data Charging Service RUR RCR database Grid Accounting & Charging Service

Conclusions and Future Work dynamic user management installed on the GT4-based WestFocus Grid will be used in the NGS GT4 testbed resource/service monitoring automatic execution installed on the NGS P-GRADE portal On-demand execution under development & testing accounting/charging - under development & testing

Websites: Thank you for your attention … Any questions? GEMLCA/P-GRADE portal demonstrations on the GOSC booth during the AHM!