w w w. h p c - e u r o p a. o r g The HPC-Europa project and GridSphere Dawid Szejnfeld Poznan Supercomputing and Networking Center (PSNC) Poland GridSphere and Portlets workshop, Edinburgh, 4 March 2005

w w w. h p c - e u r o p a. o r g Outline HPC-Europa JRA2: Single Point of Access Multi-Grid Interface Job Submission Interface Implementation issues Generic Job Submission Portlet Status & Next Steps

w w w. h p c - e u r o p a. o r g HPC-Europa HPC-Europa - Pan-European Research infrastructure on High Performance Computing for the Science of 21st century Goal: to provide advanced computational services in an integrated way to the European research community Budget: ~20 mln EURO 14 Partners across Europe Project activities –Transnational Access Programme –Networking Activities –Joint Research Activities (JRA1, JRA2)

w w w. h p c - e u r o p a. o r g JRA2: Single Point of Access Motivation –To provide a uniform access to resources of all centers, transparently and regardless of user physical location Main objectives –ease of use, –improving availability, –global optimization of resources utilization To achieve these goals –JRA2 builds a HPC-Europa portal providing uniform, flexible and intuitive user access to Grid resources from anywhere –Develops and/or adapts tools and services needed to establish full operational Grid environment –Will develop a meta-broker responsible for users’ access to all consortium’s resources

w w w. h p c - e u r o p a. o r g HPC-Europa SPA Grid Portal / Grid Resource Broker HPC center Grid Middleware A Grid Middleware B Grid Middleware C Grid Middleware D VO

w w w. h p c - e u r o p a. o r g Middleware in HPC Centers OpenPBS LSF SGE Pre-WS Globus 3.2GRMS (GridLab Resource Management System) PSNC Condor GRIA (Grid Resources for Industrial Applications) NTUA LoadLeveler SGE Globus 3.2JOSH (JOb Scheduling Hierarchically) EPCC LoadLeveler OpenPBS NQS Globus 3.2eNANOS (Grid Resource Broker) CEPBA OpenPBS ERS-NQS PBS Pro UNICORE - HLRS LoadLeveler OpenPBS with MAUI UNICORE - CINECA LRMSUnderlying Grid technology High-level middlewareCenter

w w w. h p c - e u r o p a. o r g GridSphere We have decided to use the GridSphere as a portal framework for the SPA on the basis of Grid portals' evaluation The main reasons of our choice are as follows: –portlet-based & JSR 168 compliant –experience in Grids useful tools, services, and core & Grid portlets pluggable access to Grid services using the Portlet Services concept abstract & generic Grid Portlets model –good support & documentation –relatively straightforward to use (customization mechanism and a flexible easy to use interface for a wide range of end users) –open source

w w w. h p c - e u r o p a. o r g Multi-Grid Interface (MGI) Extension of the GridSphere’s Grid Portlets concept by –Dynamic loading of multiple plugins for a single interface –Interface based on the GGF JSDL and analysis of functionality of several Grid systems Definition of the common interface –For each functionality, e.g. job submission, resource information etc. –Based on standards where possible (e.g. JSDL) –Taking into account both gathered requirements and the functionality of Grid middleware deployed in HPC centers Capability check –description of implemented capabilities in the form of the XML document –used to disable not available options (in portal) or to select sites which can accept given job description (meta-broker)

w w w. h p c - e u r o p a. o r g Portal layers Portlets Portlet Components Portlet Services Service Plugin Service Plugin Service Plugin Presentation Layers Business Logic Portlet Resources

w w w. h p c - e u r o p a. o r g MGI Architecture - Job Submission View Components of a common job submission interface –JSS - Job Submission Service –JSSI - Job Submission Service Interface –JSP - Job Submission Plugin –JSPI - Job Submission Plugin Interface Other common interfaces Auxiliary components

w w w. h p c - e u r o p a. o r g Auxiliary Components Information System –Central information indexing service –Provides a list of available HPC centers, URLs of Grid middleware, etc. Application Repository –Logical names, meta-data, binaries compiled for various systems, required environment (preinstalled libraries, environment variables, etc.), possible configurations & parameters Data management system –Management of logical file names –Goal of the HPC-Europa NA3 Plugin Factory –Initializes and loads needed plugins

w w w. h p c - e u r o p a. o r g Job Submission Interface (JSI) Job submission operations –submitJob (UNICORE, GRMS, JOSH, eNANOS, GRIA) –cancelJob (UNICORE, GRMS, JOSH, eNANOS, GRIA) –getJobInfo (UNICORE, GRMS, JOSH, eNANOS, GRIA) –findResources (UNICORE, GRMS, eNANOS) –submitJobToBestResource (GRMS, JOSH, eNANOS, GRIA) –findBestResource (JOSH) Data elements descriptions –Job Description XML schema –Job Information XML schema Auxiliary operations –getMiddlewareName –getMiddlewareDesc –getMiddlewareMethods –getMiddlewareSchema Job states –GRID_QUEUED, BROKER_QUEUED, LOCAL_QUEUED, PREPROCESSING, RUNNING, POSTPROCESSING, SUSPENDED, FINISHED, FAILED, CANCELLED

w w w. h p c - e u r o p a. o r g Job Description Based on the GGF Job Submission Description Language (JSDL) specification Subset of JSDL defines the common interface The specification is still evolving Several extensions have been added

w w w. h p c - e u r o p a. o r g Capabilities Provided by Plugins Some plugins may implement only a part of the common interface Two descriptions must be returned by plugins –description of available methods –description of data structures (i.e. job description) The XML document based on the Job Description schema Two main kinds of information included –elements of the document define which parts of interface are implemented by service plugins –for each element a list of feasible values is defined The Job Submission Portlet uses this information to disable appropriate fields and limit lists of feasible values

w w w. h p c - e u r o p a. o r g Capabilities Provided by Plugins - example..... LINUX, FreeBSD, IRIX, Solaris, HPUX, Windows_XP..... The NetworkBandwidth element is not handled by a plugin (is not translated to the system-specific job description language) The list of feasible operating systems (values of the OperatingSystemType element) is limited to a few most common systems

w w w. h p c - e u r o p a. o r g Implementation issues – JSR 168 Intention to create portable web application Lack of services in the JSR 168 specification –Gridsphere JSR services implementation is not compatible with spec –Solution - external mechanism of creating services Spring – application framework –Easy to use (spring.jar added to application lib directory) –portlet-services web app - local and shared services (LocalServiceManager and SharedServiceManager) –Services as java beans, can be initialized with simple and complex values Configuration files –/WEB-INF/portlet-services.xml and shared-portlet-services.xml

w w w. h p c - e u r o p a. o r g Implementation issues – services

w w w. h p c - e u r o p a. o r g Generic Job Submission Portlet

w w w. h p c - e u r o p a. o r g Generic Job Submission Portlet (GRMS)

w w w. h p c - e u r o p a. o r g Generic Job Submission Portlet - Resource Requirements

w w w. h p c - e u r o p a. o r g Generic Job Submission Portlet - Resource Requirements(GRMS)

w w w. h p c - e u r o p a. o r g Status & Next Steps Status –Common job submission interface designed –Job submission portlet and plugin management started Next steps –Development and testing of the job submission portlet –Development and testing of the interoperability of authentication models –Design interfaces for resource information, accounting, data management and additional tools (e.g. application repository) –Demonstration planned in June