1 SCI-BUS gateway technology for a large variety of user communities Peter Kacsuk MTA SZTAKI SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI

2 SCI-BUS EU FP7 project SCI-BUS (SCIence gateway Based User Support) provides gateway framework and science gateway building technology 3-year project: 1 Oct 2011 – 30 Sep 2014 Objectives of SCI-BUS –Support both WF developers and end-user scientists –Create a generic-purpose science gateway framework –Elaborate a science gateway instance development technology –Establish production SG instance services both for national grids (horizontal user communities) and various science communities (vertical user communities) –Develop business models to guarantee sustainability and commercial exploitation

SCI-BUS Architecture 3

4 Community tools and user roles in SCI-BUS Portlets Repository Portlets Repository Applications (e.g. SHIWA) repositories Applications (e.g. SHIWA) repositories fMRI MRI simulator Sequencing …. NonLinSyst simulator NonLinSyst simulator SCI-BUS Generic Framework based on WS- PGRADE and Liferay SCI-BUS Generic Framework based on WS- PGRADE and Liferay Application Developers Portlet Developers Gateway Developers SCI-BUS Generic Framework SCI-BUS Generic Framework Portlet Application DCI Domain specific Gateway Scientists Gateway Operators Data transfer Profile Monitoring Function Workflow Job submission …. Workflow Template A Data management Portlet Function

5 Project partners NoParticipant organisation namePart. short nameCountryExpertise & Experience 1Magyar Tudomanyos Akademia Szamitastechnikai es Automatizalasi Kutato Intezet MTA SZTAKIHungaryCoordinator in EDGeS, EDGI, SHIWA, developer of WS-PGRADE 2Academisch Medisch Centrum bij de Universiteit van Amsterdam AMCThe NetherlandsGateway developer for the medical user community 3Simsoft Bilgisayar Teknolojileri Ltd. Sti.SIMSOFTTurkeyGateway developer for business process modeling 4E-GROUP Ict Software Informatikai ZrtEGHungaryGateway developer for Web 2.0 5Eidgenössische Technische Hochschule ZürichETH ZurichSwitzerlandGateway developer for bioscience 6Middle East Technical UniversityMETUTurkeyGateway developer for seizmology 7Scaletools Ltd.STSwitzerlandGateway developer for clouds 8Eberhard Karls Universität TübingenEKUTGermanyGateway developer for computational chemistry 9University of WestminsterUoWUKDeveloper of UK NGS P-GRADE 10Universidad de ZaragozaUnizarSpainGateway developer for PireGrid SMEs 11CloudBroker GmBhCBSwitzerlandGateway developer for clouds 124D SOFT SZAMITASTECHNIKAI KFT4D SOFTHungaryGateway developer for ETICS-2 13Istituto Nazionale di AstrofisicaINAFItalyGateway developer for astrophysics 14Laurea-ammattikorkeakoulu oyLaureaFinnlandGateway developer for Blender 15The Provost Fellows & Scholars of the College of the Holy and Undivided Trinity of Queen Elizabeth near Dublin TCDIrelandGateway developer for helio- physics

WS-PGRADE/gUSE Generic-purpose gateway framework 6 Based on Liferay WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) gUSE (Grid User Support Environment) architecture General purpose Workflow-oriented portal framework Supports the development and execution of parameter sweep, Monte-Carlo applications Enables the multi-DCI execution of any WF Support the fast development of SG instances by a customization technology

Scalable architecture based on collaborating services

Flexibility of using various DCIs by WS-PGRADE/gUSE Seamless access to various types of DCIs: Clusters (PBS, LSF, MOAB, SGE) Cluster grids (ARC, gLite, GT2, GT4, GT5, UNICORE) Supercomputers (e.g. via UNICORE) Desktop grids (BOINC) Clouds (Via CloudBroker Platform) 8

Multi-cloud Integrated WS-PGRADE/CloudBroker Platform to access multi-clouds Cloud Broker Platform WS- PGRADE n IaaS Cloud 1 IaaS Cloud N SEQ WS- PGRADE 1 Supported clouds: Amazon, OpenStack, Eucalyptus, OpenNebula, IBM SaaS solution: Preregistered services/jobs can run from WS-PGRADE Supported from gUSE IaaS solution: any services/jobs (e.g. BoT jobs) can run from WS- PGRADE Supported from gUSE

WS-PGRADE UI to configure nodes for cloud services 10 Cloud selection During configuration WS-PGRADE queries the software list

Flexible usage scenarios/business models by WS-PGRADE/gUSE Workflow developer view and support (full gateway framework view) End-user view and support (limited portlets) Customized user interface to support the creation of domain specific gateways (ASM API) Provide workflow execution service on top of many different DCIs (Remote API) 11

Gateway types based on WS-PGRADE/gUSE 12 For developing workflow (WF) applications for a large set of various DCIs Developed WF Internal gUSE Repository External SHIWA Repository Existing Community Gateway End User mode WS- PGRADE/gUSE gateway generated by configuration Customized gUSE gateway generated by ASM API Generic WS-PGRADE/gUSE gateway WF execution gUSE gateway Remote API

The flexibility of using WS- PGRADE/gUSE Security: Flexible management of –Individual users’ certificate –Robot certificates Data bridge (coming feature) –to access data storages in different DCIs –To transfer files among different DCIs Support for workflow interoperability based on SHIWA technology –CGI approach for integrating many different type of workflows as meta-workflows –Using other workflows available in IWIR format in the SHIWA repository 13

CTA requirements met by the software The vast majority of the requirements are met Instead of listing those that are met rather we show those that are not currently met 14

CTA requirements not met by the software UR-SG-0020 The Science Gateway must provide access to the data collections (Archive) resulting from the Science data collection and to the MC simulation results through a data selection application. –This will be developed according to the needs of CTA community. Similar solution is under development, for example, for the agINFRA project UR-SG-0920 All data obtained by CTA observatory must be made public through an archive following a period of proprietary use. The proprietary period could be different for each observation data set. –The portal is already connected to the public SHIWA Workflow Repository. Connecting to a public archive cannot be a problem. 15

Support for workflow developers in collaboration with ER-flow based on SHIWA technology SHIWA WF App. Repository Sharing WF appls via the repository The SG framework provides access to a large set of clouds (and DCIs) to make these WF applications run The SG framework provides graphical WF editor, WF manager Cloud N Science Gateway (SG) framework Cloud 1 DCI (Grid, etc.) 16

Support for science gateway instance developers SHIWA App. Repository Science Gateway (SG) framework with access to various DCIs (including clouds) and access to SHIWA App. Repository (WS-PGRADE/gUSE) Cloud N Cloud 1 17 Customization Science Gateway (SG) Instance with inherited access to App. Repo, clouds and DCIs 17 different SG instances for biologists, chemists, etc.

Types of gateways to be developed from the core WS-PGRADE/gUSE framework 1.Generic purpose gateways for national grids –Core WS-PGRADE/gUSE (e.g. Greek, Italian NGIs) 2.Generic purpose gateway for a particular DCI –EDGI gateway based on WS-PGRADE/gUSE 3.Generic purpose gateway for specific technologies –SHIWA gateway for workflow sharing and interoperation 4.Domain specific science gateway instances –Swiss proteomics portal, MoSGrid gateway –Autodock gateway, Seizmology gateway, VizIVO, etc. 18

Communities developing WS- PGRADE/gUSE based gateways 11 partner communities 6 subcontractors 5 associated members EU projects –agINFRA, DRIHM, –VIALACTEA, –EDGI, IDGF-SP, –SHIWA, ER-Flow, –CloudSME 19

gUSE based gateways 20

gUSE Roadmap based on community effort 21

gUSE download statistics at sourceforge 22

23 Where to find further information? SCI-BUS web page: – gUSE/WS-PGRADE: – gUSE on sourceforge – – –

24

Conclusions Why to select WS-PGRADE/gUSE and join the SCI-BUS community? 1.Robustness –Already large number of gateways used in production 2.Sustainability –The sustainability and commercialization plan of the SCI-BUS project guarantees it, as well as the CloudSME project 3.Functionalities –Rich functionalities that are growing according to the SCI-BUS and sourceforge community needs 4.How easy to adapt for the needs of the new user community? –Already large number of gateways customized from gUSE/WS- PGRADE 5.You can influence the progress of WS-PGRADE/gUSE 25 Join SCI-BUS as associated member