SHIWA SIMULATION PLATFORM = SSP Gabor Terstyanszky, University of Westminster e-Science Workflows Workshop Budapest 09 nd February 2012 SHIWA is supported by the FP7 Capacities Programme under contract No. RI

SSP Objectives and Services Objectives: To create an environment which enables seamless execution of workflows of different workflow systems through workflow interoperability Services: supports the whole workflow lifecycle: editing, uploading, browsing downloading and executing workflows provides coarse- and fine-grained workflow interoperability solution offers Distributed Computing Infrastructure interoperability solution provides desktop computer and portal interfaces to manage workflows Key actors: researchersworkflow engine developers workflow developers 2

3 Workflow description plain text which describes the inputs and outputs and explains what the workflow does Workflow implementations It contains all implementation files or references to these files (via eg. URLs) and also holds other data/metadata necessary to run the workflow on its associated workflow engine. Workflow configurations It contains pre-defined input files and parameters or references to these files and parameters and other data/metadata of a workflow. Workflow engines It contains files and other data/metadata necessary to execute a workflow engine on a grid site or references to them SHIWA Data Structure

4 J2 WF4 J1 J3 GEMLCA client GEMLCA Service Workflow Engine B Workflow of Workflow Engine B Workflows of Workflow Engine A Coarse-Grained Interoperability DCI Workflow Engine A native workflows:J1, J2, J3 non-native workflows:WF4 - black boxes which are managed as legacy code applications

5 SHIWA Science GatewayResources native WEPGRADElocal resources: invocation of locally deployed WEs portalPGRADE v2.4.1WE submission to local cluster repositoryGEMLCA + SHIWA repo remote resources: through remotely pre-deployed submitterGEMLCA with GIBWEs to gLite and Globus DCIs proxy managementSHIWA Proxy Server SHIWA Portal GEMLCA admin SHIWA Science Gateway WE1WEp GEMLCA Repository WF1WFm SHIWA Repository WF1WFn WS-PGRADE Workflow engine WS-PGRADE Workflow editor gLite DCI Globus DCI pre-deployed- WEs MOTEUR WE Kepler WE Taverna WE Triana WE local cluster MOTEUR WE GWES WE ASKALON WE SHIWA VO ASKALON WE GEMLCA Service GEMLCA with GIB SHIWA Proxy Server Proxy Server CGI Infrastructure

6 SHIWA Portal: Workflow Editor

7 SHIWA Portal: Configuring Workflow

8 SHIWA Portal: Executing Workflow

9 SHIWA Workflow Repository Browse Workflows Browse Implementations Search Advanced Search Sort

10 SHIWA Workflow Repository

11 SHIWA RepositorySHIWA Portal WF1 SHIWA Science Gateway GEMLCA Service WFn WE1WEp GEMLCA Repository WE + WF WF1WFm GEMLCA with GIB WF list PGRADE Workflow engine PGRADE Workflow editor edit WF s2 search WF s1 s5 s4 gLite DCI MOTEUR WE GWES WE Globus DCI pre-deployed- WEs MOTEUR WE Kepler WE Taverna WE Triana WE local cluster ASKALON WE SHIWA VO ASKALON WE researcher invoke WE s6 CGI User Scenario: Native WE GEMLCA admin SHIWA Proxy Server Proxy Server s3 s6 submit WE

12 SHIWA RepositorySHIWA Portal WF1 SHIWA Science Gateway GEMLCA Service WFn WE1WEp GEMLCA Repository WF1WFm GEMLCA with GIB PGRADE Workflow Engine PGRADE Workflow Editor search WF s1 WE + WF s5 gLite DCI MOTEUR WE GWES WE Globus DCI pre-deployed- WEs MOTEUR WE Kepler WE Taverna WE Triana WE local cluster ASKALON WE SHIWA VO ASKALON WE researcher invoke WE s6 CGI User Scenario: Non-native WE GEMLCA admin GEMLCA Client MOTEUR Workflow Engine MOTEUR Workflow editor s2 s3 s4 GEMLCA UI

13 DCI Interoperability JSDL Translator Workflow Engine DCI Bridge Metabroker Proxy Server DCI workflow for DCI B J2 J1 J4 J3 workflow for DCI B J2 J1 J4 J3 workflow for DCI A

14 Fine-Grained Interoperability front-end plug-in: converts workflow of workflow engine A into IWIR representation back-end plug-in:converts from IWIR representation into workflow of workflow engine B IWIR workflow J2 J1 J4 J3 WFA ->IWIR frontend plug-in J2 J1 J4 Workflow of Workflow Engine A J3 WFA ->IWIR backend plug-in Workflow Engine B DCI J2 J1 J4 J3 Workflow of Workflow Engine B

SHIWA Repository SHIWA Portal WF1 GEMLCA admin SHIWA Science Gateway WFn WE1WFp GEMLCA Repository WF1WFm gLite DCI Globus DCI DCIs ARC DCI Unicore DCI GEMLCA submitter Kepler WE GWES WE Taverna WE Triana WE MOTEUR WE ASKALON WE WS-PGRADE Workflow engine WS-PGRADE Workflow editor PGRADE WE pre-deployed- WEs SSP DCI & FGI Infrastructure SHIWA Science GatewayResources native WEASKALON, MOTEUR, WS-PGRADElocal WEs: Kepler, MOTEUR, Taverna & Triana WEs portalWS-PGRADE v3.2.2 submit to the local cluster repositoryGEMLCA + SHIWA remote WEs: ASKALON, GWES & MOTEUR WEs GEMLCA ARC, gLite, Globus and Unicore DCIs proxy server DCI Bridge Meta Broker Proxy Server JSDL Translator

16 SHIWA Repository SHIWA Portal WF1 GEMLCA admin SHIWA Science Gateway WFn WE1WFp GEMLCA Repository WF1WFm gLite DCI Globus DCI DCIs ARC DCI Unicore DCI WS-PGRADE Workflow engine WS-PGRADE Workflow editor DCI Interoperability Scenario DCI Bridge Meta Broker Proxy Server JSDL Translator search WF s1 s6 s4 s5 s3 s2 edit workflow s7 submit workflow s6

17 SHIWA Repository SHIWA Portal WF1 GEMLCA admin SHIWA Science Gateway WFn WE1WFp GEMLCA Repository WF1WFm DCIs WS-PGRADE Workflow Engine WS-PGRADE Workflow Editor FGI Interoperability Scenario c c DCI Bridge Meta Broker Proxy Server JSDL Translator search WF s1 Workflow Engine Workflow Editor s3 FP1 FP2 FPn c c BP1 BP2 BPm s4 s2 edit WF submit WF s7 gLite DCI Globus DCI ARC DCI Unicore DCI s5 s6

18 SHIWA Simulation Platform: Access SHIWA Science Gateway (SHIWA Portal + SHIWA Repository) joint portal and repository account given by the science gateway administrator DCI resources certificate given by the relevant DCI administrator, for example the VO sysadmin Access to the SHIWA Portal shiwa-portal.cpc.wmin.ac.uk/liferay-portal-6.05 Access to the SHIWA Repository shiwa-repo.cpc.wmin.ac.uk

19 SHIWA Simulation Platform: Selling Points the simulation platform supports the whole lifecycle of workflows, i.e. creating, testing, uploading, browsing, downloading and running workflows users can use workflows of their own and of other workflow systems through the same user interface, i.e. they can re-cycle or share workflows created by other research teams or even other research communities workflow developers (or e-scientists) can create, run, upload workflows through the simulation platform researchers (astro physicists, earth scientists, bio scientists, etc.) can browse the workflow repository, select and run workflows through the simulation platform the simulation platform provides access to major European e- infrastructure, the European Grid Initiative (EGI) infrastructure which offers large pool of compute and storage resources