1. Introduction to SHIWA project EGI User Forum, 21.04. 2011. Vilnius Peter Kacsuk MTA SZTAKI kacsuk@sztaki.hu

2. 2 Project name: SHaring Interoperable Workflows for Large- Scale Scientific Simulations on Available DCIs Project type: I3 Start date: 01/07/2010 Duration: 24 months Total budget: 2,101,980 € Funding from the EC: 1,800,000 € Total funded effort in person-months: 231 Web site: www.shiwa-workflow.eu Contact person: Prof. Peter Kacsuk, email: Kacsuk@sztaki.hu The Project

3. 3 Project Partners MTA SZTAKI, Hungary UIBK, Austria C-UB, Germany CNRS, France UOW, United Kingdom CU, United Kingdom AMC, Netherlands USC, USA

4. 4 Background Life Sciences, Astronomy, Computational Chemistry, etc.  Researchers use complex data and tasks Compute-intensive modelling Simulation and analysis Scientific workflows  key paradigm for managing complex tasks  unifying mechanism for handling scientific data Distributed computing infrastructures

5. Many different WF systems After selecting a WF system the user community is locked there User communities are isolated by WF technologies WF systems typically support only one type of DCIs a WF developed for a certain DCI is difficult to be ported to other DCIs Motivations

6. Background 6

7. SHIWA objectives To eliminate the isolation of user communities by WF technology –Enable a user community to publish WFs SHIWA WF Repository –Enable user communities to use/run the published WFs SHIWA Portal –Enable user communities to integrate the published WFs into their own workflows Coarse-grain WF interoperability

8. SHIWA objectives To eliminate the locking of user communities into a WF technology –Enable to transform a WF written in WF-A to WF-B IWIR (Intermediate WF Representation) Coarse-grain WF interoperability To eliminate the locking of user communities into a DCI type –Enables a single WF system to use many different DCIs Generic DCI Back-end

9. SHIWA Workflow Interoperability – EGI UF11 9 Method Leverage existing workflow-based solutions –Workflow systems used as interfaces to DCIs Coarse-Grained (CGI) and Fine-Grained Interoperability (FGI) approaches Workflow system Foreach d in D { A(d); } Foreach d in D { A(d); } Workflow language Front-end Core engine Back-end DCI Distributed data Application services Computing resources invocation data transfers

10. Coarse-grained interoperability (CGI) approach Combine existing workflow applications written in various workflow languages treats existing workflows as black box systems can be incorporated into other workflow applications as workflow nodes

11. CGI Master workflow system Workflow Language A Front-end Core engine Back-end DCI 1 Distributed data Application services Computing resources Invocations Data transfers Embedded workflow system Workflow Language B Front-end Core engine Back-end DCI 2 Distributed data Application services Computing resources Foreach d in D { A(d); } Foreach d in D { A(d); }

12. Example of using CGI Taverna workflow Kepler workflow Triana workflow P-GRADE workflow

13. Target workflow system Foreach d in D { A(d); } Foreach d in D { A(d); } Front-end Core engine Back-end Target DCI Distributed data Application services Computing resources invocations data transfers Source workflow system 1 Workflow language Front-endExport Core engine Source workflow system 2 Front-endExport Core engine IWIR representati on Export IWIR Exportation and integration through IWIR Source DCI Distributed data Application services Comp. res. Services adaptation FGI

14. SHIWA Workflow Interoperability – EGI UF11 14 Target SHIWA Simulation Platform

15. Current SHIWA Simulation Platform (SSP v1.0) Deployed at production level since March 2011 Supports coarse-grained WF interoperability Components: –SHIWA WF Repository –SHIWA Portal (NGS P-GRADE Portal) –SHIWA VO (5 NGIs contribute) Native WF system is the P-GRADE WF system Enables either embedding or invoking ASKALON, GWES, Kepler, MOTEUR, Taverna and Triana WFs Allows e-scientists to search workflows uploaded to the SHIWA WF Repository and use them in their WF applications

16. SHIWA Workflow Interoperability – EGI UF11 16 Current SHIWA Simulation Platform P-GRADE GEMLCA P-GRADE

17. Summary See details of the project at: https://www.shiwa-workflow.eu/ If you want to get hands-on experience please visit our afternoon SHIWA tutorial session: –Hands-on SHIWA workflow interoperability solution https://www.shiwa-workflow.eu/tutorial If you want to be involved more in SHIWA activities, please, register into the SHIWA User Forum at the project web page –User Forum members will get Application porting support Trainings –If you register now you will get a SHIWA t-shirt

18. Thank you. Questions? https://www.shiwa-workflow.eu/