1. Peter Kacsuk MTA SZTAKI kacsuk@sztaki.hu
SHIWA technology to enable the integration and collaboration of various workflow systems used in Europe
Peter Kacsuk
MTA SZTAKI

2. Who are the members of an e-science community from WF applications point of view?
WF System Developers (20-40)
Develop WF systems
Writes technical, user and installation manuals
WF Application Developers ( )
Develop WF applications
Publish the completed WF applications for end-users
End-users (e-scientists) ( )
Execute the published WF applications with custom input parameters by creating application instances using the published WF applications as templates 2

3. What does a WF developer need?
Portal
Using a portal/desktop to parameterize and run these applications, and to further develop them
WF App. Repository
Access to a large set of ready-to-run scientific WF applications
accessing a large set of various DCIs to make these WF applications run
Clouds
Local clusters
Supercomputers
Desktop grids (DGs)
(BOINC, Condor, etc.)
Cluster based
service grids (SGs)
(EGEE, OSG, etc.)
Supercomputer
based SGs
(DEISA, TeraGrid)
Grid systems
E-science infrastructure

4. WF developers worked in an isolated way in the past
Portal/desktop
Using a portal/desktop to develop WF applications
accessing a single DCI to make these WF applications run
Cluster based
service grids (SG)
(ARC.)
Grid systems
E-science infrastructure

5. Collaboration between WF application developers based on SHIWA technology
Publish WF applications in the repository to be continued by other appl. developers
Application developers use the portal/desktop to develop complex applications (executable on various DCIs) for various end-user communities
SHIWA App. Repository
Application developers
SSP Portal
Desktop grids (DGs)
(BOINC, Condor, etc.)
Cluster based
service grids (SGs)
(EGEE, OSG, etc.)
Supercomputer
based SGs
(DEISA, TeraGrid)
Grid systems
Local clusters
Supercomputers
Clouds

6. SHIWA Simulation Platform

7. Advantages for the various types of user communities using SHIWA
WF system developers
Better visibility: much more WF developers can access and use their WF system than before (through the applications stored in the SHIWA repo)
The joint impact is much bigger than the individual WF systems can achieve
WF developers
They can collaborate: share and re-use existing WF applications
WF application development can be accelerated
More complex WFs can be created in shorter time
They will access many different DCIs (their WF will be more popular)
End-users
much bigger set of usable and more sophisticated WF applications
These applications can run on various DCIs

8. What to do for a WF system developer to use SHIWA technology?
CGI approach using the SSP
Provide the WF engine as a legacy code for SHIWA developers
SHIWA developers will make it usable inside SSP
Estimated work: 1 month
Result: WF system A WFs can be used as components WFs in the meta-workflows constructed by the SSP

9. WF Engine A integrated into the SSP
SHIWA Science Gateway
SHIWA VO
SHIWA Portal
SHIWA Repository
WF1
WFn
gLite DCI
Globus DCI
local cluster
GEMLCA admin
GEMLCA Repository
Triana WE
WS-PGRADE Workflow editor
WF1
WFm
Taverna WE
WE1
WEp
WS-PGRADE Workflow engine
MOTEUR WE
GEMLCA with GIB
MOTEUR WE
GWES WE
ASKALON WE
Kepler WE
GEMLCA Service
ASKALON WE
SHIWA Proxy Server
Proxy Server
WF A WE

10. What to do for a WF system developer to use SHIWA technology?
CGI approach making WF system A as the native WF development and execution environment
Connect GEMLCA to WF system A
Estimated work: ~2 months
Result: In WF system A meta-workflows can be constructed and executed as now in the SSP

11. WF system A as the native WF dev. environment
SHIWA VO
local cluster
gLite DCI
Globus DCI
SHIWA Repository
WF1
WFn
search WF s1
Triana WE
Taverna WE
GEMLCA Repository
MOTEUR WE
WF1
WFm
WE + WF s5
WE1
WEp
ASKALON WE
researcher
Kepler WE
ASKALON WE
MOTEUR WE
GWES WE s2
invoke WE s6
pre-deployed- WEs
GEMLCA with GIB
WF A Workflow editor s4
GEMLCA Service s3 s4
WF A Workflow Engine
GEMLCA Client
GEMLCA UI 11

12. What to do for a WF system developer to use SHIWA technology?
DCI Bridge approach enables WF system A to run applications on various DCIs
Connect WF system A engine to DCI Bridge service
WF system A engine should be able to submit jobs as BES jobs using JSDL
Estimated work: We do not know yet how big work
Result: In WF system A workflows can run on various DCIs

13. Using DCI Bridge by WF System A
JSDL job submission
DCI Bridge
Clouds
Local clusters
Supercomputers
Desktop grids (DGs)
(BOINC, Condor, etc.)
Cluster based
service grids (SGs)
(EGEE, OSG, etc.)
Supercomputer
based SGs
(DEISA, TeraGrid)
Grid systems
Various DCIs will be usable

14. What to do for a WF system developer to use SHIWA technology?
SHIWA WF Bundle enables common WF representation in WF repositories
Implement SHIWA WF bundle in order to easily store your WF in the SHIWA WF repositories (and in other WF repositories, e.g. myExperiment)
Make various WF repositories interoperable
Estimated work: 1-2 months
Result: WF system A workflows will be easily accessible by other WF systems

15. What to do for a WF system developer to use SHIWA technology?
SHIWA IWIR enables common WF representation to transform WF A into WF B
Implement WF A to IWIR translator
Implement IWIR to WF A translator
Estimated work: 2-3 months each
Result:
WF system A workflows will be easily executable by other WF systems
Workflows written in other WF systems will be easily executable by WF system A

16. Conclusions
SHIWA brings advantage for all the 3 kinds of user communities:
WF system developers: higher visibility
WF developers: sharing and re-using WFs
End-users:
much bigger set of usable WF applications
These applications can run on various DCIs
With relatively little effort
WF system developers can adapt SHIWA technology
Or WF systems can join the SSP