1. WS-VLAM Introduction presentation WS-VLAM Workflow Engine System and Network Engineering group Institute of informatics University of Amsterdam

2. WS-VLAM Introduction presentation Introduction This presentation contains information related to WS-VLAM engine It helps the reader understand how the workflow engine works, describes the current state and present the ongoing activities and planned ones. NOTE :  We are currently still working on the User ’ s and developer ’ s Guide, they are not available at this time. The presentations do not replace the documentation it just give a quick introduction.  Do not hesitate to contact us for further information Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome WS-VLAM developers gvlam-dev@lists.vl-e.nl

3. WS-VLAM Introduction presentation Needed background … To be able to follow the content of this presentation you will need to know about:  WSRF= Web Service Reference Framework WSRF  GT4= Globus Toolkit version 4 GT4 More detailed description of the presented work can be found in the section of publication of the WS-VLAM home pageWS-VLAM

4. WS-VLAM Introduction presentation Objective of this presentation… Why you should look at this presentations  You want to know how WS-VLAM works and interacts with the Grid middleware. Why you don ’ t have to worry if you do not understand the content on this presentation:  You are an end-user: it will be enough for you to read the User ’ s guide or the WS-VLAM composer presentationUser ’ s guide WS-VLAM composer  You are an application developer: it will be enough for you to read the Developer ’ s guide & the WS-VLAM developers presentationDeveloper ’ s guide WS-VLAM developers

5. WS-VLAM Introduction presentation Outline Introduction Architecture of the WS-VLAM Hierarchical workflow compositions & execution Conclusions

6. UvA, Amsterdam June 2007WS-VLAM Introduction presentation Process Management Stack Objectives: A WSRF enabled workflow engine Data Management Stack Network & Computing Resources Network & storage Resources Grid Middleware: Process & resource management Grid Middleware: Data management Workflow execution Workflow composition Workflow Management system Grid Services Grid Services Grid Services Application Templates Web service Web Service Interface OGSA DAI Workflow Engine Web service Workflow Engine Web service Web Service Interface Application web service Distributed Workflow

7. WS-VLAM Introduction presentation WS-VLAM Engine characteristics Implemented as GT4 WSRF service Uses GT4 delegation service for security Uses GSI enabled, private VNC for Graphical modules Uses notifications mechanisms provides by GT4 for monitoring the execution

8. UvA, Amsterdam June 2007WS-VLAM Introduction presentation GRAM services GT4 Java Container RTSM Factory Delegation service Worker nodes pre-ws-GRAM Client Job functions Delegate Service host(s) and compute element(s) WS-VLAM Engine: architecture RTSM Instance Workflow components

9. WS-VLAM Introduction presentation A three step execution process ClientDelegation Service RTSM Factory GRAM RTSM Instance Step 1 Step 2 Step 1: Create Delegated Grid Credentials Step 2: Instantiate the workflow components Step 3: Monitor the workflow execution Workflow components Step 3

10. UvA, Amsterdam June 2007WS-VLAM Introduction presentation Sequence-diagram WS-vlam Client GT4 Delegation Service RTSM Instance RTSM Factory 1. Create: delegation credential 2. Submit workflow execution plan GT4 GRAM Get the delegation credential EPR Get the RTSM instance EPR 4. Create: RTSM instance 3. Create: RTSM instance 5. subscribe: to notification events Get the notification events Step 1 Step 2 Step 3

11. WS-VLAM Introduction presentation Current Setting… Not yet available DAS-3 Sara SRB VU TUDELFT UvA Leiden University Server host Where the WS-VLAM engine is running

12. WS-VLAM Introduction presentation Step1: Create Delegated Credential VLAM Client RTSM Factory. proxy Initialize user proxy cert. Create delegated credential resource Credential EPR returned EPR GRAM Worker nodes Delegation Credential RP GT4 Service Container EPR = End Point Reference

13. WS-VLAM Introduction presentation Step 2: instantiates the workflow components and Create a RTSM instance VLAM Client Delegation RTSM Factory. Credential RP Request the creation RTSM instance Pass delegated credential EPR RTSM instance EPR returned EPR Access delegated credential resource Contact GRAM to instantiates the workflow components GRAM RTSM Instance Worker nodes GRAM instantiates the workflow components Workflow components Workflow components EPR = End Point Reference GT4 Service Container

14. WS-VLAM Introduction presentation Step3: monitors the application workflow VLAM Client RTSM Factory. Contact RTSM instance and subscribe for notification events GRAM RTSM Instance Worker nodes RTSM instance subscribes for notification events from GRAM and Workflow components Workflow components Workflow components GRAM and Workflow components generate events Notification events Delegation Credential RP GT4 Service Container

15. WS-VLAM Introduction presentation Workflow components with graphical output VLAM Client RTSM Factory. GRAM RTSM Instance Delegation Credential RP Worker nodes Workflow components Workflow components RTSM Instance GUI proxy GT4 Service Container vnc server GSI enabled connection

16. WS-VLAM Introduction presentation Support hierarchical workflows … workflow components can be composed of a collection of other components The advantage is now both the composition & the execution of complex workflows become easier

17. WS-VLAM Introduction presentation Server host Where the WS-VLAM engine is running Server host Where the WS-VLAM engine is running Setting for distributed hierarchical workflows … DAS-3 Sara SRB VU TUDELFT UvA Leiden University

18. UvA, Amsterdam June 2007WS-VLAM Introduction presentation Step2: instantiates the distributed workflow components & Create a RTSM instances GT4 Service Container VLAM Client RTSM Factory EPR GRAM RTSM Instance Worker nodes Workflow components Workflow components GRAM Worker nodes Workflow components Workflow components GT4 Service Container RTSM Instance GUI proxy Data proxy GUI proxy Data proxy Distributed RTSM RTSM Factory Distributed RTSM

19. UvA, Amsterdam June 2007WS-VLAM Introduction presentation Step2: instantiates the distributed workflow components & Create a RTSM instances GT4 Service Container VLAM Client RTSM Factory EPR GRAM RTSM Instance Worker nodes Workflow components Workflow components GRAM Worker nodes Workflow components Workflow components GT4 Service Container RTSM Instance GUI proxy Data proxy Data proxy GUI proxy Distributed RTSM RTSM Factory Distributed RTSM

20. WS-VLAM Introduction presentation Conclusions WSRF compliant middleware such as GT4 offers useful basic services to develop a truly grid enabled workflow engine:  Delegation service & ws-notification mechanisms proved to be very useful Separating the workflow composition from execution suits better the Grid environments

21. Some References 1.Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527 2.A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system, The 2 nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007 3.A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011 4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA UvA, Amsterdam June 2010WS-VLAM Introduction presentation

22. Some References 5.A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications, Future Generation Comp. Syst. 25 (5): 525-527 (2009) 6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007 7.Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007 UvA, Amsterdam June 2010WS-VLAM Introduction presentation

23. UvA, Amsterdam June 2007WS-VLAM Introduction presentation http://www.vl-e.nl/