1. WS-VLAM Tutorial: Part III (part3): Learn how to WS-VLAM JAVA API Adam Belloum

2. Outline Part I: Hands on the User Graphical Interface Part III: Port a Legacy application to WS-VLAM Part III: Learn how to use WS-VLAM API Notation (Title background) : –Generic step –Tutorial specific step Part I: Hands on the User Graphical Interface Part III: Port a Legacy application to WS-VLAM Part III: Learn how to use WS-VLAM API

3. Outline (Part III): Learn how to use WS-VLAM API Step 1: Define an Interface for your component Step 2: Create a description for your component Step 3: Define your component dependencies Step 4: JAVA Tutorial (create a WS-VLAM JAVA components) Step 6: C/C++ Tutorial ( create a WS-VLAM C/C++ components) Step 5: PYTHON Tutoria l (create a WS-VLAM PYTHON components) Step 6: Deploy the component/dependencies lib Step 7: Compose a workflow using the component Step 8: execute the workflow / monitor

4. Step 1: Define the component interface The Interface defines how WS-VLAM will interactwith your components – Parameter Interface: is use to interact at runtime with your module – Port interface: is used to stage in/out the data consumed/produced by your module Port in allows to stage in the consumed data Port out allows to stage out the produced data

5. Step 1: Example File Reader The file reader “ fileReader ” opens a file specified as a URI using the parameter Interface “ filename ” and sends the content through an output port “ port_out ”. – fileReader support many protocols: GSIFTP, ftp, http, file

6. Step 2.1: Create a description To create a description of your component : use CLAM workflow component configuration tool Action: Click on CLAM Button in the WS- VLAM menubar

7. Step 2.2: Create a description CLAM workflow component configuration toolallows you to define: the name of the component,default host, ports, parameters, classification, and category Action: Enter the name of thecomponent Entre the name of thedefault host“ fs2.das3.science.uva.nl ”

8. Step 2.3: Create a description CLAM workflow component configuration tool allowsyou to define: the name of the component, defaulthost, ports, parameters, classification, and category Action: Click on ports tab Click + to add new input or output port Type the name of theports exactly as you will define them thesource code.

9. Step 2.4: Create a description To add a port a pop up window appears “ Add input Port ” or “ Add output port ” Action: Type the name of the ports exactly as you didit in source code Specify the type of the port either a simple data type, a file type, or Complex data type (see documentation formore details)

10. Step 2.5: Create a description Create a description of your component : use CLAM workflow component configuration tool Action: Click on save tocreated the descriptionfile “ component- name.xml ”

11. Step 2.6: Create a description Create a description of your component : use CLAM workflow component configuration tool Action: You will see in the filed file path the fullpath to the create Component-name

12. Step 2.7: Example File Reader Create a description of fileReader : use CLAM workflow component configuration tool The FileReader has One output: port-name = “port_out” and Type=“file” One parameter parameter-name = “filename” and Type=“String” Action: Perform the tasks described in step 2.1 to step 2.6 to create the fileReader.xml

13. Step 3.1: Define the component dependencies Dependencies are libraries & auxiliary software that are required by your component but are not available on standard Operating systems You have to list the name of these libraries & auxiliary software in a file called “dependencies” Every libraries & auxiliary software listed in the dependencies files have to be packaged as a“pseudo-module”

14. Step 3.2: Create a pseudo-module A pseudo module is just asimple software packagefor which you need tocreate a profile.sh in the root directory Using your favorite texteditor created a file callprofile.sh # ############ NOTE #################### # In the profile.sh export all the environment # variables needed to execute your package # # bash environment # # export VARIABLE_NAME=VARIABLE_VALUE # # Add your VARIABLES bellow the line ######################################

15. Step 4.1: YOUR JAVA Module From a developer ’ s point of view your components are applications that use a speciallibrary provided by the WS-VLAM framework:the “ vlport ” library

16. Step 4.2: JAVA modules (pre-requisite) Before you build the j wrapper make sure that the SWIG toolkit and Python are installed in your system. The SWIG version should be higher then 1.3.19. To compile the java wrapper – the environment variable VLAM_INSTALL must be set. – Just type ‘‘make dist‘‘. – You will find the library in dist/lib subdirectory. – The wrapper consists of an ”adapter” shared library with thename ” jwrapper.jar ” and ” jvlport.jar ”

17. Step 4.3: Extend your JAVA code 1. Initialize– import nl.wtcw.vlamg.jwrapper. ∗ ; 2. Create a reference to the input port – portIRef = VLIPort.getInstance(" portIn "); 3. Create a reference to the output port– VLOPort portORef =VLOPort.getInstance(" portOut "); – Note: should provide the same names for the ports asspecified at the initialization stag

18. Step 4.4: API for reading from input port void readInt() – reads an integer number from an input port; void readFloat() – reads a float number from an input port; void readDouble() – reads a double precision number from an inputport; void readString() – reads a string from an input por

19. Step 4.4: API for reading from input port void hardSync ( ) – prepares an output bu ff er and waits until it is sent to network. void readBytes (byte[ data) ] – reads a byte array (raw data) from an input port.Array must be prepared before; void readShort ( ) – reads a short number from an input port; » void readLong () – reads a long integer number from an input port;

20. Step 4.5: API for writing to output port void writeBytes (byte[ data)] – writes a byte array to an output port void writeShort (short number) – writes a short to an output port » void writeLong (long number) – writes a long integer to an output port » void writeInt (int number) – writes an integer to an output port

21. Step 4.5: API for writing to output port void writeDouble (double number) – writes a double precision number to an outputport; void writeString (String str) – writes a string to an output port; void flush () – prepares an output bu ff er to be sent over network (flush internal library bu ff er)

22. Step 4.6: API for reading/writing to/from parameter getParameter (String param−id ) – Reads the value of “ param−id ” setParameter ( String param−id, String para−value) – Writes the value of “ para−value ” to “ param−id ”

23. Step 4.7: JAVA component XXXXXXXX: // YOU MUST IMPORT THE JWRAPPER PACKAGE import nl.wtcw.vlamg.jwrapper. ∗ ; VLOPort outputPortRef = null; VLIPort inputPortRef = null; … // IT IS ASSUMED THAT REFERENCES TO THE PORTS ARE ALREADY TAKEN try { double rndn = 12345.12345; // WRITING TO THE WS −VLAM Output Port outputPortRef.writeDouble(rndn);... // READ FROM THE WS −VLAM INPUT PORT rndn = inputPortRef.readDouble();... } catch(VLAPPException e){ // HANDLE EXCEPTION... }...

24. Step 4.8: JAVA code of the File Reader The Example : import nl.wtcw.vlamg.jvlport.VLAppFactory; import nl.wtcw.vlamg.jvlport.Vlport; import nl.wtcw.vlamg.jvlport.OutputStream; import java.io.*; /** * Created by IntelliJ IDEA. * User: dvasunin * Date: Jul 29, 2005 * Time: 3:49:34 PM * To change this template use File | Settings | File Templates. */ public class JFileReader { static public void main(String args[]){ String line = null; String[] outputPorts; try { outputPorts = new String[] {"port_out"}; Vlport vlport = (vlport.getOutputPort("port_out"))); VLAppFactory.activate(null, outputPorts); PrintWriter pw = new PrintWriter( new OutputStreamBufferedReader reader = new BufferedReader( new FileReader("/etc/passwd")); while((line = reader.readLine()) != null){ System.out.println(line); pw.println(line); } pw.flush(); } catch (IOException e) { e.printStackTrace(); } finally{ VLAppFactory.deactivate(); System.exit(0); }

25. Step 4.9: build your JAVA component build.xml for FileReader (½) Project : VLE-wsvlam Component : JFileReader source Info: Ant file to call 'make'. Builds wsvlam java modules. This component can be used as standalone. In that case the Makefile should be called directly.

26. Step 4.9: build your JAVA component build.xml for FileReader (2/2)

27. Step 4.10: package your FileReader Create a directory with thefileReader name as the component Copy the files you havecreated: – build.xml (step 4.9) – fileReader.xml (step 2.7) Copy any other files ordirectory related to yourcomponent: – src/fileReader.java (step 4.8) (1) Compile the fileReader $ cd fileReader $ ant # this will created java classes in # fitleReader/bin (2) Create a tar file of your component: $ tar cf fileReader.tar fileReader (3) Compress the tar file you have created $ gzip fileReader.tar

28. Step 5: Deploy the fileReader Test mode : you can test your component – Perform the stepsdescribed in the bluebox Shared mode : you can share your modulewith other users: – by moving the tar fileyou have created instep 4 to a componentrepository # On a machine where both GT4 & WS-VLAM # runtime system are installed # Create in your HOME directory called “.wsvlam” (1) save fileReader.tar.gz to the default location $ mkdir.wsvlam $ cd.wsvlam/ $ mkdir modules $ cd modules #untar the component tar file you have created # in step 4 (2) untar fileReader.tar.gz $ tar –zxvf fileReader.gz

29. Step 6: Complete examples in JAVA components Complete examples in JAVA can be found here: – http://www.science.uva.nl/~gvlam/wsvlam/Documentation/wsvlam-module-examples.tar.gz Download wsvlam-module-examples.tar.gz untar the file and save the examples in a directory:$HOME/.wsvlam/modules (see step 5)

30. Step ?: Template for component in JAVA Is this needed ???????? Dmitry do westill needthis?????? your-java-main-class -server /path/to/something.jar

31. Step 6.1: Compose a workflow fileReader In the component list window you will see the created FileREader components as well as othercomponents Action: Click on thefileReader in the listand drag and drop inthe composition panel

32. Step 6.2: Compose a workflow using LA An instance of fileReader will be instantiated Action: Click on the fileReader in the list and drag anddrop in the compositionpanel

33. Step 6.3: Compose a workflow using LA Instantiate the fileWriter component: This component will move the file produced by fileReaderto any remote location the user specifies Action: Click on the WriterReader in the list and drag and drop in thecomposition panel

34. Step 6.4: Compose a workflow using LA Create a communication channel: This component will move the file produced by FileReaderto any remote location the user specifies Action: Create a communicationchannel by dragging between an output port ( read square) and input port ( blue square)

35. Step 7.1: customize the composed workflow In the step you should check if all the default valuesof the parameters of the workflow componentscomposing your workflow are set correctly Action: Click on the fileReader in composition panel In the property window select the parameter tab Type in the URI to the file you want stage in FileReadersupport many protocols:GSIFTP, http, file, ftp

36. Step 7.1: customize the composed workflow In the step you should check if all the default valuesof the parameters of the workflow componentscomposing your workflow are set correctly Action: Click on the fileWrite r in composition panel In the property window select the parameter tab Type in the URI to the file you want stage in FileReadersupport many protocols:GSIFTP, http, file, ftp

37. Step 7.1: customize the composed workflow Input file: fileReader parameter fs2.das2.science.uva.nl: Type in the following URI for parameter in each fileReader gsiftp:// fs2.das.3science.uva.nl/home2/ / output file: fileWriter parameter fs0.das3.cs.vu.nl : Type in the following URI for parameter in each fileReader gsiftp:// fs0.das3.cs.vu.nl/home0/ /

38. Step 8: execute the composed workflow Now you are ready to execute the omnimatch workflow, which automatically stage in the data fileneeded for the omnimatch application, and stage outthe result files Action: Click on the run button

39. Some References 1.Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527 2.A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS- VLAM: a GT4 based workflow management system, The 2 nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007 3.A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011 4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA

40. Some References 5.A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications, Future Generation Comp. Syst. 25 (5): 525-527 (2009) 6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007 7.Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007