1. 1 2010-04-27 EDGI European Desktop Grid Initiative gLite job submission to EDGI EDGI is supported by the FP7 Capacities Programme under contract nr RI-261556

2. ContentsContents Single job submission to EDGI through gLite Single job submission to EDGI through gLite Metajob submission to EDGI through gLite Metajob submission to EDGI through gLite GBAC submission to EDGI through gLite GBAC submission to EDGI through gLite For each type of submission, there will be a step-by-step explanation in order to ease the understanding. 2

3. Single job submission 3

4. Step 1: Choose application from the EDGI AR 4 Let us select the dsp app…

5. Step 2: Get the gridftp url of its glite binary 5 gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp

6. Step 3: Get the VO and site list and select 6 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

7. Step 4: create jdl Create your dsp.jdl: Create your dsp.jdl: Executable = "dsp"; Arguments = "-f 22 -i 22 -p 723 -n pools.txt"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp", "pools.txt"}; OutputSandbox = {"cost.txt"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo“; 7 Executable reference in the EDGI AR Input file

8. Step 5: submit jdl Proxy creation: Proxy creation: > glite-voms-proxy-init -voms edgiprod.vo.edgi-grid.eu:/edgiprod.vo.edgi-grid.eu/Role=edgidemo Submit: Submit: > glite-wms-job-submit -a -o id dsp.jdl Connecting to the service https://wms.grid.edges-grid.eu:7443/glite_wms_wmproxy_server https://wms.grid.edges-grid.eu:7443/glite_wms_wmproxy_server ================== glite-wms-job-submit Success =================== The job has been successfully submitted to the WMProxy Your job identifier is: https://lb.grid.edges-grid.eu:9000/mhma6si6x__9zyfhG8KCuw The job identifier has been saved in the following file: /home/lpds/smith/TEST-EDGIDEMO/DSP/id======================================================== 8

9. Step 6: get status Status: Status: glite-wms-job-status -i id ================== glite-wms-job-status Success ================= BOOKKEEPING INFORMATION: Status info for the Job : https://lb.grid.edges-grid.eu:9000/YFlhyw24Oh6AhZB6Yyk8Gg Current Status: Running Status Reason: unavailable Destination: cr1.edgi-grid.eu:8443/cream-pbs-edgidemo Submitted: Wed May 16 14:59:48 2012 CEST ===================================================== 9

10. Step 7: logging Logging: Logging: glite-wms-job-logging-info -v 2 -i id Check UserTags! It returns: 3gbridge_internal_id3gbridge_dg_url3gbridge_dg_id The last one is a URL pointing to the WU! 10

11. Step 8: track the job in BOINC Inspect BOINC WU Inspect BOINC WU 11

12. Step 9: get the result 12

13. Metajob submission 13

14. Step 0: prepare your inputs Upload your individual input files to web server: Upload your individual input files to web server: http://somewhere.com/pools1.txt http://somewhere.com/pools1.txt … http://somewhere.com/pools10000.txt http://somewhere.com/pools10000.txt Create the description of your metajob: Create the description of your metajob: %Required 100% %SuccessAt 100% %Comment pools1.txt Arguments = "-i 0 -n pools.txt -f 22 -p 723“ Input = pools.txt=http://somewhere.com/pools1.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652 Queue[….] %Comment pools10000.txt Input = pools.txt=http://somewhere.com/pools10000.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652 Queue 14

15. Step 1: Choose application from the EDGI AR 15 Let us select the dsp app…

16. Step 2: Get the gridftp url of its glite binary 16 gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp

17. Step 3: Get the VO and site list and select 17 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

18. Step 4: create and submit jdl Create your jdl: Create your jdl: Executable = "dsp"; Arguments = "-f 22 -i 22 -p 723 -n pools.txt"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp", "pools.txt", "_3gb-metajob-dsp-10000" }; OutputSandbox = {"cost.txt"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo"; Submit: Submit: glite-wms-job-submit -o id my-metajob-10000.jdl 18 Reference to the app in the EDGI AR Basic input file Extra input file

19. Step 5: Query the logging and get the url 19 Location of the status description

20. Step 6: Query the status 20 Metajob handling rules Actual status of jobs Location of the mapping

21. Step 7: Download (and extract) the results 21 Download result as usual: glite-wms-job-output -i id --dir outputs =>./outputs/cost.txt Extract it: tar zxvf cost.txt =>./outputs/ /cost.txt See the mapping between your individual job definition and the jobids (which gives the name of directories storing the output files of your app) => one subjob id one job

22. GBAC job submission 22

23. EDGIDemo BOINC project … Overview 3G Bridge … BOINC Client GBAC Application and Inputs BOINC Client GBAC Application and Inputs 1.A User submits her application to EDGIDemo trough gLite. Using GBAC as a virtualization framework requires the real binaries and inputs defined as GBAC inputs. 2.Clients (who have VirtualBox installed) will download the BOINC native GBAC application with the submitted application and its inputs. 3.GBAC starts a Linux Virtual Machine (using VirtualBox). 4.GBAC copies the applications and inputs into the Virtual Machine. 5.The application is executed in the Linux VM. 6.The result is fetched from the VM by GBAC. 7.The VM is shut down and discarded. 8.GBAC finishes and the result is returned to EDGIDemo from the Client. 9.The results are returned to gLite through 3G Bridge from EDGIDemo. 10. User can retrieve the results by using gLite commands

24. Step 1: Get the gridftp URL of GBAC To submit a virtualised app choose the GBAC app from the EDGI AR To submit a virtualised app choose the GBAC app from the EDGI AR Then query the Grid FTP url of the file “gbac.nobin” under the gLite implementation (as you did it during the single submission) Then query the Grid FTP url of the file “gbac.nobin” under the gLite implementation (as you did it during the single submission) Gridftp url: Gridftp url:gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1355/1512/gbac.nobin 24

25. Step 2: Get the VO and site list and select 25 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

26. Step 3: Create “helloworld” application Notes: package must have the name X.app.tgz, where X is the name of the executable which must be first launched after unpackaging. Notes: package must have the name X.app.tgz, where X is the name of the executable which must be first launched after unpackaging. 26

27. Step 4: create jdl Create your jdl: Create your jdl: Executable = “gbac.nobin"; Arguments = “anything"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1355/1512/gbac.nobin", "myhelloworld.sh.app.tgz" }; OutputSandbox = {"output"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo"; 27 GBAC reference in the EDGI AR Your hello world applicationName of GBAC app framework

28. Step 5: Submit jdl Submitting the job: Submitting the job: glite-wms-job-submit –a –o id helloworld.jdl 28

29. Step 6a: Check the status of the job under BOINC 29

30. Step 6b: Waiting for the job to finish Get the status of the job: Get the status of the job: glite-wms-job-status -i id 30

31. Step 7: get the result Get the result: Get the result: glite-wms-job-output -i id --dir outputs 31

32. Controlled hands-on has finished at this point, please play with the tools you learnt or try any of the following tasks: 1.Try another application in the EDGI AR 2.Assume dsp is an MC type application, submit it 20 times with the same input (“queue 20”) 3.Take a real application and submit it using GBAC 4.For advanced users: combine MetaJob and GBAC: submit the helloworld application 10 times a)With different inputs b)With the same input 32

33. Thank you for your attention! www.edgi-project.eu 33