gLite L. Stepanova1 Luidmila Stepanova Dubna, Jul 4, 2008 WMProxy Python API tests

gLite L. Stepanova2 Introduction The WMProxy (Workload Manager Proxy) is service providing access to the WMS (Workload Management System) functionality through a Web Services based interface. A set of API allows calling the WMProxy server. The main Python class of these APIs is called Wmproxy: it is stored inside wmproxymethods module and it allows calling all WMProxy services. A client object can be created using the following constructor: Wmproxy(url, ns=““, proxy=““) url= : /glite_wms_wmproxy_server Default ns is Default proxy is /tmp/X509up_u port is 7443

gLite L. Stepanova3 How to start the test The test contains several sub tests in directory test. The test can be run by executing./start_tests.sh WMSLB_HOST [-output filename] subtests listed in the file test-sequence.lst will be executed. The test can be run by executing./start_tests.py WMSLB_HOST [filename] The start_test.py is based on the Python's ConfigParser module. The module provides a parser for simple configuration WMProxy_API.conf file, where: if the subtest is defined as TRUE then subtest will be executed. The WMSLB_HOST is obligatory commandline argument. If the option is given the output should be directed to the file

gLite L. Stepanova4 Script getversion The subtest getversion calling client.getVersion and if version and later client.getTransferProtocols methods. getVersion - gets the version of the service. getTransferProtocol - gets transfer protocols

gLite L. Stepanova5 Example of getversion output ######## script name is getversion ######## time is :09: | OK | getVersion | OK | getTransferProtocol #################################### ====== script name is getversion ======== WmProxy_host: grinr05.inr.troitsk.ru ========== OK protocol version = 2.2.0=========== ======= OK get available Tranfer Protocols: ======== gsiftp https **************** test getversion is OK *******************

gLite L. Stepanova6 Credentials Delegation Sequence Diagram glite-proxy-cert

gLite L. Stepanova7 Script delegation This subtest use the method getProxyReq for get proxy request from the server. Proxy request signing by command: glite-proxy-cert -e DELEGATION_REQUEST were -e gets the proxy request from machine environment DELEGATION_REQUEST. The method putProxy finishes the delegation procedure by sending the signed proxy certificate to the server. Following Wmproxy Methods are included in the test: getVersion - gets the version of the service. getGrstNs - gets GridSite Delegation name space string representation proxyRequest - gets proxy request from the server. putProxy - finishes the delegation procedure by sending the signed proxy certificate to the server. getDelegatedProxyInfo - this operation returns the Delegated Proxy information.

gLite L. Stepanova8 Examples of delegation output ############### script name is delegation ############### time is :09: | OK | getVersion | OK | getGrstNs | OK | proxyRequest | OK | putProxy | OK | getDelegatedProxyInfo ######################################## ################ ============ script name is deligation =========== WmProxy_host: grinr05.inr.troitsk.ru ========== OK protocol version = 2.2.0=========== =========== Delegation of user credential ======== OK GridSite Delegation namespace string representation:

gLite L. Stepanova9 Examples of delegation output ===============proxyRequest=============== -----BEGIN CERTIFICATE REQUEST----- MIHIMHQCADAQMQ4wDAYDVQQKEwVEdW1teTBcMA0GCSqGSIb 3DQEBAQUAA0sAMEgC QQCyK4+iYJJTedfRzsLEkqKyxaV50cZphQXjYOiGyo4JtCDlXgvLLI7Z R/tqANR2 qOAxNBFJbsGpIYxOsksUSw7dAgMBAAGgADANBgkqhkiG9w0BAQ QFAANBAJz+3E2d hNzdN1bZMl30xJKS+tDo9PFRwRec/XnTER5DWRwHEKC1KtpovUC sG/6TcbMa1EP/dEDG+Q+qhbkO9zs= -----END CERTIFICATE REQUEST----- ==== Path of X509_USER_PROXY is /tmp/x509up_u3088 ==== glite-proxy-cert -e DELEGATION_REQUEST ==== client.putProxy(delegationId, proxyRequest) is OK ==== ************ test deligation is OK *****************

gLite L. Stepanova10 Script jobstart_submit The subtest jobstart_submit is the Single Job Submission. The sequence of WMProxy methods necessary for the delivery of the user's job to the WM queue is used. Following WMProxy Methods are included in the test: 1. the delegation procedure getVersion getGrstNs proxyRequest putProxy getDelegatedProxyInfo 2. jobListMatch -This method provides the list of CEs matching the requirements of the job specified in the JDL. 3. the job submit procedure jobSubmit - This operation submits a job. The JDL description of the job provided by the client is validated by the service, registered to the LB and finally passed to the Workload Manage.The unique identifier assigned to the job is returned to the client. 4. the job start procedure jobRegister - This operation registers a job for submission getJobProxyInfo - Retrieve useful information about the user delegated proxy, the delegated identification details of the user, the jdl. getACLItems - This operations gets a list of items from the job Access Control List.

gLite L. Stepanova11 Script jobstart_submit getSandboxDestURI -This operation returns a destination URI associated to the job, identified by the jobId provided as input, where the job input sandbox files can be uploaded by the client on the WMS node. getSandboxBulkDestURI -This operation returns the list of destination URIs associated to a compound job. In the given test it is equivalent to the getSandboxDestURI. getFreeQuota - This operation returns the remaining free part of available user disk quota (in bytes). getMaxInputSandboxSize -This operation returns the maximum Input sandbox size (in bytes) a user can count-on for a job submission if using the space managed by the WM. jobStart -Trigger the submission a previously registered job. jobCancel -This operation cancels a previously submitted job identified by its JobId.

gLite L. Stepanova12 JDL file for Single Job Submission JDL file is created by the test. Executable = "/bin/hostname"; Arguments = ""; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out","std.err"}; VirtualOrganisation = "dteam"; requirements = true; rank = 1;

gLite L. Stepanova13 Examples of jobstart_submit output ################ script name is jobstart_submit ############### time is :25: | OK | getVersion | OK | getGrstNs | OK | proxyRequest | OK | putProxy | OK | getDelegatedProxyInfo | OK | jobListMatch | OK | jobRegister | OK | getJobProxyInfo | OK | getACLItems | OK | getSandboxDestURI | OK | getSandboxBulkDestURI | OK | client.getFreeQuota | OK | client.getMaxInputSandboxSize | OK | jobStart | OK | jobCancel | OK | jobSubmit | OK | jobCancel

gLite L. Stepanova14 Examples of jobstart_submit output ################ script name is jobstart_submit ############### time is :25: | OK | getVersion | OK | getGrstNs | OK | proxyRequest | OK | putProxy | OK | getDelegatedProxyInfo | OK | jobListMatch | OK | jobRegister | OK | getJobProxyInfo | OK | getACLItems | OK | getSandboxDestURI | OK | getSandboxBulkDestURI | OK | client.getFreeQuota | OK | client.getMaxInputSandboxSize | OK | jobStart | OK | jobCancel | OK | jobSubmit | OK | jobCancel

gLite L. Stepanova15 Examples of jobstart_submit output ======script name is jobstart_submit =============== ============ WmProxy_host: cg08.ific.uv.es=========== ========== protocol version = 2.2.0=========== ============ Delegation of user credential is OK ======== GridSite Delegation namespace string representation : ============ proxyRequest is OK =============== Path of X509_USER_PROXY is /tmp/x509up_u500 glite-proxy-cert -e DELEGATION_REQUEST ==== client.putProxy(delegationId, proxyRequest) is OK ==== ======= VirtualOrganisation =dteam =========== Count of CE Ids satisfying the job Requirements specified in the JDL: listSize=26

gLite L. Stepanova16 Scripts jobcllection and jobdag This subtests differ in JDL attributes A Direct Acyclic Graph (DAG) is a set of jobs (sub- nodes) where the input, output, or execution of one or more jobs depends on one or more other jobs. The jobs are graph nodes (vertices), the edges (arcs) identify the dependencies. A Job Collection is a group of jobs without dependencies. Delegation in these tests is executed by means of the command : glite-wms-job-delegate-proxy -d delegationId -- endpoint url were -d delegationId - the proxy that will be delegated is identified by. --endpoint url - the operations are performed contacting the WMProxy service represented by the given serviceURL.

gLite L. Stepanova17 Wmproxy Methods are included in the jobcol and jobdag scripts Methods for Job DAG and Job Collection: getSandboxBulkDestURI -returns the list of destination URIs associated to a compound job. This feature is particularly useful when submitting compound requests: DAG, Collections. getChildren -returns the list of all sons (as a list of JobIdStruct) getJobID -returns the JobId string representation getNodeName -returns, if the istance represents a dag or collection node, the name of the node. Next methods described earlier: getDelegatedProxyInfo jobRegister getJobProxyInfo jobSubmit jobCancel

gLite L. Stepanova18 JDL file for DAG Job Submission [ Type = "dag"; Executable = "/bin/hostname"; VirtualOrganisation = "dteam"; Requirements = true; Rank = other.GlueCEPolicyMaxRunningJobs - \ other.GlueCEStateRunningJobs; FuzzyRank = true; nodes= [nodeA = [Description = [Executable = "/bin/hostname" ; StdOutput = "jobA.out"; StdError = "jobA.err"; OutputSandbox = {"jobA.err", "jobA.out" };]; ]; nodeB = [Description = [Executable ="/bin/hostname"; ]; ]; dependencies = { { nodeA, nodeB}}; ]; ]

gLite L. Stepanova19 JDL file for Collection Job Submission [ Type = "collection"; Executable = "/bin/hostname"; VirtualOrganisation = "dteam"; Requirements = other.GlueCEUniqueid == \ "grce001.inr.troitsk.ru:2119/jobmanager-lcgpbs-dteam"; Rank = other.GlueCEPolicyMaxRunningJobs - \ other.GlueCEStateRunningJobs; Nodes = { [JobType = "normal"; Executable = "/bin/hostname" ; Arguments = ""; StdOutput = "job1.out"; StdError = "job1.err"; OutputSandbox = {"job1.err", "job1.out" }; ],[JobType = "normal"; Executable = "/bin/hostname"; Arguments = ""; StdOutput = "job2.out"; StdError = "job2.err"; OutputSandbox = {"job2.err", "job2.out" };] }; ]

gLite L. Stepanova20 Examples of jobdag output ############## script name is jobdag ####### time is :45: | OK | getDelegatedProxyInfo | OK | jobRegister | OK | getJobProxyInfo | OK | jobSubmit | OK | getSandboxBulkDestURI | OK | getChildren | OK | getNodeName = nodeB | OK | getNodeName = nodeA | OK | jobCancel ######################################## ====== script name is jobdag========= =====WmProxy_host: cg08.ific.uv.es===== =======VirtualOrganisation =dteam=====

gLite L. Stepanova21 Examples of jobdag output ==the list of destination URIs associated to a compound job=== gsiftp://cg08.ific.uv.es:2811/var/glite/SandboxDir/z6/https_3a_2f _2fcg08.ific.uv.es_3a9000_2fz64WSVXjl97TG_5fhwhAsszg/inputht tps://cg08.ific.uv.es:7443/SandboxDir/z6/https_3a_2f_2fcg08.ific. uv.es_3a9000_2fz64WSVXjl97TG_5fhwhAsszg/inputhttps://cg08.if ic.uv.es:9000/f-X9wumIHqmxJCb8--lCoQ gsiftp://cg08.ific.uv.es:2811/var/glite/SandboxDir/f- /https_3a_2f_2fcg08.ific.uv.es_3a9000_2ff-X9wumIHqmxJCb8-- lCoQ/inputhttps://cg08.ific.uv.es:7443/SandboxDir/f- /https_3a_2f_2fcg08.ific.uv.es_3a9000_2ff-X9wumIHqmxJCb8-- lCoQ/inputhttps://cg08.ific.uv.es:9000/dA- MMnBPzCkHDY0Qn65VZg gsiftp://cg08.ific.uv.es:2811/var/glite/SandboxDir/dA/https_3a_2f _2fcg08.ific.uv.es_3a9000_2fdA- MMnBPzCkHDY0Qn65VZg/inputhttps://cg08.ific.uv.es:7443/Sandb oxDir/dA/https_3a_2f_2fcg08.ific.uv.es_3a9000_2fdA- MMnBPzCkHDY0Qn65VZg/input

gLite L. Stepanova22 Examples of jobdag output =========task.getChildren====== NodeJobId = vmUEaqi8j5Q NodeName = nodeB NodeJobId = FTMNcpgs5MeCkLOkYA NodeName = nodeA OK: Job Tis5vKr4q9rQ was cancel ****** test jobdag is OK ******

gLite L. Stepanova23 Examples of jobcol output ####### script name is jobcollection ####### time is :49: | OK | getDelegatedProxyInfo | OK | jobRegister | OK | getJobProxyInfo | OK | jobSubmit pQ | OK | getSandboxBulkDestURI | OK | getChildren | OK | getNodeName = Node_0 | OK | getNodeName = Node_1 | OK | jobCancel pQ ################################# ======= script name is jobcollection ======== ======WmProxy_host: cg08.ific.uv.es======= =======VirtualOrganisation =dteam=====

gLite L. Stepanova24 Examples of jobcol output ==== task.getChildren ==== NodeJobId = UueoFZnA NodeName = Node_0 NodeJobId = PtWyoA NodeName = Node_1 OK: Job XALHynpQ was cancel ***** test jobdcollection is OK ******

gLite L. Stepanova25 Further information See also: AMTest#WMProxy_Python_API bin/glite.cgi/org.glite.testsuites.ctb/WMS -api/python