2-1.1 Job Submission Slides for Grid Computing: Techniques and Applications by Barry Wilkinson, Chapman & Hall/CRC press, © Chapter 2, pp For educational use only. All rights reserved. Aug 24, 2009

Types of jobs to be submitted to a Grid Programs written in C, C++, … that need to be compiled. Java programs that need a Virtual Java Machine Pre-compiled application packages 2-1.2

Submitting a job that needs to be compiled Fig. 2.1

Java programs Quite similar to compiling C programs, except Java compiler (javac) creates class file (bytecode) that is interpreted by a Java Virtual Machine (java). It is the Java Virtual Machine that is the executing program and the class file is an input file. Other class files usually need to be called too, found in path specified by CLASSPATH variable, so this variable must be set up properly

Submitting a Java job Fig. 2.2

Java programs offer more portability because class file could be sent to any remote computer having a Java Virtual Machine installed. However, speed of execution may be less than executing fully compiled binaries. Some studies have shown Java programs to run at 70% of equivalent C programs. Many internal components of Grid middleware software such as Globus actually use a mixture of Java and C. Java commonly used to create Web service components

Types of Applications Since Grid is a collection of computers, user might wish to use these computers collectively to solve problems. Two ways: Parallel programs -- Break problem down into tasks that need to be done to solve problem and submit individual tasks to different computers to work on them simultaneously. Parameter sweep problems -- Run same job on different computers at same time but with different input parameters. Particularly attractive for Grid computing platforms because no dependences between each sweep (usually)

Grid Resource Allocation Management (GRAM) Principal job submission component of Globus 2-1.8

Data Management Security Common Runtime Execution Management Information Services Web Services Components Non-WS Components Pre-WS Authentication Authorization GridFTP Grid Resource Allocation Mgmt (Pre-WS GRAM) Monitoring & Discovery System (MDS2) C Common Libraries GT2GT2 WS Authentication Authorization Reliable File Transfer OGSA-DAI [Tech Preview] Grid Resource Allocation Mgmt (WS GRAM) Monitoring & Discovery System (MDS4) Java WS Core Community Authorization Service GT3GT3 Replica Location Service XIO GT3GT3 Credential Management GT4GT4 Python WS Core [contribution] C WS Core Community Scheduler Framework [contribution] Delegation Service GT4GT4 Globus Open Source Grid Software I Foster GRAM

Job submission components Fig. 2.3

Running simple jobs across a Grid computing environment Fig. 2.4

Specifying the job Two basic ways a job might be specified: Directly by name of executable with required input arguments or By a job description file – more powerful

Directly For very simple jobs, one can submit a single job using -c option, e.g., globusrun-ws -submit -c prog1 arg1 arg2 which executes program prog1 with arguments arg1 and arg2 on local host. -c option actually causes globusrun-ws to generate a job description with the named program and arguments that follow. -c option must be the last globusrun-ws option (why?)

Example globusrun-ws –submit –c /bin/echo hello Globus job monitoring output created on command line and will indicate that the job completes. However, output from echo program (hello) not displayed and is lost as is any standard output without further specification (see later). 1b.14

Job Description File Gives details such as: Job Description - Name of executable - Number of instances - Arguments - Input files - Output files - Directories - Environment variables, paths,... Resource requirements - Processor - Number, cores,... - Type - Speed,... - Memory Used to match job with resources

Job Description Languages Several languages invented. Globus - specific: –Globus 1 and 2 used their Resource Specification language RSL (version 1) –Globus 3 used an XML version called RSL-2 –Globus 4 uses a variation of RSL-2 in a JDD (Job Description Document) Job Submission Description Language (JSDL) –A recent industry-wide standard (2005)

Resource Specification Language RSL version 1 A meta-language describing job and its required execution. Provides specification for: Job description - directory, executable, arguments, environment Resource requirements - machine type, number of nodes, memory, etc.

RSL Version 1 examples Constraints Example Conjunction (AND): & To create 3-5 instances of myProg, each on a machine with at least 64 Mbytes memory available to me for 1 hours: & (executable=myProg) (count>=3)(count =64) (max_time=60)

Constraints Example Disjunction (OR): | To create 5 instances of myProg, each on a machine with at least 64 Mbytes memory or 7 instances of myProg, each on a machine with at least 32 Mbytes memory : &(executable=myProg) (|(&(count=5)(memory>=64)) (&(count=7)(memory>=32)))

Requesting multiple resources multirequest: + To execute 5 instances of myProg1 on a machine with at least 64 Mbytes memory and execute 2 instances of myProg2: +(&(count=5)(memory>=64)) (executable=myProg1)) (&(count=2)(executable=myProg2))

XML Job Description languages With introduction of XML in early 2000’s, job description languages began to be changed to XML

Using XML Much more elegant and flexible, and in keeping with Web services. Can use XML parsers. Allows more powerful mechanisms with job schedulers. Resource scheduler/broker applies specification to local resources.

Resource Specification Language, RSL version 2 XM job description language used Globus version 3 (GT3). An XML language.

RSL-2 XML version of RSL 1 Can specify everything from executable, paths, arguments, input/output, error file, number of processes, max/min execution time, max/min memory, job type etc. etc.

GT 3 RSL-2 Example Specifying Executable (executable=/bin/echo)

RSL and GT 3.2 RSL-2 comparison for echo program &((executable=/bin/echo) (directory="/bin") (arguments="Hello World") (stdin=/dev/null) (stdout="stdout") (stderr="stderr") (count=1) ) <rsl:rsl xmlns:rsl=" xmlns:gram=" xmlns:xsi=" xsi:schemaLocation=" c:/ogsa-3.0/schema/base/gram/rsl.xsd c:/ogsa-3.0/schema/base/gram/gram_rsl.xsd">

Job Description Document (JDD) RSL-2 renamed and called JDD used in more recent Globus 4 (GT4) documents. Similar to original RSL-2 but simplified syntax. Not completely interchangeable.

GT 4 JDD Example Specifying Executable executable=/bin/echo /bin/echo

GT 4.0 JDD for echo program /bin/echo ${GLOBUS_USER_HOME} Hello World ${GLOBUS_USER_HOME}/stdout ${GLOBUS_USER_HOME}/stderr

Job Submission Description Language (JSDL) A standard introduced by GGF (Global Grid forum) in 2005 and beginning to be widely adopted

Basic JSDL structure

For executables operating in a Linux environment, replace with... Portable Operating System Interface, a collection of IEEE standards that define APIs, compatible to most versions of Unix/Linux

Sample Linux job description <jsdl:JobDefinition xmlns:jsdl=" xmlns:jsdl-posix=" Test Job Hello world Job /bin/echo hello, world ${GLOBUS_USER_HOME}/stdout ${GLOBUS_USER_HOME}/stderr

describes requirements of resources for job and can include:

Submitting a job

GT4 job submission command globusrun-ws Submit and monitor GRAM jobs Written in C, for faster startup and execution than earlier Java version Supports multiple and single job submission Handles credential management Streaming of job stdout/err during execution

Simple job submission Step 1: Create proxy with: grid-proxy-int command. Step 2: Issue globusrun-ws with parameters to specify job.

Some globusrun-ws flags (options) for job submission

Running GT 4 Job using XML job description file Command: globusrun-ws –submit –f prog.xml where prog.xml specifies job in JDD. -submit causes job to be submitted Submitted to localhost (machine that is executing command) as no contact resource specified. Submitted immediately using “fork”

With named executable -c option Example: Submit program echo with argument hello to default localhost. globusrun-ws –submit –c /bin/echo hello -c Causes globusrun-ws to generate job description with named program and arguments. -c option, if used, must be last option. Only useful for very simple single jobs.

Output modes -submit Submits (or resubmits) a job in one of three output modes: batch interactive, or interactive-streaming. Default (without additional flags to specify) is interactive.

Interactive mode Example Submit program echo with argument hello to default localhost. % globusrun-ws –submit –c /bin/echo hello Submitting job...Done. Job ID: uuid:d23a7be0-f87c-11d9-a53b aae1f Termination time: 07/20/ :44 GMT Current job state: Active Current job state: CleanUp Current job state: Done Destroying job...Done. Output Job ID Job goes thro several states

Streaming Refers to sending contents of a stream of data from one location to another location as it is generated. Often associated with Linux standard output and standard error streams, stdout and stderr. For a program that creates output on remote machine, need: Files to hold output and error messages,or Re-direct output and error messages to user console

Provides for capturing program output and error messages and re-directing them to user’s console (output of globusrun-ws) or to specified files Interactive-streaming mode -s option

Interactive-streaming mode Re-direction to user console -s option Example globusrun-ws -submit -s -c /bin/echo hello Output (hello) redirected to (globusrun-ws) stdout Error messages redirected to (globusrun-ws) stderr

s for streaming output and –so to specify output file –se to specify error file Interactive-streaming mode Re-direction to files - s option with –so and –se options

Example globusrun-ws -submit -s -so outfile -se errorfile -c /bin/echo hello name of file holding output Argument for echo name of file holding error messages

Example (JDD) /bin/echo Hello jobOut jobErr Specify streaming to files using Job description file

Batch submission A long-standing Computer Science term from early days of computing where jobs submitted to system in a group (a batch) and wait their turn to be executed sometime in the future. Originally appeared when programs were submitted by punched cards to a shared system, perhaps to be run perhaps overnight. (The author remembers those days with frustration.) Batch submission really part of a scheduling approach

Batch submission -b option In globusrun-ws, batch referred to as an output mode because of way output generated. Once job submitted, control returned to command line, and one will need to query system to find out status of job

For example, suppose we ran the job: globusrun-ws –submit /bin/sleep 100 in interactive mode. Would return when program (sleep for 100 seconds in this case) completes. We would get normal globusrun-ws output, such as: Submitting job...Done. Job ID: uuid:d23a7be0-f87c-11d9-a53b aae1f Termination time: 07/20/ :44 GMT Current job state: Active Current job state: CleanUp Current job state: Done Destroying job...Done. only each line would appear as process moves to next status condition

Alternatively, could execute sleep in batch output mode: (-b option): globusrun-ws –submit –b /bin/sleep 100 Output would immediately appear of the form: Submitting job…Done JoB ID: uuid:f c5-11d9-97e3-0002a5ad41e5 Termination time: 01/08/ :05 GMT Displays ManagedJob EPR as job ID (more on this later). Control returned to command line. Program may not have finished. In this case it will not for 100 seconds

Now one has to query state of job to find out when it completes. Need job ID (ManagedJob EPR) Convenient to have that put in a file using –o option when submitting job, e.g. globusrun-ws –submit –b -o jobEPR /bin/sleep 100 where jobEPR holds the job ID (ManagedJob EPR)

To watch status of submitted job “Attach” interactive monitoring with -monitor option. Job ID (ManagedJob EPR) provided with -j option, e.g.: globusrun-ws –monitor –j jobEPR where jobEPA holds ManagedJob EPR. Then can see stages job goes through with interactive output immediately: job state: Active Current job state: CleanUp Current job state: Done Requesting original job description...Done. Destroying job...Done although job itself still batch output job

Some other options -status Reports the current state of the job and exits -kill Requests immediate cancellation of job and exits.

2-1.56

Specifying where job is submitted Request to run job processed by “factory” service called ManagedJobFactoryService. Default URL:

To specify where job is submitted -F Specifies “contact” for the job submission. globusrun-ws –submit –F –f prog1.xml Job submitted to localhost Globus container that hosts services running on port 8440 Factory service still located at. wsrf/services/ManagedJobFactoryService

Selecting a different host Example globusrun-ws –submit –F services/managedJobFactoryService –f prog1.xml

Many other options Example -term time Set an absolute termination time, or a time relative to successful job creation

Transferring Files Job submission command, for example: globusrun-ws –submit –F –c prog1 requires prog1 to be existing on the remote machine in the default directory ( ${GLOBUS_USER_HOME} ). Up to user to ensure executable is in place.

GridFTP A Globus component that provides for: Large data transfers Secure transfers Fast transfers – Parallel transfers -- employing multiple virtual channels sharing a single physical network connection – Striping -- employing multiple physical channels using multiple hardware interfaces. Reliable transfers Third party transfers

Data Management Security Common Runtime Execution Management Information Services Web Services Components Non-WS Components Pre-WS Authentication Authorization GridFTP Grid Resource Allocation Mgmt (Pre-WS GRAM) Monitoring & Discovery System (MDS2) C Common Libraries GT2GT2 WS Authentication Authorization Reliable File Transfer OGSA-DAI [Tech Preview] Grid Resource Allocation Mgmt (WS GRAM) Monitoring & Discovery System (MDS4) Java WS Core Community Authorization Service GT3GT3 Replica Location Service XIO GT3GT3 Credential Management GT4GT4 Python WS Core [contribution] C WS Core Community Scheduler Framework [contribution] Delegation Service GT4GT4 Globus Open Source Grid Software I Foster GridFTP

Third party transfers Transferring a file from one remote location to another remote location controlled by a party at another location (the third party). Already seen third party transfers in Grid portal at file management portlet. There, user can initiate a transfer between two locations from portal running on a third system

GridFTP third party transfers Fig 2.5

ReliableFileTransfer (RFT) service GridFTP is not a Web/Grid service. ReliableFileTransfer (RFT) service provides service interface and additional features for reliable file transfers (retry capabilities etc.). RFT uses GridFTP servers to effect actual transfer

Data Management Security Common Runtime Execution Management Information Services Web Services Components Non-WS Components Pre-WS Authentication Authorization GridFTP Grid Resource Allocation Mgmt (Pre-WS GRAM) Monitoring & Discovery System (MDS2) C Common Libraries GT2GT2 WS Authentication Authorization Reliable File Transfer OGSA-DAI [Tech Preview] Grid Resource Allocation Mgmt (WS GRAM) Monitoring & Discovery System (MDS4) Java WS Core Community Authorization Service GT3GT3 Replica Location Service XIO GT3GT3 Credential Management GT4GT4 Python WS Core [contribution] C WS Core Community Scheduler Framework [contribution] Delegation Service GT4GT4 Globus Open Source Grid Software I Foster RFT

Globus file transfer command globus-url-copy Example globus-url-copy gsiftp:// file:///home/abw/ copies file hello from coit-grid02.uncc.edu to the local machine using GridFTP. Users needs valid security credentials (a certificate and proxy) Source URL Destination URL

Question Why three /’s in file URL, i.e. file:/// ? Answer The general form of file URL is file://host/path. If host omitted, it is assumed to be localhost, left with three /’s, i.e. file:///

File Staging Moving complete files to where they are needed. Usually associated with input and output files. Input file need to be moved to where program located Output files generated need to be moved back to user, or as input to other programs. Note different to input and output streaming, which moving a series of data items as a stream as it happens

File staging Fig 2.6

Staging example in JDD … file:///prog1Out gsiftp://coit-grid05.uncc.edu:2811 /prog1Out …

Staging example in JSDL /inputfiles/prog1Input overwrite gsiftp://coit grid05.uncc.edu:2811/prog1Input

Sources of GT 4 information

Questions (multiple choice)

When one issues the GT4.0 command: globusrun-ws -submit -F localhost:8440 -s -so hello1 -c /bin/echo hello what is hello? (a)A java class (b)An xml file containing the description of the job to be run (c)The executable to run in Globus (d)The argument for the program that will be executable

When one issues the GT4.0 command: globusrun-ws -submit -F localhost:8440 -s -so hello1 -c /bin/echo hello is the order of the flags important, and if so why? (a)Not important (b)Important: -c must be last as it uses the remaining arguments (c)Important: -s must be before -so (d)Important: -F must be first

When one issues the GT4.0 command: globusrun-ws -submit -F localhost:8440 -s -so hello1 -c /bin/echo hello what is localhost? (a)The server logged into running globusrun-ws. (b)The computer you are using to log into the server (c)None of the other answers.

What does the tag specify in an RSL-2/JDD file? (a) The number of different jobs submitted. (b)The number of computers to use. (c)The number of identical jobs to submit. (d)The number of arguments.