JSDL/BES Tutorial Moreno Marzolla, INFN Padova OMII-EU All Hands Meeting, Stockholm, jul 31—aug 2, 2007

EU project: RIO31844-OMII-EUROPE Talk outline Introduction to job submission The JSDL specification The OGSA-BES specification Conclusions

EU project: RIO31844-OMII-EUROPE Aim of this presentation This presentation aims at providing the audience with an overview of the Job Submission Description Language (JSDL) and OGSA Basic Execution Service (BES) specifications Presentation will be implementation-agnostic –We focus here on technologies, not specific implementations

EU project: RIO31844-OMII-EUROPE Introduction to Job Submission Job Submission is one of the basic functionalities of any Grid system User submits a (computational) job to an appropriate Execution Service, and uses the same service to manage the job –Start/Stop execution –Pause/Resume execution –Query job status

EU project: RIO31844-OMII-EUROPE So, what is the problem? Different Grid Systems provide different interfaces for job submission and management Those interfaces provides mostly a common set of functionalities, but with slightly different semantics and/or syntax –Different names of job management operations –Different syntax used to describe a job –Different state models –...

EU project: RIO31844-OMII-EUROPE The OGSA/GGF Solution Job Submission Description Language (JSDL) –OGF Specification –XML-based notation for describing computational jobs and their resource requirements OGSA Basic Execution Service (BES) –OGSA Specification –WebService-based interface for job submission and management –Uses JSDL to describe the jobs

EU project: RIO31844-OMII-EUROPE JSDL Job Submission Description Language Most recent specification is Version 1.0 –Version 1.1 being worked out Specification (V1.0) available at JSDL Working Group

EU project: RIO31844-OMII-EUROPE JSDL: Motivations Many organizations accommodate a variety of job management systems –Each system has its own language for describing job submission requirements. –This makes interoperability between these systems for job management difficult Grid environments may involve the interaction of a number of different types of job management systems –The description of a job may be transformed by intermediaries –This description may be passed between systems or instantiated on resources matching the resource requirements for that job.

EU project: RIO31844-OMII-EUROPE Job Submission Scenario Job Manager Super-scheduler, or Broker, or … A Grid Information System Local Information System WS intermediary DRM Local Resource (e.g., super- computer) Another JSDL System J J J J J J JSDL document: WS Client J J Source: JSDL Specification, version 1.0

EU project: RIO31844-OMII-EUROPE JSDL Scope What JSDL is –JSDL is a language for describing the submission requirements of individual jobs. What JSDL is not –It does not attempt to address the entire lifecycle of a job or the relationship between individual jobs –It does not address job management (BES does that) –It does not consider Workflows –It does not mandate how resource matching should be performed –...

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * JobDefinition describes the job and its requirements This is the root of the JSDL document The (optional) id attribute can be specified

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * JobDescription describes the job and its requirements This is the actual content of the JSDL document

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * JobIdentification is a complex XML type and contains elements to identify the job The multiplicity of this element is zero or one

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * Application describes the application which is meant to be executed –This description is independent of the OS Application has a complex XML type The multiplicity of this element is zero or one –If the Application element is missing, the null job is assumed

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * Resources describes the resource requirements of the job This element supports only a limited set of core requirements Additional requirements can be defined through extensions –Even better, using GlueSchema?

EU project: RIO31844-OMII-EUROPE JSDL Document Structure ? * * DataStaging describes the data requirements of the job –Which files are used as input (stage in) –Which files are produced as output (stage out) This element can be repeated more than once if more than a single file is required/produced

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: JobIdentification ? * *

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: JobIdentification JobName (xsd:string) is the user- defined name for this job –Can be the same for multiple JSDL documents Description (xsd:string) is a textual description of the job JobAnnotation (xsd:string) contains informations which (unlike Description) MAY be used by the receiving system to process the job JobProject (xsd:string) specifies the project to which the job belongs. –The project could be used by accounting systems or access control systems ? * ?

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: Application ? * *

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: Application ApplicationName is a string that defines the name of the application and is used to identify the application independent of the location of its executable on a host or system ApplicationVersion is a string that defines the version of the application to execute Description is a string with a textual description of the application ? *

EU project: RIO31844-OMII-EUROPE Example Hello World This job prints the Hello World message OMII-EU uuid:090fe040e0 Echo 1.0 This application prints the Hello World message to the standard output

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: Resources Element ? * *

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: Resource Element Contains the resource requirements of the job. If this element is not present then the consuming system MAY choose any set of resources to execute the job. ? * ? * ?

EU project: RIO31844-OMII-EUROPE CandidateHosts elements Specifies the set of named hosts which may be selected for running the job. If this element is present then one or more hosts from the set MUST be chosen to run the job. A named host may be a single host (e.g., a machine name), a logical group of hosts (e.g., a named logical group or cluster), a virtual machine, and so on. + ? bach.example.com mozart.example.com beethoven.example.com

EU project: RIO31844-OMII-EUROPE FileSystem element Describes a filesystem that is required by the job. It may contain the location where the filesystem should be made available, the required amount of disk space and the type of the filesystem. The filesystem may be local to the resource (e.g., on a local disk), or may be remote (e.g., an NFS mount). ? *

EU project: RIO31844-OMII-EUROPE FileSystem element Textual description of the FS xsd:string that describes a local location that MUST be made available in the allocated resources for the job. xsd:string that describes a remote location that MUST be made available locally for the job. This is a range value (see later) that describes the required amount of disk space (in bytes) on the containing FileSystem element for the job. This is a token that describes the type of filesystem (see next slide) ? *

EU project: RIO31844-OMII-EUROPE FileSystemType enumeration

EU project: RIO31844-OMII-EUROPE FileSystem element examples The root of the main system filesystem. Not safe to assume that it is writeable. The actual root of the filesystem depends on the OS normal Ali’s home /home/ali normal

EU project: RIO31844-OMII-EUROPE Resource Element: ExclusiveExecution boolean that designates whether the job must have exclusive access to the resources allocated to it by the consuming system. –True—run exclusively on the resources –False—others can run at the same time ? * ? * ?

EU project: RIO31844-OMII-EUROPE Resource Element: OperatingSystem Complex type that defines the operating system required by the job. ? * ? * ? ? * ?

EU project: RIO31844-OMII-EUROPE Operating System Enumeration

EU project: RIO31844-OMII-EUROPE Resource Element: CPUArchitecture xsd:string specifying the CPU architecture required by the job in the execution environment. ? * ? * ? other cell

EU project: RIO31844-OMII-EUROPE CPUArchitecture Name Enumeration

EU project: RIO31844-OMII-EUROPE Range specification A range value is a complex type that allows the definition of exact values (with an optional “epsilon” argument), left-open or right-open intervals and ranges. All numbers given are of type xsd:double. UpperBoundedRanges and LowerBoundedRanges are limited to the upper or lower bound, respectively. Ranges may be “unlimited” to either negative or positive infinity

EU project: RIO31844-OMII-EUROPE Ranges pseudoschema xsd:double ? xsd:double ? xsd:double * xsd:double xsd:double *

EU project: RIO31844-OMII-EUROPE Example The pseudo-expression “5, , 7.0, [50.3,99.5), [100-” (which indicates a disjoint range consisting of the values 5, , 7.0, all values inclusive of from 50.3 up to, but not including, 99.5 and all values from 100 on up) can be encoded in a RangeValue as follows:

EU project: RIO31844-OMII-EUROPE Matching semantics A numeric value, N, matches a RangeValue, R, if and only if at least one of the following conditions is true: –R contains an UpperBoundedRange, U, with a false exclusiveBound attribute and where N ≤ U. –R contains an UpperBoundedRange, U, with a true exclusiveBound attribute and where N < U. –R contains a LowerBoundedRange, L, with a false exclusiveBound attribute and where N ≥ L. –R contains a LowerBoundedRange, L, with a true exclusiveBound attribute and where N > L. –R contains an Exact, E, with an epsilon attribute e, where E-e ≤ N ≤ E+e. –R contains a Range with LowerBound, L, and UpperBound, U, such that both of the following are true: L has a false exclusiveBound attribute and N ≥ L, or L has a true exclusiveBound attribute and N > L. U has a false exclusiveBound attribute and N ≤ U, or U has a true exclusiveBound attribute and N < U.

EU project: RIO31844-OMII-EUROPE Resource Element: Individual*, Total* Requirements on resources provided by each single node, or by the aggregate allocated resource pool ? * ? * ?

EU project: RIO31844-OMII-EUROPE Example ? Each resource has at least 4 CPUs at 1Ghz each All the resources must provide at most 16 CPUs

EU project: RIO31844-OMII-EUROPE Beware of unsatisfiable requirements! Here we request: –At most 8 resources... –...each one providing exactly 1 CPU... –...with a total CPU count of exactly ?

EU project: RIO31844-OMII-EUROPE JSDL Core Element Set: DataStaging Element ? * *

EU project: RIO31844-OMII-EUROPE Data Staging Elements Data staging defines the files that should be moved to the execution host (stage in) and the files that should be moved from the execution host (stage out). Files are staged in before the job starts executing. Files are staged out after the job terminates. –More complex file transfers (e.g., conditional transfers based on job termination status or pre-warming of grid- enabled file-systems) are out of scope of the JSDL specification. Permission and access control for the staged files should be handled by the implementation and is out of scope of the JSDL specification.

EU project: RIO31844-OMII-EUROPE DataStaging element Note: the ordering of the DataStaging elements in the JSDL document is not significant ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: FileName This element is a string specifying the local name of the file (or directory) on the execution host. The FileName MUST be a relative path ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: FileSystemName If the FileSystemName is specified then the FileName is relative to the specified FileSystem declaration referenced by the name. In this case there MUST also be a FileSystem element with the same name. ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: CreationFlag This element determines whether the file created on the local execution system can overwrite or append to an existing file. A typical value for this element, expected to be commonly supported, is “overwrite.” ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: DeleteOnTermination This is a boolean that determines whether the file should be deleted after the job terminates. –If true the file is deleted after the job terminates or after the file has been staged out. –Otherwise the file remains, subject to the persistency of the FileSystem it is on. If not present, behavior is unspecified ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: Source A Source element contains the location of the file or directory on the remote system. This file or directory MUST be staged in from the location specified by the (optional) URI before the job has started. ? ? *

EU project: RIO31844-OMII-EUROPE DataStaging element: Target A Target element contains the location of the file or directory on the remote system. This file or directory MUST be staged out to the location specified by the (optional) URI after the job has terminated. ? ? *

EU project: RIO31844-OMII-EUROPE Examples Staging in a file control.txt overwrite true

EU project: RIO31844-OMII-EUROPE Examples Staging in a file relative to a filesystem... control.txt HOME overwrite true

EU project: RIO31844-OMII-EUROPE Examples It is possible to define both Source and Target elements so that the file is first staged in before the job starts execution and staged out after the job finishes. state.txt overwrite true

EU project: RIO31844-OMII-EUROPE Examples Multiple stage out operations may be specified by using the same FileName (on the same FileSystem) in separate DataStaging elements. result.txt result.txt

EU project: RIO31844-OMII-EUROPE JSDL attribute extensions Every JSDL element allows for additional attributes, as many as needed (provided they have a namespace other than the normative JSDL one) Example: define an attribute called “order” that defines the order of staging in files <jsdl:JobDefinition xmlns=" xmlns:jsdl=" xmlns:o=" foo overwrite bar overwrite

EU project: RIO31844-OMII-EUROPE JSDL element extensions Similarly for elements h933fsolenri900wnmd90mm34...

EU project: RIO31844-OMII-EUROPE Normative extension: POSIX Application This normative extension defines a schema describing an application executed on a POSIX compliant system. The PosixApplication element describes a POSIX style Application and its requirements. –It contains Executable, Argument, Input, Output, Error, WorkingDirectory, Environment, various POSIX limits elements as well as User and Group names. –If it is present as a sub-element of the JSDL Application element it MUST appear only once.

EU project: RIO31844-OMII-EUROPE POSIX Application Normative Schema ? * ? * ?

EU project: RIO31844-OMII-EUROPE Final JSDL Example / 1 <jsdl:JobDefinition xmlns=" xmlns:jsdl=" xmlns:jsdl-posix=" xmlns:xsi=" My Gnuplot invocation... gnuplot /usr/local/bin/gnuplot control.txt input.dat output1.png

EU project: RIO31844-OMII-EUROPE Final JSDL Example / 2 control.txt overwrite true input.dat overwrite true output1.png overwrite true rsync://spoolmachine/userdir

EU project: RIO31844-OMII-EUROPE Basic Execution Service (BES) The BES specification defines Web Services interfaces for creating, monitoring, and controlling computational entities –UNIX or Windows processes, Web Services, or parallel programs—called activities—within a defined environment. Clients define activities using JSDL A BES implementation executes each activity that it accepts on an appropriate computational resource –the resource may be a single computer; a cluster; a Web Service hosting environment; or even another BES implementation.

EU project: RIO31844-OMII-EUROPE BES Basic Execution Service Latest version is Draft v35 –Right out of the public comment phase –Should become the final specification v1.0 Specification (draft v35) available at OGSA-BES Working Group

EU project: RIO31844-OMII-EUROPE BES The BES specification defines three Web Services port-types, aimed at different types of client: –the BES-Factory port-type allows clients to create, monitor, and control sets of activities, and to monitor BES attributes, such as a characterization of the resources the BES makes available to activities and the number of activities it currently has instantiated. This port-type is intended for use by ordinary clients. –the BES-Activity port-type allows clients to create, monitor, and control individual activities. This port-type is intended for use by ordinary clients. –the BES-Management port-type allows clients to monitor the details of and control the BES itself. This port-type is intended for use by system administrators.

EU project: RIO31844-OMII-EUROPE BES service port-types BES Service StopAcceptingNewActivities StartAcceptingNewActivities CreateActivity GetActivityStatuses TerminateActivities GetActivityDocuments GetFactoryAttributesDocument GetStatus (?) Terminate (?) GetDocument (?) GetActivityAttributesDocument (?) BES- Management port-type BES-Factory port-type BES- Activity port-type (optional)

EU project: RIO31844-OMII-EUROPE Basic State Model All activities MUST traverse the following set of states: –Pending: The service has created a record for an activity but not yet instantiated it on a suitable computational resource or enabled it to start execution on such a resource. –Running: The activity is executing on some computational resource. –Finished (a terminal state): The activity has terminated successfully. Successful termination implies that the activity exited of its own accord rather than due to some failure in the BES or of the computational resources on which the activity was running. A successfully terminating activity may nevertheless return an error code as its return value. –Terminated (a terminal state): The client – which might be some system administrator (and hence not necessarily the client who originated the request to create the activity) – has issued a TerminateActivity request. –Failed (a terminal state): The activity has failed due to some system error/failure event, such as failure of a computational resource that the activity was running on.

EU project: RIO31844-OMII-EUROPE Basic State Model PendingRunning Failed Terminated Finished TerminateActivity request Succesful termination of activity System error or failure event

EU project: RIO31844-OMII-EUROPE State Specialization It is possible to define “state profiles” providing detailed sub-state information to compliant clients Example: Pending Running: Stage-in Failed Terminated Finished Running: Execute Running: Stage-out

EU project: RIO31844-OMII-EUROPE State Specialization Rules A specialization can introduce sub-states only by replacing a state in the state model that it is specializing with a graph of sub-states and transitions among those sub-states. Transitions MUST maintain consistency with the basic state model (see example in next slide)

EU project: RIO31844-OMII-EUROPE Invalid State Specialization example Pending Running : S1 Failed Terminated Finished Runnin g: S2 NO!!

EU project: RIO31844-OMII-EUROPE BES-Management port-type StopAcceptingNewActivities –This operation is used to request that the BES stop accepting new activities. –No input or output data is required StartAcceptingNewActivities –This operation is used to request that the BES start accepting new activities. –No input or output data is required

EU project: RIO31844-OMII-EUROPE BES-Factory port-type The BES-Factory port type contains operations that support the creation and manipulation of activities controlled or maintained by the BES. It also contains an operation for retrieving attribute information about the BES itself which effectively provides an abstract characterization of a particular BES-Factory.

EU project: RIO31844-OMII-EUROPE BES-Factory operations: CreateActivity This operation is used to request the creation of a new activity Input –ActivityDocumentType ActivityDocument: An XML document describing a single activity that is to be created Output –CreateActivityResponseType Response: On success, Response contains an ActivityIdentifier (EPR) identifying the requested activity. Faults –NotAuthorizedFault –NotAcceptingNewActivities –UnsupportedFeatureFault –InvalidRequestMessageFault... *

EU project: RIO31844-OMII-EUROPE BES-Factory operations: GetActivityStatuses This operation queries the status of a set of activities Input –EPR[] ActivityIdentifier: A vector of one or more EPRs (obtained from previous CreateActivity operations), indicating activities from which we require status information. Output –GetActivityStatusResponseType[] Response: An XML document containing a vector of ActivityStatus elements (see below), one for each EPR provided in the input. Note that because many activities can be queried, this document may become large. Faults –UnknownActivityIdentifierFault: The specified EPR cannot be matched to an activity known to the container.

EU project: RIO31844-OMII-EUROPE BES-Factory operations: GetActivityStatuses This operation requests that a specified set of activities be terminated. The BES attempts to terminate each activity specified Input –EPR[] activities: A vector of one or more EPRs identifying the activities that are to be terminated. Output –TerminateActivityResponseType[] Response: A vector detailing the responses of the BES to the termination requests. The Terminated element is a boolean value: If true is returned, then the associated activity is now in the Terminated state. If false then the activity MAY eventually transition to the Terminated state. Faults –UnknownActivityIdentifierFault: The specified EPR cannot be matched to an activity known to the container.

EU project: RIO31844-OMII-EUROPE BES-Factory operations: GetActivityDocuments This method is used to get the JSDL associated to an activity Input –EPR[] ActivityIdentifiers: A vector of one or more EPRs identifying the activities for which ActivityDocuments are requested. Output –GetActivityDocumentResponseType [] Response. The output from this operation returns a vector of ActivityDocumentResponse elements. Faults –UnknownActivityIdentifierFault: The specified EPR cannot be matched to an activity known to the container. The EPR may reference an activity that has come to an end and been removed from the system.

EU project: RIO31844-OMII-EUROPE BES-Factory operations: GetFactoryAttributesDocument This operation is used to request a document containing the BES-Management attributes. Input –None Output –BESResourceAttributesDocumentType BESResourceAttributesDocument: An XML document containing the various attributes listed in the next slide Faults – None

EU project: RIO31844-OMII-EUROPE BES-Factory attributes Type: BES denotes a BES specific attribute, while BR denotes a basic resource attribute (i.e., one that applies only to underlying resources)

EU project: RIO31844-OMII-EUROPE BES-Activity port-type The BES-Activity port-type defines operations for monitoring and managing individual activities. –These operations are intended to be applied to EPRs returned by previous CreateActivity operations. No operations are defined for the BES-Activity port type. –These would be expected to include ‘single’ activity versions of the operations on the BES-Factory port type, i.e. GetStatus, Terminate, GetDocument, GetActivityAttributesDocument.

EU project: RIO31844-OMII-EUROPE Optional Extensions: Idempotent Execution Semantics If idempotent execution is required, the following element MUST be used to uniquely identify an activity to the BES using a client- generated identifier If present in an activity description, the BES MUST NOT execute the activity containing this identifier more than once –Should a BES receive a second CreateActivity request that includes the same identifier as a previously received request, the BES MUST NOT create the requested activity a second time if it already created the activity for the first request. wsa:AttributedURI

EU project: RIO31844-OMII-EUROPE Optional Extensions: Subscription to Notification Events A BES that allows its clients to subscribe for messages concerning activity state changes MUST do so using either the WS-Eventing or WS-Notification protocols. Compliant BESs implementing this extension MUST support the (optional) inclusion of one of the following subscription request elements as an extension to the ActivityDocument element in the input to the CreateActivity request: Then, one of the following associated subscription reference elements MUST be added as an extension to the ActivityDocument element returned in the response to CreateActivity OR

EU project: RIO31844-OMII-EUROPE Optional Extensions: Lifetime Management A BES that implements the WS-ResourceLifetime operation SHOULD use the following element to indicate the requestor’s suggestion for the initial setting of the termination time resource property (WS-ResourceLifetime) of the activity: xsd:dateTime

EU project: RIO31844-OMII-EUROPE Security considerations A BES will typically want to control who can invoke its various operations. There may also be a need for identity mapping between a grid credential and the BES’s authorization and authentication space. These considerations are outside the scope of this document. –The WS-I Basic Security Profile provides advice to implementers as to how to provide interoperable security solutions. This problem is being addressed in OMII JRA3/Security

EU project: RIO31844-OMII-EUROPE Conclusions The BES specification is succesful in addressing the most basic functionalities required by job submission and management, with at least two weak aspects Resource Modeling –Resource description and matching is extremely basic –Users cannot express complex requirements –Focus in OMII-EU/JRA2 GlueSchema Security –No autentication/authorization mechanism is mandated –Nobody is willing to implement BES without any security –Focus in OMII-EU/JRA3

EU project: RIO31844-OMII-EUROPE Thank you! This tutorial is available at