COMP3019 Coursework: Introduction to GridSAM Steve Crouch School of Electronics and Computer Science

A Reminder…  For materials (inc. slides, software, materials, handout): – 9/ 9/  Coursework deadline: March 27, 2014 at 4pm

GridSAM Job Submission and Monitoring

GridSAM Overview  What is GridSAM? – A Job Submission and Monitoring Web Service – Allows compute resources to be accessible securely over the internet – Uses Job Submission Description Language (JSDL) to describe requirements for jobs

GridSAM Overview  What is GridSAM to the resource owners? – A Web Service to expose heterogeneous execution resources uniformly  Single machine through Forking or SSH  Condor  Portable Batch Scheduler (PBS)  Load Sharing Facility (LSF)  Sun Grid Engine  Globus – Acts as a client to these resources

GridSAM Overview  What is GridSAM to end-users? – A set of end-user tools and client-side APIs to interact with GridSAM Web Services  Submit and start jobs  Monitor jobs  Terminate jobs  File transfer  Client-side submission scripting  Client-side Java API

GridSAM Server and Client  Server comprised of: – Web Services Container  Tomcat/Axis - Web Services provider/container  Optional WS-Security - handles authentication (contains X509 certificate for security)  Can be run over HTTP or HTTPS – GridSAM Java web service  Client comprised of: – Client security ‘container’.  Contains Configuration for security (including X509 certificate security credentials)  Used by client applications to interact with services on OMII server – GridSAM Java client

Web Service Definition  “A service is the logical manifestation of some physical or logical resources (databases, programs, devices, humans, etc) and/or some application logic that is exposed to the network”  Service interaction is facilitated by message exchanges

Web Services (WS) I  XML: Platform neutral mechanism to describe data  SOAP: Mechanism to describe message exchange – Simple Object Access Protocol  Not simple and nothing to do with Objects! – Service Oriented Access Protocol  Re-engineering of acronym to fit current use!  WSDL: Defines the service interface – Web Services Definition Language

Web Services (WS) II  Services have to reside in a supporting environment: – Called: hosting environment or container  e.g. Axis on Tomcat – Marshals requests into and response out of the service – Service can discover local configuration parameters – Provides a standard infrastructure for service developers  Processing incoming requests & outgoing responses – Called: Message handlers – Manipulates elements of the message header  Primarily the SOAP header – Handlers can be applied to message traffic into or out of the whole container or a specific service

Architecture WS-Security AXIS TOMCAT GridSAM … Client … …

GridSAM – Publications & Enabled Activities

GridSAM Architecture  A staged event-driven architecture – Submission pipeline is constructed as a network of stages connected by event queues – Each stage performs a specific action upon incoming events Example Pipeline: Condor

Scenario: Condor Pool GSIFTP FTP WEBDAV HTTP …

Example using File Staging  Objectives: submit simple job with data input and output requirements and monitor progress OMII GridSAM Client OMII GridSAM Server submit JSDL monitor OMII GridSAM FTP Server 1 output file 2 input files

JSDL Example  /gridsam/data/examples/remotecat-staging.jsdl: … bin/concat dir2/subdir1/file2.txt stdout.txt stderr.txt dir1/file1.txt …

JSDL Example bin/concat overwrite ftp://ftp.do:55521/concat.sh dir1/file1.txt overwrite ftp://ftp.do:55521/input1.txt dir2/subdir1/file2.txt overwrite ftp://ftp.do:55521/input2.txt stdout.txt overwrite true ftp://ftp.do:55521/stdout.txt