Basic Grid Projects - Globus Sathish Vadhiyar Sources/Credits: Project web pages, publications available at Globus site. Some of the figures were also taken from the same

Globus Open source toolkit used for building Grids Software for Security (GSI) Security (GSI) Information infrastructure (MDS) Information infrastructure (MDS) Resource management (GRAM, job manager, gatekeeper) Resource management (GRAM, job manager, gatekeeper) Data management (GridFTP, DataGrid) Data management (GridFTP, DataGrid) Communication (Nexus) Communication (Nexus) Fault detection, and Fault detection, and Portability Portability Now moving to web services - OGSA

Timeline I-WAY experiment – 1994 Formal beginning st version – 1997 Version 1.0 – – – latest Show GT2 history powerpoint

GT4 Planned architecture

Grid Security Infrastructure (GSI) Supports security across organizations. Single sign-on Delegation of credentials Digital signatures based on public key cryptography for verification of messages

Globus/Grid Security Infrastructure (GSI) based on PKI GSI is: PKI (CAs and Certificates) SSL/ TLS Proxies and Delegation PKI for credentials SSL for Authentication And message protection Proxies and delegation (GSI Extensions) for secure single Sign-on PKI: Public Key Infrastructure, SSH: Secure Socket Layer TLS: Transport Level Security Credits: Globus course material

Verification of messages / digital certificates Message Hash(message) Encyrpted hash Encypted hash + message Hash1 = hash(Message) Hash2 = decrypt hash If Hash1 = Hash2 ?

GSI Every resource identified by a certificate. Certificate provided and signed by CA. Certificate = resource identity + public key of resource + certificate authority + digital signature of CA Uses SSL for mutual authentication Parties trust CA’s – possess CA’s public keys

Mutual Authentication I want to communicate. This is my certificate AB CA Did CA sign the certificate or is the certificate tempered? Verify digital signature OK. CA signed the certificate. Are you really A or did you steal the certificate from A? Send a random message

Authentication with Proxy and delegation Encrypted file for storing private keys. Needs passphrase Proxy and delegation - More convenience and less security Also for dynamic delegation for third-party services and dynamic entities Owner signs proxy certificate Proxy’s private key are stored in unencrypted files since proxies are for short durations Chain of trust is established

Mutual Authentication with Proxy Proxy’s certificate. A’s certificate A’s proxyB First validate proxy’s certificate and then owner’s certificate

GSS API GSI implemented using GSS- API GSS API provides both transport and mechanism independence. Provides functions for obtaining credentials, performing authentication, signing messages and encrypting messages GSI – X.509 public key certification, public key infrastructure, SSL protocol, X.509 proxy certificates

X.509 Proxy Certificates To allow users to: Create identities for new entities dynamically and light-weight Create identities for new entities dynamically and light-weight Delegate privileges to those entities dynamically Delegate privileges to those entities dynamically Perform single sign-on Perform single sign-on Proxy certificate Subject name (identity) – scoped by the subject name of the issuer – subject name of the issuer + RDN (Relative Distinguished Name) + serial number Subject name (identity) – scoped by the subject name of the issuer – subject name of the issuer + RDN (Relative Distinguished Name) + serial number Public key – different from subject’s public key Public key – different from subject’s public key PCI – Proxy Certificate Information – policy method identifier + policy field PCI – Proxy Certificate Information – policy method identifier + policy field

Proxies

Single sign-on and Proxies

Delegation over Network

Globus Resource Allocation and Management

Globus Resource Management Architecture For remote job submission and resource management Designed to address following problems in metacomputing: Site autonomy (resource managers) Site autonomy (resource managers) Heterogeneous substrate (resource managers) Heterogeneous substrate (resource managers) Co-allocation (co-allocators) Co-allocation (co-allocators) Online control (RSL and resource brokers) Online control (RSL and resource brokers)

Resource Management Architecture

DUROC Dynamically-Updated Request Online Coallocator coallocator is used to coordinate transactions with each of the RMs and bring up the distributed pieces of the job

RSL spec. E.g.: Multi-request

Local resource Management - GRAM GRAM simplifies the use of remote systems by providing a single standard interface for requesting and using remote system resources for the execution of "jobs". 3 main functions: Processes RSL specifications Processes RSL specifications Enables resource monitoring and management Enables resource monitoring and management Periodically updates MDS Periodically updates MDS

GRAM Provides interfaces to local job scheduling mechanisms Provides mechanisms to map GSI identities to local user accounts Processes the requests for resources for remote application execution, allocates the required resources, and manages the active jobs. Also returns updated information regarding the capabilities and availability of the computing resources to the Metacomputing Directory Service (MDS). Provides an API for submitting and canceling a job request, as well as checking the status of a submitted job.

GRAM A Gatekeeper runs on the remote host Creates jobmanager for the job Gatekeeper: mutually authenticates with the client, mutually authenticates with the client, maps the requestor to a local user, maps the requestor to a local user, starts a job manager on the local host as the local user, and starts a job manager on the local host as the local user, and passes the allocation arguments to the newly created job manager. passes the allocation arguments to the newly created job manager.Jobmanager: Common component Common component Machine-specific component Machine-specific component

GRAM

Advanced reservation and co- allocation - GARA

Globus References / sources / credits A Resource Management Architecture for Metacomputing Systems. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, S. Tuecke. Proc. IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing, pg , Describes the resource management architecture implemented as part of the Globus system. A Distributed Resource Management Architecture that Supports Advance Reservations and Co-Allocation. I. Foster, C. Kesselman, C. Lee, R. Lindell, K. Nahrstedt, A. Roy. Intl Workshop on Quality of Service, Describes the new Globus Architecture for Reservation and Allocation, which integrates CPU and network QoS.

Globus References / sources / credits A Security Architecture for Computational Grids. I. Foster, C. Kesselman, G. Tsudik, S. Tuecke. Proc. 5th ACM Conference on Computer and Communications Security Conference, pp , Describes techniques for authentication in wide area computing environments. proxy-cert-final.pdf proxy-cert-final.pdf A National-Scale Authentication Infrastructure. R. Butler, D. Engert, I. Foster, C. Kesselman, S. Tuecke, J. Volmer, V. Welch. IEEE Computer, 33(12):60-66, Describes our experience designing, developing, and deploying the Grid Security Infrastructure.

JUNK !

GRAM The most common use (and the best supported use) of GRAM is remote job submission and control. This is typically used to support distributed computing applications For remote job submission and resource management

GRAM RSL attributes The specifications are written by the user in the Resource Specification Language (RSL), and is processed by GRAM as part of the job request. (directory=value) (executable=value) (arguments=value [value] [value]...) (jobType=single|multiple|mpi|condor) (count=value) (hostCount=value) (two_phase= ) (restart= )

DUROC RSL attributes LabelresourceManagerContactsubjobCommsTypesubjobStartType

Example (executable = a.out) (directory = /home/nobody ) (arguments = arg1 "arg 2") (count = 1)

WS GRAM A set of OGSI compliant services that provide remote job execution (Master) Managed Job Factory Service (MJFS) (Master) Managed Job Factory Service (MJFS) Managed Job Service (MJS) Managed Job Service (MJS) File Stream Factory Service (FSFS) File Stream Factory Service (FSFS) File Stream Service (FSS) File Stream Service (FSS) Resource Specification Language (RSL-2) schema is used to communicate job requirements Remote jobs run under local users account Client to service credential delegation is done user to user, *not* through a third party

RSL-2 Example GNS = “ GNS = “

Managed Job (Factory) Service Defines an OGSI/GWSDL interface for submitting, monitoring and controlling a job Defines an OGSI/GWSDL interface for submitting, monitoring and controlling a job MJS uses the File Stream Factory Service to manage the job’s stdout and stderr file streaming MJS uses the File Stream Factory Service to manage the job’s stdout and stderr file streaming MJS exposes the stdout and stderr File Stream Factory Grid Service Handles (GSH) in Service Data Element MJS exposes the stdout and stderr File Stream Factory Grid Service Handles (GSH) in Service Data Element

The MJS instances can monitor jobs in two ways: Resource Information Provider Service (RIPS) Resource Information Provider Service (RIPS) A specialized notification service A specialized notification service Maintains job information from the scheduler Maintains job information from the scheduler Scheduler info provider outputs queue and job data in XML Scheduler info provider outputs queue and job data in XML Poll the scheduler directly Only option for FORK Only option for FORK MJS to Resource Interface: can support custom- schedulers through well defined templates

WS GRAM Architecture

OGSA and WS MDS Index service Standard interfaces for Grid services in the form of WSDL porttypes GridService porttype for querying and updating GridService data MDX index service consists of following interfaces: Factory – for creating a grid service instance and return GSH Factory – for creating a grid service instance and return GSH GSH – to refer to a grid service instance GSH – to refer to a grid service instance GSR – describes how a client can communicate with a grid service GSR – describes how a client can communicate with a grid service Query – query language support Query – query language support Registry - Supports discovery by returning the GSHs of a set of Grid services Registry - Supports discovery by returning the GSHs of a set of Grid services Notification – for registering interest in a service Notification – for registering interest in a service