Grid Computing Grid computing is all about achieving greater performance and throughput by pooling resources on a local, national, or international level. Issues: –Determination of resources to be allocated to the VO. –Determination of the resources to be allocated to a given application. –Communication between heterogeneous resources from different locations. –How to parallelize an application to make it suitable for execution on a Grid. –How to secure the shared resources.

More Formal Definition A grid is a system that: –Coordinates resource sharing in a de-centralized manner (i.e., different VOs). –Uses standard, open, general purpose protocols and interfaces. –Delivers non-trivial qualities of service. Guaranteed bandwidth for application. Guaranteed CPU cycles. Guaranteed latency.

The Sentence That Almost Killed the Golden Goose “Hence, our Grid architecture is first and foremost a protocol architecture, with protocols defining the basic mechanisms by which VO users and resources negotiate …” –Led to the Grid revolt of 2003 and displaced the OGSI with a Web services architecture.

Layered Architecture of a Grid Fabric: Resources to be shared in Grid systems. Connectivity: All of the protocols that allow resources to communicate. –For example: TCP/IP, HTTP, XML, SOAP. Resource: Services and protocols that enable the management of individual resources: –Information Protocols: Allows us to access information about the resource. –Management Protocols: Manage aspects of the resources to a certain degree. E.g., monitoring the way resources are being used.

Collective: Services and protocols for managing multiple resources. –Take a collection of resources and make them work together to solve a common task. –Examples include: Resource registries. Allocation and scheduling services. Monitoring Services. Data Management services.

Applications– Fundamental user of Grid Services.

Summary Grid architecture: –Fabric –Connectivity –Resource –Collective –Applications

Open Grid Services Architecture Developed by the Global Grid Forum to define a common, standard, and open architectures for Grid- based applications. –Provides a standard approach to all services on the Grid. VO Management Service. Resource discovery and management service: Job management service. Security services. Data management services.

Built on top of and extends the Web Services architecture, protocols, and interfaces.

Web Services: W3C Working Group Definition: A Web service is a software system identified by a URI [RFC 2396], whose public interfaces and bindings are defined and described using XML. Its definition can be discovered by other software systems. These systems may then interact with the Web service in a manner prescribed by its definition, using XML based messages conveyed by Internet protocols.[RFC 2396]

Figure A stateless Web Service invocation

Figure A stateful Web Service invocation

Homework up for grabs! The following topics will be presented in next weeks class: –Web services architecture. –XML and SOAP. –Web services resource framework. –Overview of Globus TK4.

Summary Grid architecture: –Fabric –Connectivity –Resource –Collective –Applications

Recall that most web services are stateless (do not save information between successive invocations. OGSA specification requires stateful services. This resulted in the Web Services Resource Framework (WSRF) that specifies how to make web services stateful (among other things). Joint effort between Web Services and Grid communities (OASIS).

Web Services A client/server system Concept similar to Remote Procedure Call (RPC), Remote Method Invocation (RMI), only applied over HTTP Specifications for/of making Web Services Stateful. Defines common standards, and open architecture for grid-based applications.

Relationship between OGSA, GT4, WSRF, and Web Services

Web Services Allow creation of client/server applications. Platform and language independent based on XML. Most use HTTP for transporting messages (firewalls generally do not attack HTTP traffic. Lend themselves naturally to build loosely coupled distributed systems.

Disadvantages: –Transmitting data in XML is not too snappy. –Still evolving.

Invocation Discover Web Service –Universal Description, Discovery and Integration (UDDI) protocol also being standardized by OASIS. Receive addresses of requested service from discovery service. Ask service to describe itself (i.e., how to invoke it). Service responds with Web Service Description Language (WSDL). Invoke service using a SOAP message. Responds with SOAP message.

Web Services

Web Services Architecture The Web Services Architecture is specified and standardized by the World Wide Web Consortium, the same organization responsible for XML, HTML, CSS, etc.

Web Services Addressing Addressed via URI (Uniform Resource Identifiers). –Basically same as URLs. –Difference is that URLs are for humans, URIs are for machines. Services generally invoked by program stubs. –Generate SOAP requests and interpret SOAP responses. –Stubs generated by some tool (generally) based on WSDL received from server.

1.Client will call the client stub. The client stub will turn this 'local invocation' into a proper SOAP request. This is often called the marshaling or serializing process. 2.The SOAP request is sent over a network using the HTTP protocol. The server receives the SOAP requests and hands it to the server stub. The server stub will convert the SOAP request into something the service implementation can understand (this is usually called unmarshaling or deserializing) 3.Once the SOAP request has been deserialized, the server stub invokes the service implementation, which then carries out the work it has been asked to do.

4.The result of the requested operation is handed to the server stub, which will turn it into a SOAP response. 5.The SOAP response is sent over a network using the HTTP protocol. The client stub receives the SOAP response and turns it into something the client application can understand. 6.Finally the application receives the result of the Web Service invocation and uses it.

1.Web Service: Code that provides the service. 2.SOAP Engine: Handles SOAP requests and responses (e.g., Apache Axis). 3.Application Server: Living space for required applications (e.g., the SOAP engine). Apache Jakarta Tomcat is an application server. 4.HTTP Server.

Web Services Resource Framework Web Service + Stateful Resource = WS-Resource WS-Addressing specifies an endpoint reference. –Each resource has a unique key included in address.

0xF56EA72DD

Stateful resource has three characteristics –It is composed by state data defined in XML format. –It has a life cycle. –It can be manipulated by one or more Web Services.

Four Required Characteristics of WS- Resources Atomicity: Stateful resource updates within a transactional unit are made in an all-or-nothing fashion. Consistency: Stateful resources should always be in a consistent state even after failures. Isolation: Updates to stateful resources should be isolated within a given transactional work unit. Durability: Provides the permanence of stateful resource updates made under the transactional unit of work.

Four Required Characteristics of WS- Resources Atomicity: Stateful resource updates within a transactional unit are made in an all-or-nothing fashion. Consistency: Stateful resources should always be in a consistent state even after failures. Isolation: Updates to stateful resources should be isolated within a given transactional work unit. Durability: Provides the permanence of stateful resource updates made under the transactional unit of work. This sounds pretty similar to the mutual exclusion problem (recall the dining falcons problem). –Can semaphores provide all of these characteristics?

Resource Properties Provide a view on the current state of the resource. Service Data Values: –Properties of the service, operation results, runtime information, etc –MetaData Information regarding the service data values. –E.g., last modified, last use of data, etc. –Information needed to manage the state. Similar to metadata but refers to resource as a whole.

WSRF Specification WS-ResourceProperties –Defines how the state of a WS-resource can be manipulated by Web Service. –A resource property maps to an individual component of the resource stat. –Methods to retrieve, change, and delete visible properties of a WS- Resource. WS-ResourceLifetime –Mechanisms to manage the lifecycle of our resource. WS-ServiceGroup –Related to groups of Web services or groups of WS-resources. Add new service to the group, find a service in a group, remove the service from the group, etc.

WSRF Specification WS-Resource Lifecycle –Mechanisms to manage the lifecycle of our resource. –Creation: Usually created by a resource factory. –Destruction: The means by which a stateful resource is destroyed and system resources recovered. –Resource Identifier: Must have at least one resource identifier. WS-ServiceGroup –Related to groups of Web services or groups of WS-resources. Add new service to the group, find a service in a group, remove the service from the group, etc.

WSRF Specification WS-ServiceGroup –Organize collections of WS-Resources to build registries or services that can perform collective operations. –Defines means for managing heterogeneous collections of Web Services. –Uses memberships, rules, and constraints to define groups. –A group is a collection of members that meets some constraints defined using resource properties.

WSRF Specification WS-BaseFaults –Standard way of representing faults in execution of service. WS-Notification. Publish/subscribe mechanism. Defines interfaces that clients can use to subscribe to topics of interest, and receive notifications asynchronously. –Example: Changes to resource properties.

WS-Notification Members: –WS- BaseNotification –Defines mechanisms that allow a subscriber to register receive notifications from a producer –Ws-Topics –Hierarchical organization of notification messages that subscribers can understand and subscribe to. –WS-BrokeredNotification: Defines interface to Notification Broker that manages subscriptions in the systems.

Why Move to Stateful Resource Model from OGSI Too bulky: Not a clean separation to support incremental adoption. Object-oriented framework not deemed suitable by large segments of the Grid/Web Services communities. The Web Services framework was already heavily used, with standard protocols. No need to create an additional set of protocols. Web services are open-source.

Web Service Description Recall use an Interface Description language (IDL) called WSDL to formally describe a service, what is does, how it is accessed, etc.

Homework Write a web service called “calculator”. –Provide both the client and the server. –Develop an information service that can be contacted to find this service. Can use IP address and port number. Define how the client will interact with information server to retrieve URL of calculator service. –Send message to calculator asking it what operations it can perform and parameters to operations. You define how this interaction takes place. –Invoke service.