Porting applications to Globus Toolkit 3.0 and designing an OGSI-based architecture Pawel Plaszczak and Rich Wellner Perfringo Grid Solutions Tutorial materials:

Porting to GT3 & designing OGSI - Perfringo Grid Solutions About the authors From Foster’s Globus team in Argonne Co-founded Perfringo Grid Solutions Focus: industry level Grid applications Currently: NeesGRID - distributed earthquake simulations

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Prerequisities Necessary: distributed systems & software design, purpose of Grid Useful: Java, Web services, XML, Grid services (will be briefly explained) Not necessary: coding experience We will speak on decision making and design. No hands-on programming

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Plan Part I: Grid services - why and when? Part II: OGSI and OGSA technology review Part III: Use case: NTCP Part IV: Porting to Grid services and GT3

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Plan of Part I: Grid services - why and when? Overview: Globus Alliance and Toolkit Overview: GT2 vs GT3, OGSA Why go Grid? Why comply with OGSA? Why port to GT3?

Porting to GT3 & designing OGSI - Perfringo Grid Solutions The Globus Alliance Globus Project ™, since 1996  Ian Foster (Argonne National Lab),  Carl Kesselman (University of Southern California’s Information Science Institute) Develop protocols, middleware and tools for Grid computing Globus Alliance, since Sept 2003 International scope  University of Edinburgh’s EPCC  Swedish Center for Parallel Computers (PDC)  Advisory council of Academic Affiliates from Asia- Pacific, Europe, US

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Globus Toolkit GT2 (2.4 released in 2002): reference implementation of Grid fabric protocols  GRAM for job submissions  MDS for resource discovery  GridFTP for data transfer  GSI security GT3 (3.0 released July 2003): redesign  OGSI based  Grid services, built on SOAP and XML

Porting to GT3 & designing OGSI - Perfringo Grid Solutions OGSI and OGSA OGSI = Open Grid Service Infrastructure  Specs from GGF OGSI working group  Defines what makes a Grid service  Based on Web service  Naming, life cycle, state, notification  portTypes definitions, WSDL 1.2 draft OGSA = Open Grid Service Architecture  Specs from GGF OGSA working group  Defines a list of fundamental Grid services, and how they cooperate  Work in progress

Porting to GT3 & designing OGSI - Perfringo Grid Solutions What’s new in GT3.0 Emphasis on OGSI compliant Grid services OGSA implemented Uniform SOAP/XML communication layer Focus on Java implementation Old GT2 services redesigned Backward compatibility where possible

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Meaning of GT3 in the community Most commonly referred project Traditionally “de facto standard” Acknowledged leadership by academia and industry (IBM,…) BSD style license allows for commercial usage However, it is only a reference implementation. Now standards = GGF GT undergoes constant changes With business entering grid, commercial implementations may soon catch up

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Alternatives to GT3 Protocol level interoperability:  “Grid compliant” >= “implements OGSI” Other OGSI implementations  OGSI.NET (U.Virginia)  pyGlobus (LBNL) .NET (U.Edinburgh)  PERL (U. Manchester)  UNICORE (Fujitsu) Commercial OGSI compliant products by:  Avaki, Platform, Data Synapse, … Web service alternative: Grid App Framework

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Part I contd. Overview: Globus Alliance and Toolkit Overview: GT2 vs GT3, OGSA Why? Why go Grid? Why comply with OGSA? Why port to GT3? upgrading from GT2

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Why go Grid? Hot subject Try it, experience it to learn the potential Will enable true ubiquitous computing in future Today, proven in some areas: intraGrids But still long way to World Wide Grid State of art techniques, tools are difficult Short term goals? Use another technology Does your system have Grid characteristics?  Distributed users, large scale and heterogeneous resources, across domains

Porting to GT3 & designing OGSI - Perfringo Grid Solutions OGSI architecture: why and when? Commonly accepted Grid standard  But there are alternatives: WS-GAF, GT2,… Extends Service-Oriented Architecture  Modular, scalable, maintainable, interoperable Extensions: use only when needed Uniform communication layer Be compatible and inter-operable with other grid systems (that’s what grids are about!)

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting to GT3: why and when? Implementing OGSI is plenty of work! Piggyback on GT3 for OGSI/A compliance (watch commercial products) Most serious middleware to consider Language bindings: traditionally Unix/C, now Java leads But: unstable, new versions bring changes Traditionally, for hard core hackers Traditionally, documentation was lacking  recent signs of improvement, also IBM redbook

Porting to GT3 & designing OGSI - Perfringo Grid Solutions GT3 stability GT3 took only 1.5 year to develop. It will change many times! Anticipated or considered changes:  Refactor GridFTP control channel into XML, DAI  Security: add CASS  GRAM: ongoing improvements  OGSA-C hosting environment to come OGSA specs is draft OGSI specs are completed (v. 1.0)

Porting to GT3 & designing OGSI - Perfringo Grid Solutions GT3.2 Expected January 2004 Features:  CAS (Community Authorization Service)  Index Service reimplemented  GridFTP server reimplemented  GridFTP java client: connection reuse, MLST  GRAM performance improvements  CRL support (certificate revocation list)  several bug fixes in core

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Java CoG Kit status Before: Java client for GT2 Now integrated with GT3 Provides low level API for GT3 services: security, GridFTP Experimental features to come soon (not part of GT3):  Workflow management, compliant with GT2 and GT3  Comes with example Jetspeed portlets

Porting to GT3 & designing OGSI - Perfringo Grid Solutions roadmap Part I: Grid services - why and when? Part II: OGSI and OGSA technology review Part III: Use case: NTCP Part IV: Porting to Grid services and GT3

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Plan of Part II: OGSI and OGSA technology review Web service XML and SOAP Service Oriented Architecture Grid service (OGSI) OGSA How does GT3 implement OGSA Examples

Porting to GT3 & designing OGSI - Perfringo Grid Solutions What is a Web service Not much to do with HTML Interface to a well- explained, self- contained function (module, component) Usu. small grained Defined in WSDL Communication: XML and SOAP OnlineStore Web service queryItems() orderItem() Web service client SOAP/XML

Porting to GT3 & designing OGSI - Perfringo Grid Solutions More on Web services Specs  W3 Consortium IBM, Microsoftwww.w3.org,  OASIS (WS security) For developers: “Building Web services with Java” by Steve Graham et al For managers: “Web services and Service- Oriented Architectures” by Doug Barry (the savvy manager’s guide) Stay tuned: IBM developerWorks articles  ibm.com/developerworks/webservices/

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Service-Oriented Architecture System composed of several components exhibiting WS interfaces Discovery through a well-known directory, such as UDDI External services use internal ones Modular, scalable, maintainable, interoperable

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Service-Oriented Architecture in practice OnlineStore WS queryItems() orderItem() Web service client StockMgr WS query() ship() OrderMgr WS order() bill() query order ship warehouse

Porting to GT3 & designing OGSI - Perfringo Grid Solutions What is a Grid service Defined by OGSI (GGF working group) Is a Web service with additions, which are:  Name (handle GSH, reference GSR)  Lifetime management (factories, persistent and transient services)  State (Service Data)  Notification as well as querying WSDL 1.2 draft (gwsdl: namespace) Definitions of portTypes

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Grid services in practice: notifications and service data OnlineStore GS query() subscribe() Grid service client Subscribe: “tell me when you have it on stock” Notify: “it just arrived.” Notification source Notification sink StockItem SDE Service data element

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Grid services in practice: naming and life cycle OnlineStore Persistent GS queryItems() StockMgr Persistent GS getItem() StockItemAF45X Transient GS Grid service client Item SDEs: dimension, price 3. create Factory 2. Get reference 1. query 4. query

Porting to GT3 & designing OGSI - Perfringo Grid Solutions What is OGSA Open Grid Service Architecture, being defined by GGF OGSA working group In ubiquitous Grid platform, there is common need for some basic services OGSA defines this set of base services OGSA builds on OGSI

Porting to GT3 & designing OGSI - Perfringo Grid Solutions How does GT3 implement OGSA Java implementation, C to follow Container approach: host small-grained services Hosting environment takes care of basic GS functions GT3 provides core and base services, users to implement their own Transient and persistent GS API Tools to generate Java GWSDL Axis engine used for SOAP handling Run with Tomcat or standalone

Porting to GT3 & designing OGSI - Perfringo Grid Solutions roadmap Part I: Grid services - why and when? Part II: OGSI and OGSA technology review Part III: Use case: NTCP Part IV: Porting to Grid services and GT3

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Plan of part III: NTCP NeesGRID project overview NTCP requirements & high level design Choosing Grid services for NTCP subsystem Choosing Grid services for Matlab subsystem Not choosing Grid services for ShoreWestern subsystem Deployment On-going work

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NeesGRID Make collective use of geographically distributed experiment facilities, heterogeneous hardware Users dislocated from facilities Applying Grid to earthquake science 3 year project involving several US institutes Distributed simulation of a building in an earthquake Scale: models costs millions of $ and several months to build. Some experiments are not repeatable (model used once)

Porting to GT3 & designing OGSI - Perfringo Grid Solutions The MOST experiment NCSA Computational Model m1m1 f1f1 UIUC Experimental Model f1f1 m1m1 f2f2 f2f2 U. Colorado Experimental Model Slide courtesy of Bill Spencer, UIUC

Porting to GT3 & designing OGSI - Perfringo Grid Solutions MOST requirements Distributed simulation of a building in an earthquake 3 experimental sites Hybrid experiment: 2 physical and 1 computational simulations Results at one time-step used to calculate inputs at the next time-step Pseudodynamics: order of 10 ms per time-step Successfully run on 30th July 2003

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP: NEESGrid Teleoperation Control Protocol Vital component of NeesGRID and MOST Telecontrol system, used to coordinate the experiment with commands Output data collected with other NEESGrid systems like NSDS

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP system architecture Coordinator (NTCP client) NTCP server NTCP server NTCP server Backend ShoreWestern Backend Matlab sim. Backend Matlab + XPC Local protocol Local protocol Local protocol NTCP NTCP subsystem local subsystems NCSAU ColoradoUIUC

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP protocol Coordinator follows the script with recorded earthquake data Coordinator orders servers to simulate one time- step at a time (pseudodynamics) Servers can refuse if parameters out of range Core commands  Propose() negotiates an action  Execute() starts an action  Cancel() issued when negotiation failed  Other: get/setParameter, open/closeSession…

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Sequence diagram (simplified) coordinator NTCP Server A openSession NTCP Server B propose execute results (accepted) (started) openSession propose execution 1 time step

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP subsystem: choosing Grid services and GT3 Long term goals System characteristics: distributed, heterogeneous resources across administrative domains Interoperability, manageability, scalability Risk: GT3 released early July, MOST scheduled end of July

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP server simplified design

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP server as a Grid service Front end: Permanent Grid service  Operates on real hardware; only admin can restart methods  Open/closeSession, propose, execute, cancel, get/setParameter, getControlPoint Service data: Transaction state Notification: Execution results Back end: LocalControlPlugin interface for implementing adapters to the local non-OGSI technologies

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Sequence diagram revisited in Grid service terms Coordinator (GS client) NTCP Grid service Subscribe for transaction status Propose transaction Execute transaction results (accepted) (started) execution 1 time step Transaction results notification Transaction Service Data create update

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Matlab subsystem: plugin and backend Coordinator (NTCP client) NTCP server NTCP server NTCP server Backend ShoreWestern Backend Matlab sim. Backend Matlab + XPC Local protocol Local protocol NTCP Local protocol

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Matlab subsystem: characteristics Command flow: 1. Coordinator 2. NTCP server frontend 3. NTCP server Matlab control plugin (Mplugin) 4. Backend machine running Matlab simulation Matlab single threaded, not event driven Information pull model enforced Matlab has Java interface, fully compliant with Java 1.3. Can use Java class library, third party jar files, multiple threads

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Matlab subsystem: choosing Grid services Choose network communication technology between Mplugin and backend  TCP/IP, RMI, Corba, Web services… Although no Grid characteristics… why not Grid services?  Pulling scenario -> “server” on the same machine as the NTCP server  OGSA container is already there  Ockham razor: one technology instead of many

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Typical Service-Oriented Arch compared to NTCP + Matlab

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Client, server and backend in Grid service model Methods:  hasCommand, pullCommand, pushResult No service data, notification Could as well be a Web service Reversed information flow, as compared to typical Service-Oriented Both NTCP and commandBuffer services are persistent; must match at start-up

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Testing the NTCP and Matlab subsystem Junit and TestServer from GT3 3 test suites, 4 test units Some units implement same interfaces 3 simulators, some deployed inside gar

Porting to GT3 & designing OGSI - Perfringo Grid Solutions ShoreWestern subsystem Coordinator (NTCP client) NTCP server NTCP server NTCP server Backend ShoreWestern Backend Matlab sim. Backend Matlab + XPC Local protocol Local protocol NTCP Local protocol

Porting to GT3 & designing OGSI - Perfringo Grid Solutions ShoreWestern subsystem: plugin and backend ShoreWestern hardware driver to operate physical experiment facility Push communication OK ShoreWestern already speaks a proprietary protocol based on TCP/IP, equivalent in functionality to NTCP

Porting to GT3 & designing OGSI - Perfringo Grid Solutions TCP/IP based prototype of NTCP reused for ShoreWestern Coordinator Backend sim. ShoreWestern Backend sim. Matlab Backend sim. Matlab TCP/IP NTCP server ShoreWestern backend TCP/IP Grid service Front end ShoreWestern ctrl plugin Same protocol

Porting to GT3 & designing OGSI - Perfringo Grid Solutions ShoreWestern subsystem: *not* choosing Grid services Backend has server role Reuse existing TCP/IP protocol Ockham razor (again!) ShoreWestern plugin = adapter to the old protocol

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Complete NTCP system for the MOST experiment Coordinator (NTCP client) Backend e.g. ShoreWestern Backend e.g. Matlab sim. Backend e.g. Matlab + XPC TCP/IP OGSI OGSA container ShoreWestern ctrl plugin OGSA container NTCP Grid service OGSA container CmdBuffer Grid service CmdBuffer Grid service NTCP Grid service NTCP Grid service OGSI

Porting to GT3 & designing OGSI - Perfringo Grid Solutions MOST experiment deployment First GT3-based application deployed in production 3 remote sites:  NCSA, UIUC, University of Colorado 5.5 hours and almost 1500 steps Monitored by several engineers and scientists Remote observers through advanced Grid technology: multiple life video, life data, chat, all via Web interface Output data to be analyzed by earthquake scientists

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP: ongoing work Very high requirements  Reliability  Safety  Error checking  Security Involved: Information Science Institute, Argonne National Lab, Perfringo Grid Solutions

Porting to GT3 & designing OGSI - Perfringo Grid Solutions NTCP Acknowledgements Bill Spencer, UIUC (NeesGRID PI) Laura Pearlman, ISI (NTCP lead) Erik Johnson, USC (Matlab engineering) Ravi Madduri, Mike D’Arcy (Globus team developers)

Porting to GT3 & designing OGSI - Perfringo Grid Solutions roadmap Part I: Grid services - why and when? Part II: OGSI and OGSA technology review Part III: Use case: NTCP Part IV: Porting to Grid services and GT3

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Plan of Part IV: Porting to Grid services and GT3 All depends on your starting point Choose the best approach High or low abstraction level? Study early experience with working systems Stay tuned

Porting to GT3 & designing OGSI - Perfringo Grid Solutions All depends on your starting point Already have a GT2 based system?  Porting GT2 -> GT3 Already have Web services based system?  Porting Web services -> Grid services Have a system in other technology? Starting from scratch?

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting GT2 -> GT3: GridFTP GT2 GridFTP is still there. Consider using Java client API from CoG (org.globus.ftp aka jftp) For 3rd party, consider 3- tier architecture with reliable file transfer (RFT) Uncertain long term protocol stability GridFTP server GridFTP server jftp client API RFT data Grid service Client API

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting GT2 -> GT3: GRAM GT2 GRAM now refactored into XML Server side: new XML front end, old backend submission scripts New XML clients: Java or C Client side API-level backward compatible: old apps only need recompiling

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting GT2 -> GT3: MDS GT3 Information Services, aka MDS3  fundamental change in technology Old GRIS: now Service Data Old GIIS: now Index Service Old LDAP/LDIF: now SOAP/XML RFC2252 MDS schema: now SDD in GWSDL Porting old data providers  Substantial amount of work  Generally, need an adapter Grid service

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting GT2 -> GT3: the rest GT2 GSI security still there New: GT3 allows creating services not based on GridFTP, GRAM and MDS. Think about redesigning your architecture

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting Web services to Grid services: theory Grid service *is* a Web service Naming: need to introduce handle and reference Life cycle  Web services are like persistent Grid services Service data - not necessary  Get/set methods can now be implemented as SDE Notification - not necessary Discovery:  GT3 uses Index Service as directory  Web services sometimes use UDDI

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Porting Web services to Grid services: your options Implement from scratch  Need to implement (at least) handle and reference management  Can retain your old Web service code Implement with GT3  Get complete OGSI feature set for granted  Hidden porting work  Is the WS already in Axis? Handler based approach  WSDL -> GWSDL  Different procedure to deploy in container  GSI security (builds on top of WS-security)  Dependencies, versioning… Use non-GT3 OGSI middleware

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Experience of NEReSC CGM team: workflow service Courtesy of Saras Parastatidis and Paul Watson, Web ServiceGrid Service

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Courtesy of Saras Parastatidis and Paul Watson, Direct port of servlet proved too difficult  Dependencies on mysql, OpenJMS, Tomcat  Versioning nightmare Architecture converted to 3-tier New Grid service adapter constructed Adapter forwards to old Web service Client adaptation costed time Experience of NEReSC CGM team: notification service

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Experience of NEReSC CGM team: conclusions Courtesy of Saras Parastatidis and Paul Watson, Web and Grid services can coexist Adapter pattern is one solution Alternative to OGSI: Grid Application Framework based on Web Services Specifications and Practices document (

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Choose the best approach Writing Grid services from scratch Redesigning legacy system in Grid service architecture Refactoring communication layer Façade pattern approach

Porting to GT3 & designing OGSI - Perfringo Grid Solutions High or low abstraction level Adapt to lowest necessary abstraction  Web services?  OGSI? (what features needed?)  OGSA? (what services needed?) Reuse highest possible abstraction  track GGF progress  track Globus plans (announcements)  Ongoing work: OGSA-DAI, OGSA-DQP, workflows, OGSA itself, WS-Agreement

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Data access & integration Global Grid Forum Data Access and Integration Services (DAIS) working group Reference implementation OGSA-DAI by Edinburgh Parallel Computing Center (EPCC) “middleware to assist with access and integration of data from separate data sources via the grid” First stage ended July 2003; software available:

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Hints Grid services can be small grained  One idea for the next GridFTP server is to represent each file as a Grid service  Existence of a handle does not imply existence of a service! It should apear as a service, but implementation can be custom Represent abstractions as services  Agreement, transaction Method or SDE?  Get/set functionality can be faster to implement as methods, but more elegant as service data

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Experience with working systems NeesGRID Quertermaster NEReSC Core Grid Middleware other

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Stay tuned Quarterly GGF: gridforum.org Globus World, January , San Francisco GridToday, May 2004 to focus on commercial applications Perfringo events  Training events, tutorials, newsletter: Questions and comments?

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Your feedback Slides available for download  Please send feedback to  What did you learn?  What did you expect?  What could be added / improved?

Porting to GT3 & designing OGSI - Perfringo Grid Solutions Thanks! Laura Pearlman (NeesGRID, Globus) Bill Spencer (NeesGRID, UIUC) Savas Parastatidis (NEReSC CGM) Lisa Childers, Jarek Gawor, Gregor von Laszewski (Globus team)