Grid Technology Review Guy Rixon AstroGrid consortium meeting 3-4 November 2003 A scavenger’s guide

2 Consortium meeting November 2003: grid-technology review Grid tech: what can we use? We are consumers of grid tech. We like: Things that make our web services better. Things that make our development faster. Complete services to re-use. Standards that get support from industry. Using other people’s equipment without having to install software there. We don’t like: “Products” that don’t work. Tech that doesn’t work with industry standards. Tech that slows down development. Computer-science curios. Grid for Grid’s sake.

3 Consortium meeting November 2003: grid-technology review SOAP bubbles IETF GSS, X.509 GGF OGSI, OGSA GSI? OASIS WS-* Globus GRAM, MDS, GSI? Fujitsu OGSI OGSA-DAI IVOA Lots of standards bodies (blue), implementors (yellow), adoptors (white); lots of contradiction, duplication and overlap. Interoperation: not easy, or automatic. Whole thing has the cohesion and robustness of bath foam. W3C SOAP, WSDL

4 Consortium meeting November 2003: grid-technology review Convergence! Saved! Grid services  OGSI spec. OGSA is OGSI-compliant services. OGSI services are web services They use XML They use WSDL Can bind them to SOAP and HTTP They use IETF-approved ID certificates They even use some WS-* standards! ?

5 Consortium meeting November 2003: grid-technology review But…divergence too! OGSI/grid services are incompatible with important higher- order web-service protocols like WS-Transaction. Corporates support both, but not in same framework. Grid apps and web-service tools are in separate spaces.

6 Consortium meeting November 2003: grid-technology review And…non-conformance! Some important standards are only drafts. E.g. WSDL 1.2, WS-SecureConversation Toolkits and meta-standards are built on the drafts. Much grid stuff extends standards. E.g GWSDL is not quite WSDL 1.2. Not good: Currently only interoperable by chance Not future-proof

7 Consortium meeting November 2003: grid-technology review Service-grid implementations OGSI: Globus Toolkit 3 (Java, Axis, Tomcat; “difficult”) Fujitsu/Unicore (Java, GLUE; beta) University of Virginia (C#,.NET; said to be good) OGSA: Mainly vapourware, but: Some bits in GT3 OGSA-DAI (looks good) Other unpublished?

8 Consortium meeting November 2003: grid-technology review Security infrastructure Globus/GGF has Grid Security Infrastructure (GSI) GSI does public-key authentication Uses X.509 certificates: single sign-on Handles delegation Exploits XML-Signature, WS-Security, WS- SecureConversation Operates at SOAP level; transport neutral. We should use GSI (unless we can do better). We must interoperate with GSI if we do any grid. Must use X.509 certificates from GSI-friendly PKI. Don’t need to use GSI code in our web services.

9 Consortium meeting November 2003: grid-technology review Data grid: files Data grid  MySpace Data grid should: Store data with managed leases (we can do this) Catalogue stored data (we can do this) Move files between sites maintaining privacy (need GridFTP for this) Maintain replicas of files (Globus/EDG replica service?) Data grid shown in pink Service grid shown in blue P2P sharing

10 Consortium meeting November 2003: grid-technology review Data grid: DB tables Want to include relational data in data grid (i.e. in MySpace)  writable databases accessible from net. No complete, ready-made solutions… But OGSA-DAI is a possible basis. An example of a grid service we can use. See Grid Data Warehouse project.

11 Consortium meeting November 2003: grid-technology review Storage Resource Broker? Storage Resource Broker (SRB): Stores files across NAS. Unifies all storage into a data grid. Makes remote files transparently available. Manages replicas and file migration. But: Doesn’t manage DB tables. Doesn’t scale well across collaborations (one partner owns/controls the SRB data grid). Requires apps to use a custom API to access files.  Excellent for “intragrids”, not suitable for public VObs infrastructure.

12 Consortium meeting November 2003: grid-technology review Grid tech in VObs? 4+1 options WW W W 1.No Grid, no way! GG G G 2. Grid throughout. WW G W 3. Grid services as leaf nodes. WW G W 4. Leaf nodes + pervasive GSI + pervasive GridFTP. 5. Grid only “within” web services WW W W G Preferred by GWS-WG of IVOA (options 4 & 5 are not mutually exclusive)

13 Consortium meeting November 2003: grid-technology review Grid at arm’s length Exploit free resources on e-Science grid. Processors, storage, DBs, tape silos, etc. Use standard (OGSAish) grid-services. Don’t put astro conventions and software into their nodes. Connect to grid services via web-service gateways Run gateways on AstroGrid machines. Translate to/from astro semantics at gateways. True grid nodes don’t appear in VObs registries. OGSA-DAI for Grid Data Warehouse is the first example.

14 Consortium meeting November 2003: grid-technology review Recommendations Use OGSI only in non-critical services. Use pre-built (OGSA) grid services, but only in non-critical places. Build our own data grid using GridFTP. Build “intragrids” hidden behind web services. Stay compatible with OGSI/OGSA by using GSI-compatible X.509 certificates. Embrace industry standards; wait for grid to converge.