The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman USC/Information Sciences Institute

Outline and Goals l Define what is (and is not) a Grid in terms of a layered protocol architecture l Understand the role of protocols, services, APIs and tools in the deployment and use of Grids l Illustrate how a layered protocol architecture works with specific examples l Propose a view of where the Grid Forum can play a particularly important role

What is a Grid? l Grid language has been driven by genesis from metacomputing, but… l In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations l Focus on how to enable, maintain and control the sharing of resources to achieve a common goal

Some Useful Definitions l Network Protocol u A formal description of message formats and a set of rules for exchange of messages u Rules define sequences of message exchange, and potentially resulting behavior l Protocol may define state-change in endpoint l Network Enabled Services u Defines a set of capabilities u Protocol defines interaction with service l All services require protocols, although not all protocols are to services

More definitions l Resource u Entity that is to be shared u Provides some capabilities, that can be accessed via interface (API) or protocol l Application Programmer Interface (API) l Software Development Kit (SDK) u Package that enables application development, consisting of one or more APIs, and programming tools

Protocols Make the Grid l Protocols vs. APIs u Protocols enable interoperability u APIs enable portability l Sharing is about interoperability, so … u Grid architecture should be about protocols l By extension, a major role for the Grid Forum is to define protocols where standards do not exist

Grid Services Architecture: Previous Perspective Grid-enabled archives, networks, computers, display devices, etc.; associated local services Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Grid Fabric Grid Services Appln Toolkits Applns... … a rich variety of applications... Remote viz toolkit Remote comp. toolkit Remote data toolkit Remote sensors toolkit Async. collab. toolkit

Characteristics of Grid Services Architecture l Identifies separation of concerns u Isolates Grids from languages and specific programming environments l Makes provisions for generic and application specific functionality l Protocols not explicit in architecture u Fails to make clear distinction between language, service, and networking issues

Grid Services Architecture v2.0: Layered Grid Protocol Architecture Connectivity Fabric Resource Grid Application User

Important Points l Being Grid-enabled requires speaking appropriate protocols u Protocol only requirement, not reachability u Protocols can be used to bridge local resources or “local Grids” u Talk about Intergrid (analog to Internet) l Built on Internet protocols l Independent of language and implementation u Focus on interaction over network l Services exist at each level

Grid Services Architecture v2.0: Protocols, Services, and Interfaces Languages/Frameworks Fabric Layer Applications Local Access APIs and protocols Grid Service APIs and SDKs Grid Services Grid Service Protocols Resource APIs and SDKs Resource Services Resource Service Protocols User Service Protocols User Service APIs and SDKs User Services Connectivity APIs Connectivity Protocols

The Integrated Grid Architecture in Practice l We now proceed to illustrate the general concepts outlined above by reference to specific examples u First, we analyze some extant systems in terms of this architecture u Then, we review example protocols, services, APIs, SDKs in each layer l Apology: We will necessarily need to refer to some specific protocols and systems

Example: A User Portal l This to be filled in showing the different services protocols APIs etc. used at each layer

Example: Network Weather Service l ditto

Example: A Data Grid l ditto

Layer 1: Fabric l Local access to logical resource u May be real component, e.g. CPU, software module, filesystem u May be logical component, e.g. Condor pool l Protocol or API mediated l Fabric elements include u SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions

Layer 2: Connectivity Protocols l Two classes of connectivity protocols underlie all other components l Internet communication u Application, transport and internet layer protocols u I.e., transport, routing, DNS, etc. l Security u Authentication and delegation u Discussed below

Connectivity Layer Element: Grid Security Protocols l Protocols u TLS with delegation l Services u K5ssl, Globus Authorization Service l APIs u GSS-API, GAA, SASL, gss_assist l SDKs u GlobusIO

Layer 3: Resource Protocols l Resource management l Storage system access l Network quality of service l Data transport l Resource information

Resource Management l Protocols u GRAM+GARA (on HTTP) l Resource services u Gatekeeper, JobManager, SlotManager l APIs and SDKs u GRAM API, Java CoG Kit Client, DUROC

Data Transport l Protocols u Grid FTP, LDAP for replica catalog l Services u FTP, LDAP replica catalog l APIs and SDKs u GridFTP client library, copy URL API, replica catalog access, replica selection

Resource Information l Protocol u LDAP V3, Registration/Discovery protocol l Service u GRIS l APIs & SDKs u C API; JNDI, PerlLDAP, ….

Layer 4: Grid Protocols l Grid Information Index Services u LDAP and Service registration protocol, … u GIIS service u LDAP APIs and specialized information API l Co-allocation and brokering u GRAM (HTTP+RSL) u DUROC service u DUROC client API, end-to-end reservation API

Grid Protocols (cont) l Online authentication, authorization services u HTTP u MyProxy, Group policy servers u MyProxy API, GAA API, l Many others (e.g.): u Resource discovery (Matchmaker) u Fault recovery

Layer 5: User Protocols l In general, there are many of these, they tend to be one off and not standardized l Examples: u Portal toolkits (e.g., Hotpage) u Netsolve u Cactus framework

Role of Standards l Standard APIs: code reuse, portability u Certainly a good thing! l Standard protocols enable resource sharing via interoperability of infrastructure u The fundamental problem l Note that Grid Forum is already doing this! u Scheduling: remote access protocol u Data: Grid FTP u Security: Grid Security protocols u Information: Standard info representations

Summary l Grids are about sharing, not making big MPI jobs l Well-defined protocol architecture is essential to understanding and progress u Provides a framework for figuring out where the pieces fit l Grid Forum can play an important role by helping to fill in missing pieces u But note that protocol design is not for the nonspecialist (or fainthearted)!