Basics Globus Toolkit™ Developer Tutorial The Globus Project™ Argonne National Laboratory USC Information Sciences Institute Copyright (c) 2002 University of Chicago and The University of Southern California. All Rights Reserved. This presentation is licensed for use under the terms of the Globus Toolkit Public License. See for the full text of this license.

June 28, Globus Toolkit™ Developer Tutorial: Basics Setting the Foundation l Strategies for the Globus Toolkit –Layered architecture, protocols & APIs l Naming conventions –Functions, variables, structures, etc. l Installation tree structure –Where things get installed l Source tree structure –Where things come from

June 28, Globus Toolkit™ Developer Tutorial: Basics Globus Approach l Software toolkit addressing key technical areas –Offer a modular “bag of technologies” –Enable incremental development of grid-enabled tools and applications –Define and standardize grid protocols and APIs (Our software development supports this goal.) l Focus is on inter-domain issues, not clustering –Supports collaborative resource use spanning multiple organizations –Integrates cleanly with intra-domain services –Creates a “collective” service layer

June 28, Globus Toolkit™ Developer Tutorial: Basics Globus Approach l Focus on architecture issues –Provide implementations of grid protocols and APIs as basic infrastructure –Use to construct high-level, domain-specific solutions l Design principles –Keep participation cost low –Enable local control –Support for adaptation Diverse global services Core Globus services Local OS A p p l i c a t i o n s

June 28, Globus Toolkit™ Developer Tutorial: Basics Layered Architecture Applications Core Services Metacomputing Directory Service GRAM Globus Security Interface Replica Catalog GASS GridFTP Local Services LSF CondorMPI NQEPBS TCP AIXLinux UDP High-level Services and Tools CactusCondor-GMPINimrod/Gglobusrun PUNCH Grid Status I/O Solaris DRM

June 28, Globus Toolkit™ Developer Tutorial: Basics Why C (Why Not C++) l Portability –Most C++ compilers do not support complete ANSI C++ standard –This is getting better, but currently C++ is still a headache for highly portable code l Linkage –Linking with Fortran can be difficult l Religion –C++ gives too much rope to hang yourself –Good C coding practices are usually adequate

June 28, Globus Toolkit™ Developer Tutorial: Basics Naming Conventions l Globus Toolkit is comprised of a set of modules, each with a unique name l A set of related modules may be part of a larger package l In order to avoid name collisions, all names begin with globus or grid

June 28, Globus Toolkit™ Developer Tutorial: Basics Function Naming Conventions l All function names and other exported symbols follow the form: globus_ _ _ l The is omitted if the module is not part of a larger package of related APIs l Examples: –globus_gram_client_job_submit(…) –globus_nexus_send_rsr(…)

June 28, Globus Toolkit™ Developer Tutorial: Basics Function Return Values l Most functions return either an int or globus_result_t value l Older APIs return value: int –Like errno –GLOBUS_SUCCESS (0) indicates that the function was successful –Any other return value is an error code –Error codes defined in module headers –Many modules still use this approach

June 28, Globus Toolkit™ Developer Tutorial: Basics Function Return Values l Newer APIs return value: globus_result_t –New approach - like exception objects –GLOBUS_SUCCESS still indicates success –Otherwise can be expanded to a globus_error_t “object” >Error type hierarchy with single inheritance >Can easily match against types (similar to “catch”) >Can be used to return extended information about the error –New 1.1 modules use this approach –Will gradually be pushed through other modules

June 28, Globus Toolkit™ Developer Tutorial: Basics Library Naming Conventions l Library names follow the same naming convention as the functions in that library: libglobus_ _.a l Currently shared libraries are not supported l Examples: –libglobus_gram_client.a –libglobus_nexus.a

June 28, Globus Toolkit™ Developer Tutorial: Basics Program Naming Conventions l All program names start with globus or grid l Dash (-) is used as a word delimiter l Many Globus programs are not directly related to a single module –May use multiple modules to implement higher level functionality >e.g. globusrun, globus-job-submit –May not be directly related to any module >e.g. globus-sh-exec –But module name used where appropriate >e.g. globus-gass-server

June 28, Globus Toolkit™ Developer Tutorial: Basics Install Tree Structure l GLOBUS_LOCATION environment variable points to the install tree, containing: –bin: Programs for your path –sbin: System binaries –include, lib: Development headers & libraries –libexec: Programs/scripts used by libraries –etc: Configuration files –man: Man pages –var: Log files, etc. –share: Other shared data files

June 28, Globus Toolkit™ Developer Tutorial: Basics Globus Toolkit Modules common (used by everything) module activation, threads, libc wrappers, callbacks, errors, etc. io gss gss_assist ldap Uses gram_client duroc_control duroc_runtime gram_myjob gass_transfer nexus gass_cache gass dc hbm_client hbm_datacollector rsl utp mp

June 28, Globus Toolkit™ Developer Tutorial: Basics New Globus Toolkit Data Grid Modules Library Program Legend globus-url-copy Replica Programs Custom Servers globus_gass_copy globus_ftp_client globus_ftp_control globus_commonGSI (security) globus_ioOpenLDAP client globus_replica_catalog globus_replica_manager Custom Clients globus_gass_transfer globus_gass Already exist

June 28, Globus Toolkit™ Developer Tutorial: Basics Security Modules l gssapi –Grid Security Infrastructure –GSS-API implementation –SSL authentication (public key) l gss_assist –Simplified interface to GSS-API

June 28, Globus Toolkit™ Developer Tutorial: Basics Resource Management Modules l RSL parser l GRAM –client API: simple client –myjob API: parallel job bootstrapping –jobmanager API: server infrastructure –gatekeeper: root security & dispatch svc l DUROC –control API: client –runtime API: job startup barriers, etc –bootstrap API: parallel job bootstrapping

June 28, Globus Toolkit™ Developer Tutorial: Basics GridFTP Modules l GridFTP control –Low level handling of control and data channels l GridFTP client –High level client >Get, put, 3 rd party transfer, cd, mkdir, rmdir, etc… –Rich plug-in support for reliability and monitoring

June 28, Globus Toolkit™ Developer Tutorial: Basics GASS (File Access) Modules l GASS url copy –url-to-url file copy, for file:, ftp: urls –Wrapped by globus-url-copy program l GASS file –open(), close(): works on files and URLs l GASS transfer –Client & server handling of get/put requests l GASS server_ez –Simple, embeddable server l GASS cache –Cache for remote files (prefetch etc.)

June 28, Globus Toolkit™ Developer Tutorial: Basics Information Services Modules l MDS tools –client programs –server tools –built on OpenLDAP

June 28, Globus Toolkit™ Developer Tutorial: Basics Miscellaneous Modules l Common –Fundamental system and portability API l IO –Fundamental I/O API l Data conversion l UTP timing package l Miscellaneous programs and scripts l Programs that use multiple APIs

June 28, Globus Toolkit™ Developer Tutorial: Basics Communication Modules l MP –Simple wrapper around message passing libraries l Duct –Used by GRAM and DUROC for startup and bootstrapping l Nexus (deprecated) –Multi-method communication library –Being deprecated

June 28, Globus Toolkit™ Developer Tutorial: Basics Summary l Hopefully you should be starting to get your bearing within the Globus Toolkit l Next each module will be explored, via: –Summary of functionality –Walk-through with sample tools –Walk-through with APIs –Discussion of interesting issues