1. Computational Grids and the Globus Project Ian Foster Argonne National Laboratory The University of Chicago Globus Co-PI: Carl Kesselman, USC/ISI

2. Ian Foster ARGONNE  CHICAGO Credits l Thanks to the Globus teams at ANL, USC/ISI, NCSA, and elsewhere –Steve Tuecke, Gregor von Laszewski, Steve Fitzgerald, Randy Butler, Brian Toonen, and many others l Research supported by DARPA, DOE, NSF, NASA; equipment from Cisco Systems

3. Ian Foster ARGONNE  CHICAGO Overview l Computational grids –Why, what, where l Globus project and its research activities –Focus on resource management l U.S. Grid deployment activities –NASA, DOE, DISCOM l Application overviews –Focus on on-line instrumentation l Summary

4. Ian Foster ARGONNE  CHICAGO Context “Imagine a computing & information infrastructure designed to power research & education by providing pervasive access to all significant information resources” or internationally... nationally...

5. Ian Foster ARGONNE  CHICAGO Grid Application Examples Teleimmersion/distance collaboration Record-setting distributed supercomputing TransAtlantic remote visualization/steering Parameter studies with deadline scheduling Online analysis of instrument data

6. Ian Foster ARGONNE  CHICAGO “The Grid” l Pervasive, reliable, secure, high- performance access to leading-edge computational, data, instrument, display & human resources l New problem-solving methodologies based on this coupling of geographically distributed resources l Advanced network services that enable of the above (“middleware”)

7. Ian Foster ARGONNE  CHICAGO What are “Grids”? l Grid infrastructures support applications that –Involve coordinated use of end-system resources, often in different admin domains –Often require high performance –Require enhanced services beyond those provided by today’s Internet l Major production infrastructures & testbeds being deployed at this moment l Grid R&D sponsored by DARPA Quorum, NSF PACIs, NASA IPG, NGI (DOE in particular)

8. Ian Foster ARGONNE  CHICAGO Creating a Usable Grid : Grid Services (“Middleware”) l Standard grid services that –Provide uniform, high-level access to a wide range of resources (including networks) –Address interdomain issues of security, policy, etc. –Permit application-level management and monitoring of end-to-end performance l Middleware-level and higher-level APIs and tools targeted at application programmers –Map between application and Grid

9. Ian Foster ARGONNE  CHICAGO Unifying Concept: An Integrated Grid Architecture Archives, networks, computers, display devices, etc.; associated local services Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Grid Fabric Grid Services Appln Toolkits Applns... … a rich variety of applications... Remote viz toolkit Remote comp. toolkit Remote data toolkit Remote sensors toolkit Async. collab. toolkit

10. Ian Foster ARGONNE  CHICAGO The Globus Project: Argonne, USC/ISI, NCSA, Aerospace, NASA Ames, LBNL, others l Basic research in grid-related technologies –Resource management, security, adaptation, etc. l Development of Globus toolkit –Core services for grid-enabled tools & applns l Construction of large grid testbed: GUSTO –Largest grid testbed in terms of sites & apps l Application experiments –Tele-immersion, distributed computing, etc.

11. Ian Foster ARGONNE  CHICAGO Remote viz toolkit Remote comp. toolkit The Globus Project and the IGA Archives, networks, computers, display devices, etc.; associated local services Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Remote data toolkit Grid Fabric Grid Services Appln Toolkits Applns... Remote sensors toolkit Async. collab. toolkit … rich variety of applications... Research Security, resource mgmt, instrumentation, adaptivity policy, etc., etc. Globus Toolkit Authentication, resource management, information, communication, fault detection, data access, other services as and when available Deployment Subsets of Globus toolkit technology deployed as production infrastructure Supported partners: e.g., tele- immersion, parameter studies, debuggers, desktop tools, PSEs, couplers, etc., etc., etc. Collaborative efforts: e.g., data grid, remote viz, commodity grid, Message Passing Interface, etc., etc., etc. Examples: SF-Express, X-ray source data, astrophysics, crystallography, collaborative design, remote quantum chem. Ongoing: HEP & climate data, combustion corridor, remote Allegra, distributed AMR, CorridorOne, HotPage,...

12. Ian Foster ARGONNE  CHICAGO GUSTO Testbed Map

13. Ian Foster ARGONNE  CHICAGO Globus Grid Services l The Globus toolkit provides a range of basic Grid services –Security, information, fault detection, communication, resource management,... l These services are simple and orthogonal –Can be used independently, mix and match –Programming model independent l For each there are well-defined APIs l Standards are used extensively –E.g., LDAP, GSS-API, X.509,...

14. Ian Foster ARGONNE  CHICAGO Grid Security Infrastructure l Single-sign on, run anywhere [if authorized] l PKI, X.509 certificates l Identity/credential mapping at each resource l Allows programs to act as user for limited period: delegation of rights

15. Ian Foster ARGONNE  CHICAGO Single sign-on via “globus-id” User User Proxy GlobusCredential Site 1 Kerberos GRAM Process GSI Ticket Site 2 Public Key GRAM GSI Certificate Process Authenticated interprocess communication CREDENTIAL GSS-API: multiple low-level mechanisms Mutual user-resource authentication Mapping to local ids Assignment of credentials to “user proxies” Technical Details: Grid Security Services Provide Single Sign-On

16. Ian Foster ARGONNE  CHICAGO An Example of Globus Services at Work: Resource Management Architecture “10 GFlops, EOS data, 20 Mb/sec -- for 20 mins” Metacomputing Directory Service GRAM Resource Broker Info service: location + selection Globus Resource Allocation Managers GRAM Fork LSF EASYLL Condor etc. “What computers?” “What speed?” “When available?” “50 processors + storage from 10:20 to 10:40 pm” “20 Mb/sec”

17. Ian Foster ARGONNE  CHICAGO Globus and the National Grid l Selected “Grid Services” are being migrated into the infrastructure –Grid information service –Grid security infrastructure –Grid resource management services l Simultaneously these and other Globus services are being applied to develop –Grid-enabled tools –Grid-enabled applications l An ongoing iterative refinement process

18. Ian Foster ARGONNE  CHICAGO Globus Project Grid Testbed Partnerships Include... NASA’s Information Power Grid The Alliance National Technology Grid Wisc. NWU UIC UChicago UIUC Other Potential University Participants ANL ESnet Abilene AADS Switch supporting MREN PVCs DiffServ Measurement Servers DiffServ Policy Tools DiffServ Enabled Router EMERGE MREN/ESnet Testbed

19. Ian Foster ARGONNE  CHICAGO tomographic reconstruction real-time collection wide-area dissemination desktop & VR clients with shared controls Advanced Photon Source Case Study: Online Instrumentation archival storage DOE X-ray source grand challenge: ANL, USC/ISI, NIST, U.Chicago

20. Ian Foster ARGONNE  CHICAGO Current Technology Focus Areas l Advanced resource management techniques –GARA: Globus Arch. for Resv. & Allocation l High-end data-intensive applications –“Data Grid” l Interfaces to commodity technologies –CoG-Kit: Commodity Grid Toolkits l Distance visualization –NOVA: Network Optimized Visualization Arch. With supporting work on info/instr., policy, accounting, authentication/authorization, etc.

21. Ian Foster ARGONNE  CHICAGO

22. Ian Foster ARGONNE  CHICAGO GARA and Differentiated Services Server Diffserv Resource Manager Client GARA API Diffserv Resource Manager

23. Ian Foster ARGONNE  CHICAGO Scheduling Bulk Transfer and High-Priority Transfers

24. Ian Foster ARGONNE  CHICAGO The Grid Forum http://www.gridforum.org l IETF-like community forum for discussion & definition of Grid infrastructure l First two meetings (June 16-18, Oct 18-20) attracted 150 people l 9 working groups established in security, information infrastructure, resource management, accounting, etc. l Next mtg: San Diego Feb/Mar 2000 l See also European Grid Forum –www.egrid.org

25. Ian Foster ARGONNE  CHICAGO Summary: Globus Contributions l Architecture that addresses inter-domain issues l Toolkit model for building grid-enabled tools and applications l Information-rich approach to adaptivity and high performance l New technologies for security, resource management, etc. l Construction of production environments and testbeds allowing realistic evaluation

26. Ian Foster ARGONNE  CHICAGO Summary l Grids require Grid services that make resources accessible and usable and Grid toolkits for application development l The Globus project is building essential services and partnering with tool developers l Widespread adoption by groups building production infrastructure l Significant application success stories in a range of problem classes

27. Ian Foster ARGONNE  CHICAGO For More Information... l Globus: www.globus.org l Grid Forum: www.gridforum.org l Grid book “The Grid: Blueprint for a Future Computing Infrastructure,” I. Foster & C. Kesselman (Eds), Morgan-Kaufmann, 1999 http://www.mkp.com/grids