The Anatomy of the Grid Enabling Scalable Virtual Organizations Ian Foster Mathematics & Computer Science Division Argonne National Laboratory and Dept. of Computer Science The University of Chicago 2 nd US-Hungarian Workshop on Cluster and Grid Computing, February 6, 2002 David S. Angulo Dept. of Computer Science The University of Chicago and Mathematics & Computer Science Division Argonne National Laboratory

University of Chicago Abstract l "Grid" computing has emerged as an important new field l Distinguished from conventional distributed computing by focus on –Large-scale resource sharing –Innovative applications –High-performance orientation (in some cases) l In this talk, this new field is defined –First, "Grid problem“ reviewed, which Ian Foster defines as >flexible, secure, coordinated resource sharing >among dynamic collections of individuals, institutions, and resources (referred to as virtual organizations) –Challenges in such settings >authentication >authorization >resource access >resource discovery >and other challenges

University of Chicago Abstract (Cont.) l This class of problem addressed by Grid technologies l Major Grid projects worldwide reviewed –Describe their contributions to the realization of this architecture. l Future Architecture Overview –Open Grid Services Architecture is presented

University of Chicago Partial Acknowledgements l Globus Toolkit TM –R&D involves >many fine scientists & engineers at ANL/UofC, USC/ISI, and elsewhere (see –Led by >Ian Argonne/UofC >Carl USC/ISI l Open Grid Services Architecture work performed by –Ian Foster, Globus Argonne/UofC –Carl Kesselman, Globus USC/ISI –Steve Tuecke, Globus Toolkit –Jeff Nick, Steve Graham, Jeff IBM l Strong collaborations with many outstanding EU, UK, US Grid projects l Support from DOE, NASA, NSF, Microsoft, IBM

University of Chicago Grid Computing

University of Chicago The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

University of Chicago Why Grids? l A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour l 1,000 physicists worldwide pool resources for petaflop analyses of petabytes of data l Civil engineers collaborate to design, execute, & analyze shake table experiments l Climate scientists visualize, annotate, & analyze terabyte simulation datasets l A home user invokes architectural design functions at an application service provider –An application service provider purchases cycles from compute cycle providers

University of Chicago Elements of the Problem l Resource sharing –Computers, storage, sensors, networks, … –Sharing always conditional: issues of trust, policy, payment, … l Coordinated problem solving –Beyond client-server: distributed data analysis, computation, … l Dynamic, multi-institutional virtual orgs –Community overlays on classic org structures –Large or small, static or dynamic

University of Chicago Grids: Why Now? l Moore’s law improvements in computing produce highly functional end systems l The Internet and burgeoning wired and wireless provide universal connectivity l Network exponentials produce dramatic changes in geometry and geography

University of Chicago Grids: Why Now? l Moore’s law improvements in computing produce highly functional endsystems l The Internet and burgeoning wired and wireless provide universal connectivity l Network exponentials produce dramatic changes in geometry and geography –9-month doubling: double Moore’s law! – : x340,000; : x4000?

University of Chicago A Little History l Early 90s –Gigabit testbeds, metacomputing l Mid to late 90s –Early experiments (e.g., I-WAY), software projects (e.g., Globus), application experiments l 2002 –Major application communities emerging –Major infrastructure deployments are underway –Rich technology base has been constructed –Global Grid Forum: >1000 people on mailing lists, 192 orgs at last meeting, 28 countries

University of Chicago The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education –Deployment, application, technology l Considerable consensus on key concepts and technologies –Globus Toolkit™ has emerged as de facto standard for major protocols & services l Global Grid Forum has emerged as a significant force –And first “Grid” proposals at IETF

University of Chicago Selected Major Grid Projects NameURL & SponsorsFocus Access Grid accessgrid; DOE, NSF Create & deploy group collaboration systems using commodity technologies BlueGridIBMGrid testbed linking IBM laboratories DISCOM discom DOE Defense Programs Create operational Grid providing access to resources at three U.S. DOE weapons laboratories DOE Science Grid sciencegrid.org DOE Office of Science Create operational Grid providing access to resources & applications at U.S. DOE science laboratories & partner universities Earth System Grid (ESG) earthsystemgrid.org DOE Office of Science Delivery and analysis of large climate model datasets for the climate research community European Union (EU) DataGrid eu-datagrid.org European Union Create & apply an operational grid for applications in high energy physics, environmental science, bioinformatics g g g g g g New

University of Chicago Selected Major Grid Projects NameURL/SponsorFocus EuroGrid, Grid Interoperability (GRIP) eurogrid.org European Union Create technologies for remote access to supercomputer resources & simulation codes; in GRIP, integrate with Globus Fusion Collaboratory fusiongrid.org DOE Off. Science Create a national computational collaboratory for fusion research Globus Project globus.org DARPA, DOE, NSF, NASA, Msoft Research on Grid technologies; development and support of Globus Toolkit; application and deployment GridLab gridlab.org European Union Grid technologies and applications GridPP gridpp.ac.uk U.K. eScience Create & apply an operational grid within the U.K. for particle physics research Grid Research Integration Dev. & Support Center grids-center.org NSF Integration, deployment, support of the NSF Middleware Infrastructure for research & education g g g g g g New

University of Chicago Selected Major Grid Projects NameURL/SponsorFocus Grid Application Dev. Software hipersoft.rice.edu/ grads; NSF Research into program development technologies for Grid applications Grid Physics Network griphyn.org NSF Technology R&D for data analysis in physics expts: ATLAS, CMS, LIGO, SDSS Information Power Grid ipg.nasa.gov NASA Create and apply a production Grid for aerosciences and other NASA missions International Virtual Data Grid Laboratory ivdgl.org NSF Create international Data Grid to enable large-scale experimentation on Grid technologies & applications Network for Earthquake Eng. Simulation Grid neesgrid.org NSF Create and apply a production Grid for earthquake engineering Particle Physics Data Grid ppdg.net DOE Science Create and apply production Grids for data analysis in high energy and nuclear physics experiments g g g g g New g

University of Chicago Selected Major Grid Projects NameURL/SponsorFocus TeraGrid teragrid.org NSF U.S. science infrastructure linking four major resource sites at 40 Gb/s UK eScience Grid grid-support.ac.uk U.K. eScience Support center for Grid projects within the U.K. UnicoreBMBFTTechnologies for remote access to supercomputers g g New Also many technology R&D projects: e.g., Condor, NetSolve, Ninf, NWS See also

University of Chicago Grid Communities & Applications: Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents

University of Chicago Grid Communities and Applications: Mathematicians Solve NUG30 l Community=an informal collaboration of mathematicians and computer scientists l Condor-G delivers 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites) l Solves NUG30 quadratic assignment problem 14,5,28,24,1,3,16,15, 10,9,21,2,4,29,25,22, 13,26,17,30,6,20,19, 8,18,7,27,12,11,23 Argonne, Iowa, NWU, Wisconsin

University of Chicago Grid Communities and Applications: Network for Earthquake Eng. Simulation l NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other l On-demand access to experiments, data streams, computing, archives, collaboration NEESgrid: Argonne, Michigan, NCSA, UIUC, USC

University of Chicago The 13.6 TF TeraGrid: Computing at 40 Gb/s HPSS 5 UniTree External Networks Site Resources NCSA/PACI 8 TF 240 TB SDSC 4.1 TF 225 TB CaltechArgonne TeraGrid/DTF: NCSA, SDSC, Caltech, Argonne

University of Chicago Intl. Virtual Data Grid Lab. Tier0/1 facility Tier2 facility 10+ Gbps link 2.5 Gbps link 622 Mbps link Other link Tier3 facility

University of Chicago Access Grid l Collaborative work among large groups l ~50 sites worldwide l Use Grid services for discovery, security l Ambient mic (tabletop) Presenter mic Presenter camera Audience camera Access Grid: Argonne, others

University of Chicago Grid Architecture & Globus Toolkit™ l The question: –What is needed for resource sharing & coordinated problem solving in dynamic virtual organizations (VOs)? l The answer: –Major issues identified: membership, resource discovery & access, …, … –Grid architecture captures core elements, emphasizing pre-eminent role of protocols –Globus Toolkit™ has emerged as de facto standard for major protocols & services

University of Chicago The Critical Role of Protocols l Need for interoperability when different groups want to share resources –E.g., IP lets me talk to your computer, but how do we establish & maintain sharing? –How do I discover, authenticate, authorize, describe what I want to do, etc., etc.? l Need for shared infrastructure services to avoid repeated development, installation, e.g. –One port/service for remote access to computing, not one per tool/application –X.509 enables sharing of Certificate Authorities

University of Chicago Grid Architecture Application Fabric “Controlling things locally”: Access to, & control of, resources Connectivity “Talking to things”: communication (Internet protocols) & security Resource “Sharing single resources”: negotiating access, controlling use Collective “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Internet Transport Application Link Internet Protocol Architecture For more info:

University of Chicago Globus Project and Toolkit l Globus Project™ –R&D project at ANL, U.Chicago, USC/ISI –Emphasis on identifying and defining core protocols and services –O(40) researchers & developers l Globus Toolkit™ –A major product of the Globus Project –Open source software: reference implementation of core protocols & services –Growing open source developer community

University of Chicago Globus & Architecture (1): Fabric Layer l Diverse resources that may be shared –Computers, clusters, Condor pools, file systems, archives, metadata catalogs, networks, sensors, etc., etc. l Speak connectivity, resource protocols –The neck of the protocol hourglass l May implement standard behaviors –Reservation, pre-emption, virtualization –Grid operation can have profound implications for resource behavior Grid resource Registration, enquiry, management, access protocol(s)

University of Chicago Globus & Architecture (2): Connectivity Layer Protocols & Services l Communication –Internet protocols: IP, DNS, routing, etc. l Security: Grid Security Infrastructure (GSI) –Uniform authentication & authorization mechanisms in multi-institutional setting –Single sign-on, delegation, identity mapping –Public key technology, SSL, X.509, GSS-API (several Internet drafts document extensions) –Supporting infrastructure: Certificate Authorities, key management, etc.

University of Chicago Site A (Kerberos) Site B (Unix) Site C (Kerberos) Computer User Single sign-on via “grid-id” & generation of proxy cred. Or: retrieval of proxy cred. from online repository User Proxy Proxy credential Computer Storage system Communication* GSI-enabled FTP server Authorize Map to local id Access file Remote file access request* GSI-enabled GRAM server GSI-enabled GRAM server Remote process creation requests* * With mutual authentication Process Kerberos ticket Restricted proxy Process Restricted proxy Local id Authorize Map to local id Create process Generate credentials Ditto GSI in Action: “Create Processes at A and B that Communicate & Access Files at C”

University of Chicago Globus & Architecture (3): Resource Layer Protocols & Services l Resource management: GRAM –Remote allocation, reservation, monitoring, control of [compute] resources l Data access: GridFTP –High-performance data access & transport l Information: MDS (GRRP, GRIP) –Access to structure & state information l & others emerging: database access, code repository access, accounting, … l All integrated with GSI

University of Chicago GRAM Resource Management Protocol l Grid Resource Allocation & Management –Allocation, monitoring, control of computations –Secure remote access to diverse schedulers l Current evolution –Immediate and advance reservation –Multiple resource types: manage anything –Recoverable requests, timeout, etc. –Evolve to Web Services –Policy evaluation points for restricted proxies Karl Czajkowski, Steve Tuecke, others

University of Chicago Data Access & Transfer l GridFTP: extended version of popular FTP protocol for Grid data access and transfer l Secure, efficient, reliable, flexible, extensible, parallel, concurrent, e.g.: –Third-party data transfers, partial file transfers –Parallelism, striping (e.g., on PVFS) –Reliable, recoverable data transfers l Reference implementations –Existing clients and servers: wuftpd, nicftp –Flexible, extensible libraries Bill Allcock, Joe Bester, John Bresnahan, Steve Tuecke, others

University of Chicago Grid Services Architecture (4): Collective Layer Protocols & Services l Community membership & policy –E.g., Community Authorization Service l Index/metadirectory/ brokering services –E.g., Globus GIIS, Condor Matchmaker l Replica management and replica selection –Optimize aggregate data access performance l Co-reservation and co-allocation services –End-to-end performance l Middle tier services –MyProxy credential repository, portal services

University of Chicago Data Grids l Grid infrastructures, tools, and applications focused on enabling distributed access to, & analysis of, large amounts of data l A specialization and extension of standard Grid technologies l Current application domains include high energy & nuclear physics, climate data analysis, astronomy, bioinformatics

University of Chicago Grid Physics Network (GriPhyN) Enabling R&D for advanced data grid systems, focusing in particular on Virtual Data concept ATLAS CMS LIGO SDSS Paul Avery, Ian Foster, Co-PIs

University of Chicago Future Directions l Initial exploration ( ; Globus 1.0) –Extensive appln experiments; core protocols l Data Grids (1999-??; Globus 2.0+) –Large-scale data management and analysis l Open Grid Services Architecture (2001-??, Globus 3.0) –Integration w/ Web services, hosting envs. –Integration with databases –Integrated set of higher-level services l Scalable systems (2003-??) –Sensors, wireless, ubiquitous computing

University of Chicago Summary l The Grid problem: Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations l Grid architecture: Protocol, service definition for interoperability & resource sharing l Globus Toolkit™ a source of protocol and API definitions—and reference implementations –And many projects applying Grid concepts (& Globus technologies) to important problems l Timely to start applying technologies to industrial problems, within & outside STC

University of Chicago For More Information l The Globus Project™ – l Global Grid Forum – l Grid architecture – /papers/anatomy.pdf l Open Grid Services Architecture (soon) – /papers/ogsa.pdf – /papers/gsspec.pdf