Peer to Peer & Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago Talk at Internet2 Peer to Peer Workshop, January 30, 2002

Abstract Peer to Peer and Grid computing are two collaborative computing technologies with different origins but many common applications and concerns. I review the origins, current state, and likely future directions of the two technologies, and discuss ways in which the university community can integrate them into their infrastructure to better support research and education. I illustrate my discussion by reference to major academic research and development projects with a significant Grid focus, such as GriPhyN ( and NEESgrid (

The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

Grids and Peer-to-Peer Computing l Common interests –Enabling the coordinated use of geographically distributed resources—in the absence of central control and existing trust relationships –Authentication, authorization, auditing, resource discovery, resource management, remote data access, fault tolerance, … –“Beyond client server” l Differences –Historically high-end vs. strongly commodity –Predominately eScience vs. eCommodity

The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education –Deployment, application, technology l Consensus on key concepts & technologies –Open source Globus Toolkit™ a de facto standard for major protocols & services –Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Global Grid Forum a significant force l Industrial interest emerging rapidly –Commercial support for Globus Toolkit; OGSA

Why Grids? (1) eScience l A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour l 1,000 physicists worldwide pool resources for peta-op analyses of petabytes of data l Civil engineers collaborate to design, execute, & analyze shake table experiments l Climate scientists visualize, annotate, & analyze terabyte simulation datasets l An emergency response team couples real time data, weather model, population data

Grid Communities & Applications: Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents

Grid Physics Network (GriPhyN) Enabling R&D for advanced data grid systems, focusing in particular on Virtual Data concept ATLAS CMS LIGO SDSS Paul Avery, Ian Foster, Co-PIs

Grid Communities and Applications: Network for Earthquake Eng. Simulation l NEESgrid: US national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other l On-demand access to experiments, data streams, computing, archives, collaboration NEESgrid: Argonne, Michigan, NCSA, UIUC, USC

Ambient mic (tabletop) Presenter mic Presenter camera Audience camera Example Grid Communities: Access Grid Collaboration l Enable collaborative work at dozens of sites worldwide, with strong sense of shared presence l Combination of commodity audio/video tech + Grid technologies for security, discovery, etc. l 40+ sites worldwide, number rising rapidly

Why Grids? (2) eBusiness l Engineers at a multinational company collaborate on the design of a new product l A multidisciplinary analysis in aerospace couples code and data in four companies l An insurance company mines data from partner hospitals for fraud detection l An application service provider offloads excess load to a compute cycle provider l An enterprise configures internal & external resources to support eBusiness workload

Open Grid Services Architecture (ANL-IBM-ISI; Globus Toolkit 3.0) l Service orientation to virtualize resources l From Web services (WSDL, WS-Ins, etc.): –Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency l Building on Globus Toolkit: –Grid service: semantics for service interactions –Management of transient instances (& state) –Factory, Registry, Discovery, other services –Reliable and secure transport l Multiple hosting targets: J2EE,.NET, “C”, …

Grids and Universities l “What can universities do to enhance their constituents’ abilities to form & participate in virtual organizations?” –Deploy standard central services: authentication, directories; policies –Create campus Grids offering distributed compute services, data services –Provide training and consulting in Grid technologies and applications l NSF middleware initiative offers support for such activities

NSF Middleware Initiative (NMI) nsf-middleware.org l GRIDS (Grid Research Integration Development & Support) Center –Integration, packaging, testing, distribution of standard NMI software (security, directory, compute, storage, etc., etc.) –Training, documentation, some support –Single POC for high-quality Grid software l EDIT (I2 + SURA) –Focus on policies and end-user tools

For More Information l The Globus Project™ – l GriPhyN – l GRIDS Center – l Global Grid Forum – l Grid architecture – /papers/anatomy.pdf