The Challenges of Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago Condor Week Presentation at via Access Grid, March 4, 2002

ARGONNE  CHICAGO Trends in Grid Computing l New application domains –Data Grids, collaboratories l Integration with commercial technologies –Web services,.NET, …, … l Commercial acceptance and adoption –Very rapid; we don’t know what has hit us l Increasing scale –Sensor nets, mobile and wireless devices, Internet-wide deployment

ARGONNE  CHICAGO The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies –Open source Globus Toolkit™ a de facto standard for major protocols & services –Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

ARGONNE  CHICAGO Grid Computing

ARGONNE  CHICAGO The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

ARGONNE  CHICAGO Why Grids? (1) eScience l A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour l 1,000 physicists worldwide pool resources for peta-op analyses of petabytes of data l Civil engineers collaborate to design, execute, & analyze shake table experiments l Climate scientists visualize, annotate, & analyze terabyte simulation datasets l An emergency response team couples real time data, weather model, population data

ARGONNE  CHICAGO Why Grids? (2) eBusiness l Engineers at a multinational company collaborate on the design of a new product l A multidisciplinary analysis in aerospace couples code and data in four companies l An insurance company mines data from partner hospitals for fraud detection l An application service provider offloads excess load to a compute cycle provider l An enterprise configures internal & external resources to support eBusiness workload

ARGONNE  CHICAGO Intelligent Infrastructure: Distributed Servers and Services

ARGONNE  CHICAGO Strategic Challenges l Managing the transition from a research area to an industry l Profiting from commercial technologies without compromising eScience needs l Innovating fast enough to meet needs of new application domains l Establishing and extending Grid community l Pursuing the research to address next- generation systems

ARGONNE  CHICAGO Our Approach l Define & promote unifying, open view of Grids, integrating commercial technologies –Open Grid Services Architecture l Coherent, high-quality open source code l Pursue realization in close collaboration –For us, Condor Group is first among equals l Reach out to, and engage, industry, emphasizing benefits of collaboration –Identify areas where industry can add value

ARGONNE  CHICAGO “Web Services” l Increasingly popular standards-based framework for accessing network applications –W3C standardization; Microsoft, IBM, Sun, others l WSDL: Web Services Description Language –Interface Definition Language for Web services l SOAP: Simple Object Access Protocol –XML-based RPC protocol; common WSDL target l WS-Inspection –Conventions for locating service descriptions l UDDI: Universal Desc., Discovery, & Integration –Directory for Web services

ARGONNE  CHICAGO Transient Service Instances l “Web services” address discovery & invocation of persistent services –Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically –Interfaces to the states of distributed activities –E.g. workflow, video conf., dist. data analysis l Significant implications for how services are managed, named, discovered, and used –In fact, much of our work is concerned with the management of service instances

ARGONNE  CHICAGO OGSA Design Principles l Service orientation to virtualize resources –Everything is a service l From Web services –Standard interface definition mechanisms: multiple protocol bindings, local/remote transparency l From Grids –Service semantics, reliability and security models –Lifecycle management, discovery, other services l Multiple “hosting environments” –C, J2EE,.NET, …

ARGONNE  CHICAGO OGSA Service Model l System comprises (a typically few) persistent services & (potentially many) transient services –Everything is a service l OGSA defines basic behaviors of services: fundamental semantics, life-cycle, etc. –More than defining WSDL wrappers

ARGONNE  CHICAGO Open Grid Services Architecture: Fundamental Structure 1) WSDL conventions and extensions for describing and structuring services –Useful independent of “Grid” computing 2) Standard WSDL interfaces & behaviors for core service activities –portTypes and operations => protocols

ARGONNE  CHICAGO Standard Interfaces & Behaviors: Four Interrelated Concepts l Naming and bindings –Every service instance has a unique name, from which can discover supported bindings l Information model –Service data associated with Grid service instances, operations for accessing this info l Lifecycle –Service instances created by factories –Destroyed explicitly or via soft state l Notification –Interfaces for registering interest and delivering notifications

ARGONNE  CHICAGO l GridService Required –FindServiceData –Destroy –SetTerminationTime l NotificationSource –SubscribeToNotificationTopic –UnsubscribeToNotificationTopic l NotificationSink –DeliverNotification OGSA Interfaces and Operations Defined to Date l Factory –CreateService l PrimaryKey –FindByPrimaryKey –DestroyByPrimaryKey l Registry –RegisterService –UnregisterService l HandleMap –FindByHandle Authentication, reliability are binding properties Manageability, concurrency, etc., to be defined

ARGONNE  CHICAGO GT3: An Open Source OGSA- Compliant Globus Toolkit l GT3 Core –Implements Grid service interfaces & behaviors –Reference impln of evolving standard l GT3 Base Services –Evolution of current Globus Toolkit capabilities l Other Grid services –Many …

ARGONNE  CHICAGO User/Application Fabric ( processing, storage, communication ) Grid Condor Globus Toolkit Condor

ARGONNE  CHICAGO For More Information l The Globus Project™ – l Grid architecture – /papers/anatomy.pdf l Open Grid Services Architecture –