1. Future Scientific Infrastructure Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster Keynote Talk, QUESTnet 2002 Conference, Gold Coast, July 4, 2002

2. 2 foster@mcs.anl.gov ARGONNE  CHICAGO Evolution of Infrastructure l 1890: Local power generation l 2002: Primarily local computing & storage –AC transmission => power Grid => economies of scale & revolutionary new devices –Internet & optical technologies => ???

3. 3 foster@mcs.anl.gov ARGONNE  CHICAGO A Computing Grid l On-demand, ubiquitous access to computing, data, and services “We will perhaps see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country” (Len Kleinrock, 1969) “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder, 2001) l New capabilities constructed dynamically and transparently from distributed services

4. 4 foster@mcs.anl.gov ARGONNE  CHICAGO Distributed Computing+Visualization Remote Center Generates Tb+ datasets from simulation code LAN/WAN Transfer User-friendly striped GridFTP application tiles the frames and stages tiles onto display nodes FLASH data transferred to ANL for visualization GridFTP parallelism utilizes high bandwidth (Capable of utilizing >Gb/s WAN links) WAN Transfer Chiba City Visualization code constructs and stores high-resolution visualization frames for display on many devices ActiveMural Display Displays very high resolution large-screen dataset animations Job Submission Simulation code submitted to remote center for execution on 1000s of nodes FUTURE (1-5 yrs) 10s Gb/s LANs, WANs End-to-end QoS Automated replica management Server-side data reduction & analysis Interactive portals

5. 5 foster@mcs.anl.gov ARGONNE  CHICAGO eScience Application: Sloan Digital Sky Survey Analysis

6. 6 foster@mcs.anl.gov ARGONNE  CHICAGO catalog cluster 5 4 core brg field tsObj 3 2 1 brg field tsObj 2 1 brg field tsObj 2 1 brg field tsObj 2 1 core 3 Cluster-finding Data Pipeline

7. 7 foster@mcs.anl.gov ARGONNE  CHICAGO Size distribution of galaxy clusters? Galaxy cluster size distribution Chimera Virtual Data System + iVDGL Data Grid (many CPUs) Chimera Application: Sloan Digital Sky Survey Analysis

8. 8 foster@mcs.anl.gov ARGONNE  CHICAGO Lift Capabilities Drag Capabilities Responsiveness Deflection capabilities Responsiveness Thrust performance Reverse Thrust performance Responsiveness Fuel Consumption Braking performance Steering capabilities Traction Dampening capabilities Crew Capabilities - accuracy - perception - stamina - re-action times - SOPs Engine Models Airframe Models Wing Models Landing Gear Models Stabilizer Models Human Models Grids at NASA: Aviation Safety

9. 9 foster@mcs.anl.gov ARGONNE  CHICAGO NETWORK IMAGING INSTRUMENTS COMPUTATIONAL RESOURCES LARGE DATABASES DATA ACQUISITION PROCESSING, ANALYSIS ADVANCED VISUALIZATION Life Sciences: Telemicroscopy

10. 10 foster@mcs.anl.gov ARGONNE  CHICAGO Business Opportunities l On-demand computing, storage, services –Significant savings due to reduced build-out, economies of scale, reduced admin costs –Greater flexibility => greater productivity l Entirely new applications and services –Based on high-speed resource integration l Solution to enterprise computing crisis –Render distributed infrastructures manageable

11. 11 foster@mcs.anl.gov ARGONNE  CHICAGO Grid Evolution

12. 12 foster@mcs.anl.gov ARGONNE  CHICAGO Grids and Industry: Early Examples Butterfly.net: Grid for multi-player games Entropia: Distributed computing (BMS, Novartis, …)

13. 13 foster@mcs.anl.gov ARGONNE  CHICAGO l Resources –Computing, storage, data l Communities –Operational procedures, … Grid Infrastructure A A A l Services –Authentication, discovery, … l Connectivity –Reduce tyranny of distance l Technologies –Build applications, services

14. 14 foster@mcs.anl.gov ARGONNE  CHICAGO Example Grid Infrastructure Projects l I-WAY (1995): 17 U.S. sites for one week l GUSTO (1998): 80 sites worldwide, experim. l NASA Information Power Grid (since 1999) –Production Grid linking NASA laboratories l INFN Grid, EU DataGrid, iVDGL, … (2001+) –Grids for data-intensive science l TeraGrid, DOE Science Grid (2002+) –Production Grids link supercomputer centers l U.S. GRIDS Center –Software packaging, deployment, support

15. 15 foster@mcs.anl.gov ARGONNE  CHICAGO Topics in Grid Infrastructure l Regional, national, intl optical infrastructure –I-WIRE, StarLight, APAN l TeraGrid: Deep infrastructure –High-end support for U.S. community l iVDGL: Wide infrastructure –Building a (international) community l Open Grid Services Architecture –Future service & technology infrastructure

16. 16 foster@mcs.anl.gov ARGONNE  CHICAGO Topics in Grid Infrastructure l Regional, national, intl optical infrastructure –I-WIRE, StarLight l TeraGrid: Deep infrastructure –High-end support for U.S. community l iVDGL: Wide infrastructure –Building a (international) community l Open Grid Services Architecture –Future service & technology infrastructure

17. 17 foster@mcs.anl.gov ARGONNE  CHICAGO Targeted StarLight Optical Network Connections Vancouver Seattle Portland San Francisco Los Angeles San Diego (SDSC) NCSA Chicago NYC SURFnet CA*net4 Asia- Pacific AMPATH PSC Atlanta IU U Wisconsin DTF 40Gb NTON AMPATH Atlanta NCSA/UIUC ANL UIC Chicago Cross connect NW Univ (Chicago) StarLight Hub Ill Inst of Tech Univ of Chicago Indianapolis (Abilene NOC) St Louis GigaPoP I-WIRE www.startap.net CERN

18. 18 foster@mcs.anl.gov ARGONNE  CHICAGO UI-Chicago Illinois Inst. Tech Argonne Nat’l Lab (approx 25 miles SW) Northwestern Univ-Chicago “Starlight” U of Chicago I-55 Dan Ryan Expwy (I-90/94) I-290 I-294 UIUC/NCSA Urbana (approx 140 miles South) ANL IITUIC UChicago Main UIUC/NCSA Starlight / NU-C ICN UChicago Gleacher Ctr Commercial Fiber Hub I-WIRE Geography Status: Done: ANL, NCSA, Starlight Laterals in process: UC, UIC, IIT Investigating extensions to Northwestern Evanston, Fermilab, O’Hare, Northern Illinois Univ, DePaul, etc.

19. 19 foster@mcs.anl.gov ARGONNE  CHICAGO UIUC/NCSA Starlight (NU-Chicago) Argonne UChicago IIT UIC State/City Complex James R. Thompson Ctr City Hall State of IL Bldg 4 12 4 2 2 4 18 410 12 2 Level(3) 111 N. Canal McLeodUSA 151/155 N. Michigan Doral Plaza Qwest 455 N. Cityfront UC Gleacher Ctr 450 N. Cityfront I-Wire Fiber Topology Fiber Providers: Qwest, Level(3), McLeodUSA, 360Networks 10 segments 190 route miles; 816 fiber miles Longest segment: 140 miles 4 strands minimum to each site Numbers indicate fiber count (strands) FNAL (est 4q2002)

20. 20 foster@mcs.anl.gov ARGONNE  CHICAGO UIUC/NCSA Starlight (NU-Chicago) Argonne UChicago IIT UIC State/City Complex James R. Thompson Ctr City Hall State of IL Bldg UC Gleacher Ctr 450 N. Cityfront I-Wire Transport Each of these three ONI DWDM systems have capacity of up to 66 channels, up to 10 Gb/s per channel Protection available in Metro Ring on a per-site basis TeraGrid Linear 3x OC192 1x OC48 First light: 6/02 Metro Ring 1x OC48 per site First light: 8/02 Starlight Linear 4x OC192 4x OC48 (  8x GbE) Operational McLeodUSA 151/155 N. Michigan Doral Plaza Qwest 455 N. Cityfront

21. 21 foster@mcs.anl.gov ARGONNE  CHICAGO UI-Chicago Illinois Inst. Tech Argonne Nat’l Lab (approx 25 miles SW) Northwestern Univ-Chicago “Starlight” U of Chicago I-55 Dan Ryan Expwy (I-90/94) I-290 I-294 UIUC/NCSA Urbana (approx 140 miles South) Illinois Distributed Optical Testbed DAS-2

22. 22 foster@mcs.anl.gov ARGONNE  CHICAGO Topics in Grid Infrastructure l Regional, national, intl optical infrastructure –I-WIRE, StarLight l TeraGrid: Deep infrastructure –High-end support for U.S. community l iVDGL: Wide infrastructure –Building a (international) community l Open Grid Services Architecture –Future service & technology infrastructure

23. 23 foster@mcs.anl.gov ARGONNE  CHICAGO www.teragrid.org TeraGrid: Deep Infrastructure

24. 24 foster@mcs.anl.gov ARGONNE  CHICAGO TeraGrid Objectives l Create unprecedented capability –Integrated with extant PACI capabilities –Supporting a new class of scientific research l Deploy a balanced, distributed system –Not a “distributed computer” but rather … –a distributed “system” using Grid technologies >Computing and data management >Visualization and scientific application analysis l Define an open and extensible infrastructure –Enabling infrastructure for scientific research –Extensible beyond the original four sites > NCSA, SDSC, ANL, and Caltech

25. 25 foster@mcs.anl.gov ARGONNE  CHICAGO An Extensible Grid l Exploit DTF homogeneity where possible –Accelerate deployment and lower user entry barriers (e.g., executable portability) l Standard service & software deployments –Leverage NSF NMI (Globus Toolkit etc.) –A basis for interoperability/expansion l Partnerships with GriPhyn, iVDGL, PPDG, NEESgrid, others –Application drivers and larger grids

26. 26 foster@mcs.anl.gov ARGONNE  CHICAGO TeraGrid Timelines Early access To McKinley At Intel Jan ‘01 Proposal Submitted To NSF Jan ‘02Jan ‘03 Early McKinleys at TG sites for Testing/benchmarking TeraGrid clusters TeraGrid Operations Center Prototype Day Ops Production Basic Grid svcs Linux clusters SDSC SP NCSA O2K Core Grid services deployment Initial apps On McKinley TeraGrid Operational TeraGrid Networking Deployment TeraGrid prototype At SC2001, 60 Itanium Nodes, 10Gbs network McKinley systems TeraGrid Prototypes Grid Services on Current Systems Networking Operations 10Gigabit Enet testing “TeraGrid Lite” Systems and Grids testbed Advanced Grid services testing Applications

27. 27 foster@mcs.anl.gov ARGONNE  CHICAGO 64 hosts Spine Switches 128-port Clos Switches 64 inter-switch links 100Mb/s Switched Ethernet Management Network (c) I/O - Storage(d) Visualization Clos mesh Interconnect Each line = 8 x 2Gb/s links 64 TB RAID 64 inter-switch links = 4 links 64 inter-switch links Local DisplayNetworks for Remote Display (e) Compute (b) Example 320-node Clos Network(a) Terascale Architecture Overview Add’l Clusters, External Networks Terascale Cluster Architecture Myrinet System Interconnect FCS Storage Network GbE for external traffic Rendered Image files

28. 28 foster@mcs.anl.gov ARGONNE  CHICAGO Initial TeraGrid Design DWDM Optical Mesh NCSA 2024 McKinley Processors (8 Teraflops, 512 nodes) 250 TB RAID storage ANL 384 McKinley Processors (1.5 Teraflops, 96 nodes) 125 TB RAID storage Caltech SDSC 768 McKinley Processors (3 Teraflops, 192 nodes) 250 TB RAID storage 384 McKinley Processors (1.5 Teraflops, 96 nodes) 125 TB RAID storage

29. 29 foster@mcs.anl.gov ARGONNE  CHICAGO NSF TeraGrid: 14 TFLOPS, 750 TB Myrinet HPSS 1176p IBM SP Blue Horizon Sun E10K 1500p Origin UniTree 1024p IA-32 320p IA-64 HPSS 574p IA-32 Chiba City HR Display & VR Facilities HPSS 256p HP X-Class 128p HP V2500 92p IA-32 NCSA: Compute-Intensive ANL: VisualizationCaltech: Data collection analysis SDSC: Data-Intensive WAN Bandwidth Options: Abilene (2.5 Gb/s,  10Gb/s late 2002) State and regional fiber initiatives plus CANARIE CA*Net Leased OC48 Dark Fiber, Dim Fiber, Wavelengths WAN Architecture Options: Myrinet-to-GbE; Myrinet as a WAN Layer2 design Wavelength Mesh Traditional IP Backbone

30. 30 foster@mcs.anl.gov ARGONNE  CHICAGO TeraGrid Interconnect Partnership with Qwest 1 2 Caltech/JPL SDSC/UCSD NCSA/UIUC ANL Physical # denotes λ count Light Paths (Logical) La Jolla San Diego LA Pasadena Chicago Urbana Argonne Phase 0 Phase 1 2 1 4 3 3 3 3

31. 31 foster@mcs.anl.gov ARGONNE  CHICAGO NSFNET 56 Kb/s Site Architecture VAX Fuzzball 1024MB 4 MB/s1 MB/s.007 MB/s 256 s (4 min) 1024 s (17 min)150,000 s (41 hrs) Across the roomAcross the country Bandwidth in terms of burst data transfer and user wait time.

32. 32 foster@mcs.anl.gov ARGONNE  CHICAGO OC-48 Cloud 0.5 GB/s78 MB/s 2000 s (33 min) 13k s (3.6h) 2002 Cluster-WAN Architecture 1 TB n x GbE (small n) OC-12 Across the roomAcross the country

33. 33 foster@mcs.anl.gov ARGONNE  CHICAGO Distributed Terascale Cluster Interconnect Big Fast Interconnect 10 TB 5 GB/s 2000 s (33 min) 10 TB 5 GB/s = 200 x 25 MB/s 25 MB/s = 200 Mb/s or 20% of GbE n x GbE (large n) OC-192

34. 34 foster@mcs.anl.gov ARGONNE  CHICAGO NSF TeraGrid: 14 TFLOPS, 750 TB HPSS 574p IA-32 Chiba City 128p Origin HR Display & VR Facilities Myrinet 1176p IBM SP Blue Horizon Sun E10K 1500p Origin UniTree 1024p IA-32 320p IA-64 HPSS 256p HP X-Class 128p HP V2500 92p IA-32 NCSA: Compute-Intensive ANL: Visualization Caltech: Data collection analysis SDSC: Data-Intensive

35. 35 foster@mcs.anl.gov ARGONNE  CHICAGO Defining Standard Services IA-64 Linux Cluster Runtime File-based Data Service Collection-based Data Service Volume-Render Service Interactive Collection-Analysis Service Grid Applications IA-64 Linux Cluster Interactive Development Finite set of TeraGrid services- applications see standard services rather than particular implementations… …but sites also provide additional services that can be discovered and exploited.

36. 36 foster@mcs.anl.gov ARGONNE  CHICAGO Standards  Cyberinfrastructure Runtime File-based Data Service Collection-based Data Service Visualization Services Interactive Collection-Analysis Service Grid Applications Interactive Development IA-64 Linux Clusters Alpha Clusters IA-32 Linux Clusters Visualization Services Data/InformationComputeAnalysis Relational dBase Data Service TeraGrid Certificate Authority Certificate Authority Grid Info Svces TeraGrid: focus on a finite set of service specifications applicable to TeraGrid resources. If done well, other IA-64 cluster sites would adopt TeraGrid service specifications, increasing users’ leverage in writing to the specification, and others would adopt the framework for developing similar services (for Alpha, IA-32, etc.) Note the specification should attempt to offer improvement over general Globus runtime environment without bogging down attempting to do everything (for which a user is better off running interactively!)

37. 37 foster@mcs.anl.gov ARGONNE  CHICAGO Strategy: Define Standard Services l Finite number of TeraGrid Services –Defined as specifications, protocols, APIs –Separate from implementation l Example: File-based Data Service –API/Protocol: Supports FTP and GridFTP, GSI authentication –SLA: All TeraGrid users have access to N TB storage, available 24/7 with M% availability, >= R Gb/s read, >= W Gb/s write, etc.

38. 38 foster@mcs.anl.gov ARGONNE  CHICAGO General TeraGrid Services l Authentication –GSI: Requires TeraGrid CA policy and services l Resource Discovery and Monitoring –Define TeraGrid services/attributes to be published in Globus MDS-2 directory services –Require standard account information exchange to map use to allocation/individual –For many services, publish query interface >Scheduler: queue status >Compute, Visualization, etc. services: attribute details >Network Weather Service >Allocations/Accounting Database: for allocation status

39. 39 foster@mcs.anl.gov ARGONNE  CHICAGO General TeraGrid Services l Advanced Reservation –On-demand services –Staging data: coordination of storage+compute l Communication and Data Movement –All services assume any TeraGrid cluster node can talk to any TeraGrid cluster node –All resources support GridFTP l “Hosting environment” –Standard software environment –More sophisticated dynamic provisioning issues not yet addressed

40. 40 foster@mcs.anl.gov ARGONNE  CHICAGO Extending TeraGrid l Adoption of TG specs, protocols, API’s –What protocols, what data formats are expected, what features can I expect (how does it behave) –Service Level Agreements (SLA) l Extension and expansion via: –Additional services not defined in TeraGrid >E.g., Alpha Cluster Runtime service –Additional instantiations of TeraGrid services >E.g., IA-64 runtime service implemented on cluster at a new site

41. 41 foster@mcs.anl.gov ARGONNE  CHICAGO Topics in Grid Infrastructure l Regional, national, intl optical infrastructure –I-WIRE, StarLight l TeraGrid: Deep infrastructure –High-end support for U.S. community l iVDGL: Wide infrastructure –Building a (international) community l Open Grid Services Architecture –Future service & technology infrastructure

42. 42 foster@mcs.anl.gov ARGONNE  CHICAGO iVDGL: A Global Grid Laboratory l International Virtual-Data Grid Laboratory –A global Grid laboratory (US, Europe, Asia, South America, …) –A place to conduct Data Grid tests “at scale” –A mechanism to create common Grid infrastructure –A laboratory for other disciplines to perform Data Grid tests –A focus of outreach efforts to small institutions l U.S. part funded by NSF (2001-2006) –$13.7M (NSF) + $2M (matching) “We propose to create, operate and evaluate, over a sustained period of time, an international research laboratory for data-intensive science.” From NSF proposal, 2001

43. 43 foster@mcs.anl.gov ARGONNE  CHICAGO Initial US-iVDGL Data Grid Tier1 (FNAL) Proto-Tier2 Tier3 university UCSD Florida Wisconsin Fermilab BNL Indiana BU Other sites to be added in 2002 SKC Brownsville Hampton PSU JHU Caltech

44. 44 foster@mcs.anl.gov ARGONNE  CHICAGO U.S. PIs: Avery, Foster, Gardner, Newman, Szalay www.ivdgl.org iVDGL: International Virtual Data Grid Laboratory Tier0/1 facility Tier2 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link Tier3 facility

45. 45 foster@mcs.anl.gov ARGONNE  CHICAGO iVDGL Architecture (from proposal)

46. 46 foster@mcs.anl.gov ARGONNE  CHICAGO US iVDGL Interoperability l US-iVDGL-1 Milestone (August 02) ATLAS CMSLIGO SDSS/NVO US-iVDGL- 1 Aug 2002 US-iVDGL- 1 Aug 2002 1 2 iGOC 1 2 1 2 1 2

47. 47 foster@mcs.anl.gov ARGONNE  CHICAGO Transatlantic Interoperability l iVDGL-2 Milestone (November 02) ATLAS CMSLIGO SDSS/NVO iVDGL-2 Nov 2002 iVDGL-2 Nov 2002 ANL BNL BU IU UM OU UTA HU LBL CIT UCSD UF FNAL JHU CS Research ANL UCB UC IU NU UW ISI CIT UTB PSU UWM UC iGOC Outreach DataTAG CERN INFN UK PPARC U of A

48. 48 foster@mcs.anl.gov ARGONNE  CHICAGO Topics in Grid Infrastructure l Regional, national, intl optical infrastructure –I-WIRE, StarLight l TeraGrid: Deep infrastructure –High-end support for U.S. community l iVDGL: Wide infrastructure –Building a (international) community l Open Grid Services Architecture –Future service & technology infrastructure

49. 49 foster@mcs.anl.gov ARGONNE  CHICAGO “Standard” Software Infrastructure: Globus Toolkit TM l Small, standards-based set of protocols for distributed system management –Authentication, delegation; resource discovery; reliable invocation; etc. l Information-centric design –Data models; publication, discovery protocols l Open source implementation –Large international user community l Successful enabler of higher-level services and applications

50. 50 foster@mcs.anl.gov ARGONNE  CHICAGO Example Grid Projects in eScience

51. 51 foster@mcs.anl.gov ARGONNE  CHICAGO User process #1 Proxy Authenticate & create proxy credential GSI (Grid Security Infrastruc- ture) Gatekeeper (factory) Reliable remote invocation GRAM (Grid Resource Allocation & Management) Reporter (registry + discovery) User process #2 Proxy #2 Create process Register The Globus Toolkit in One Slide l Grid protocols (GSI, GRAM, …) enable resource sharing within virtual orgs; toolkit provides reference implementation ( = Globus Toolkit services) l Protocols (and APIs) enable other tools and services for membership, discovery, data mgmt, workflow, … Other service (e.g. GridFTP) Other GSI- authenticated remote service requests GIIS: Grid Information Index Server (discovery) MDS-2 (Monitor./Discov. Svc.) Soft state registration; enquiry

52. 52 foster@mcs.anl.gov ARGONNE  CHICAGO Globus Toolkit: Evaluation (+) l Good technical solutions for key problems, e.g. –Authentication and authorization –Resource discovery and monitoring –Reliable remote service invocation –High-performance remote data access l This & good engineering is enabling progress –Good quality reference implementation, multi- language support, interfaces to many systems, large user base, industrial support –Growing community code base built on tools

53. 53 foster@mcs.anl.gov ARGONNE  CHICAGO Globus Toolkit: Evaluation (-) l Protocol deficiencies, e.g. –Heterogeneous basis: HTTP, LDAP, FTP –No standard means of invocation, notification, error propagation, authorization, termination, … l Significant missing functionality, e.g. –Databases, sensors, instruments, workflow, … –Virtualization of end systems (hosting envs.) l Little work on total system properties, e.g. –Dependability, end-to-end QoS, … –Reasoning about system properties

54. 54 foster@mcs.anl.gov ARGONNE  CHICAGO Globus Toolkit Structure GRAMMDS GSI GridFTPMDS GSI ??? GSI Reliable invocation Soft state management Notification Compute Resource Data Resource Other Service or Application Job manager Job manager Lots of good mechanisms, but (with the exception of GSI) not that easily incorporated into other systems Service naming

55. 55 foster@mcs.anl.gov ARGONNE  CHICAGO Grid Evolution: Open Grid Services Architecture l Refactor Globus protocol suite to enable common base and expose key capabilities l Service orientation to virtualize resources and unify resources/services/information l Embrace key Web services technologies for standard IDL, leverage commercial efforts l Result: standard interfaces & behaviors for distributed system management: the Grid service

56. 56 foster@mcs.anl.gov ARGONNE  CHICAGO Open Grid Services Architecture: Transient Service Instances l “Web services” address discovery & invocation of persistent services –Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically –Interfaces to the states of distributed activities –E.g. workflow, video conf., dist. data analysis l Significant implications for how services are managed, named, discovered, and used –In fact, much of OGSA (and Grid) is concerned with the management of service instances

57. 57 foster@mcs.anl.gov ARGONNE  CHICAGO Open Grid Services Architecture l Defines fundamental (WSDL) interfaces and behaviors that define a Grid Service –Required + optional interfaces = WS “profile” –A unifying framework for interoperability & establishment of total system properties l Defines WSDL extensibility elements –E.g., serviceType (a group of portTypes) l Delivery via open source Globus Toolkit 3.0 –Leverage GT experience, code, community l And commercial implementations

58. 58 foster@mcs.anl.gov ARGONNE  CHICAGO The Grid Service = Interfaces/Behaviors + Service Data Service data element Service data element Service data element Implementation GridService (required) Service data access Explicit destruction Soft-state lifetime … other interfaces … (optional) Standard: - Notification - Authorization - Service creation - Service registry - Manageability - Concurrency + application- specific interfaces Binding properties: - Reliable invocation - Authentication Hosting environment/runtime (“C”, J2EE,.NET, …)

59. 59 foster@mcs.anl.gov ARGONNE  CHICAGO Grid Service Example: Database Service l A DBaccess Grid service will support at least two portTypes –GridService –DBaccess l Each has service data –GridService: basic introspection information, lifetime, … –DBaccess: database type, query languages supported, current load, …, … l Maybe other portTypes as well –E.g., NotificationSource (SDE = subscribers) Grid Service DBaccess DB info Name, lifetime, etc.

60. 60 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider “I want to create a personal database containing data on e.coli metabolism”...... Database Factory

61. 61 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... “Find me a data mining service, and somewhere to store data” Database Factory

62. 62 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... GSHs for Mining and Database factories Database Factory

63. 63 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... “Create a data mining service with initial lifetime 10” “Create a database with initial lifetime 1000” Database Factory

64. 64 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner “Create a data mining service with initial lifetime 10” “Create a database with initial lifetime 1000”

65. 65 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Query

66. 66 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Query Keepalive

67. 67 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Keepalive Results

68. 68 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Keepalive

69. 69 foster@mcs.anl.gov ARGONNE  CHICAGO Example: Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Keepalive

70. 70 foster@mcs.anl.gov ARGONNE  CHICAGO GT3: An Open Source OGSA- Compliant Globus Toolkit l GT3 Core –Implements Grid service interfaces & behaviors –Reference impln of evolving standard –Multiple hosting envs: Java/J2EE, C, C#/.NET? l GT3 Base Services –Evolution of current Globus Toolkit capabilities l Many other Grid services GT3 Core GT3 Base Services Other Grid Services GT3 Data Services

71. 71 foster@mcs.anl.gov ARGONNE  CHICAGO OGSA Definition and Delivery (Very Approximate!!) Grid Service Specification Globus OGSI Reference Impln GGF OGSI WG Prototype Feedback Other Systems Other specs: Databases Etc. GGF WGs Other OGSA- based software Other core specs: Security Res. Mgmt. Etc. GGF WGs Globus Toolkit Version 3 TIMETIME

72. 72 foster@mcs.anl.gov ARGONNE  CHICAGO Q2: What Higher-Level Services?

73. 73 foster@mcs.anl.gov ARGONNE  CHICAGO Summary: Grid Infrastructure l Grid applications demand new infrastructure beyond traditional computers and networks –Network-accessible resources of all types –High-speed networks –Services and operational procedures –Software technology for building services (which must also be treated as infrastructure) l TeraGrid, iVDGL, StarLight, DOT –Connections to international sites?

74. 74 foster@mcs.anl.gov ARGONNE  CHICAGO Summary: Open Grid Services Architecture l Open Grid Services Architecture represents (we hope!) next step in Grid evolution l Service orientation enables unified treatment of resources, data, and services l Standard interfaces and behaviors (the Grid service) for managing distributed state l Deeply integrated information model for representing and disseminating service data l Open source Globus Toolkit implementation (and commercial value adds)

75. 75 foster@mcs.anl.gov ARGONNE  CHICAGO For More Information l Survey + research articles –www.mcs.anl.gov/~foster l I-WIRE: www.iwire.org l TeraGrid: www.teragrid.org l iVDGL: www.ivDGL.org l The Globus Project™ –www.globus.org l GriPhyN project –www.griphyn.org l Global Grid Forum –www.gridforum.org –www.gridforum.org/ogsi-wg –Edinburgh, July 22-24 –Chicago, Oct 15-17