1. Grid-related High Performance Middleware and Laboratories Dr. Carl Kesselman Director Center for Grid Technologies

2. EO Grid Middleware How do we solve problems? l Communities committed to common goals -Virtual organizations l Teams with heterogeneous members & capabilities l Distributed geographically and politically -No location/organization possesses all required skills and resources l Adapt as a function of the situation -Adjust membership, reallocate responsibilities, renegotiate resources

3. EO Grid Middleware The Grid Vision “ Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations” -On-demand, ubiquitous access to computing, data, and services -New capabilities constructed dynamically and transparently from distributed services “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder)

4. EO Grid Middleware A Little History (U.S. Perspective) l Early 90s -Gigabit testbeds, metacomputing l Mid to late 90s -Early experiments (e.g., I-WAY), software projects (e.g., Globus), application experiments l 2001 -Major application communities emerging -Major infrastructure deployments are underway -Rich technology base has been constructed -Global Grid Forum: >1000 people on mailing lists, 192 orgs at last meeting, 28 countries

5. EO Grid Middleware Selected Major Grid Projects NameURL & SponsorsFocus Access Grid www.mcs.anl.gov/FL/ accessgrid; DOE, NSF Create & deploy group collaboration systems using commodity technologies BlueGridIBMGrid testbed linking IBM laboratories DISCOM www.cs.sandia.gov/ discom DOE Defense Programs Create operational Grid providing access to resources at three U.S. DOE weapons laboratories DOE Science Grid sciencegrid.org DOE Office of Science Create operational Grid providing access to resources & applications at U.S. DOE science laboratories & partner universities Earth System Grid (ESG) earthsystemgrid.org DOE Office of Science Delivery and analysis of large climate model datasets for the climate research community European Union (EU) DataGrid eu-datagrid.org European Union Create & apply an operational grid for applications in high energy physics, environmental science, bioinformatics g g g g g g

6. EO Grid Middleware Selected Major Grid Projects NameURL/SponsorFocus EuroGrid, Grid Interoperability (GRIP) eurogrid.org European Union Create technologies for remote access to supercomputer resources & simulation codes; in GRIP, integrate with Globus Fusion Collaboratory fusiongrid.org DOE Off. Science Create a national computational collaboratory for fusion research Globus Project globus.org DARPA, DOE, NSF, NASA, Msoft Research on Grid technologies; development and support of Globus Toolkit; application and deployment GridLab gridlab.org European Union Grid technologies and applications GridPP gridpp.ac.uk U.K. eScience Create & apply an operational grid within the U.K. for particle physics research Grid Research Integration Dev. & Support Center grids-center.org NSF Integration, deployment, support of the NSF Middleware Infrastructure for research & education g g g g g g

7. EO Grid Middleware Selected Major Grid Projects NameURL/SponsorFocus Grid Application Dev. Software hipersoft.rice.edu/ grads; NSF Research into program development technologies for Grid applications Grid Physics Network griphyn.org NSF Technology R&D for data analysis in physics expts: ATLAS, CMS, LIGO, SDSS Information Power Grid ipg.nasa.gov NASA Create and apply a production Grid for aerosciences and other NASA missions International Virtual Data Grid Laboratory ivdgl.org NSF Create international Data Grid to enable large-scale experimentation on Grid technologies & applications Network for Earthquake Eng. Simulation Grid neesgrid.org NSF Create and apply a production Grid for earthquake engineering Particle Physics Data Grid ppdg.net DOE Science Create and apply production Grids for data analysis in high energy and nuclear physics experiments g g g g g g

8. EO Grid Middleware Selected Major Grid Projects NameURL/SponsorFocus TeraGrid teragrid.org NSF U.S. science infrastructure linking four major resource sites at 40 Gb/s UK Grid Support Center grid-support.ac.uk U.K. eScience Support center for Grid projects within the U.K. UnicoreBMBFT Technologies for remote access to supercomputers SCECwww.scec.org Nsf Integrated geophysics modeling g g New Also many technology R&D projects: e.g., Condor, NetSolve, Ninf, NWS See also www.gridforum.org

9. EO Grid Middleware The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies -Open source Globus Toolkit™ a de facto standard for major protocols & services -Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

10. EO Grid Middleware Layered Grid Architecture Application Fabric “Controlling things locally”: Access to, & control of, resources Connectivity “Talking to things”: communication (Internet protocols) & security Resource “Sharing single resources”: negotiating access, controlling use Collective “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Internet Transport Application Link Internet Protocol Architecture

11. Globus Toolkit l Globus Toolkit is the source of many of the protocols described in “Grid architecture” l Adopted by almost all major Grid projects worldwide as a source of infrastructure l Open source, open architecture framework encourages community development l Active R&D program continues to move technology forward l Developers at ANL, USC/ISI, NCSA, LBNL, and other institutions www.globus.org

12. Globus Toolkit Components Include … l Core protocols and services -Grid Security Infrastructure -Grid Resource Access & Management -MDS information & monitoring -GridFTP data access & transfer l Other services -Community Authorization Service -DUROC co-allocation service l Other Data Grid technologies -Replica catalog, replica management service

13. EO Grid Middleware User process #1 Proxy Authenticate & create proxy credential GSI (Grid Security Infrastruc- ture) Gatekeeper (factory) Reliable remote invocation GRAM (Grid Resource Allocation & Management) Reporter (registry + discovery) User process #2 Proxy #2 Create process Register The Globus Toolkit in One Slide l Grid protocols (GSI, GRAM, …) enable resource sharing within virtual orgs; toolkit provides reference implementation ( = Globus Toolkit services) l Protocols (and APIs) enable other tools and services for membership, discovery, data mgmt, workflow, … Other service (e.g. GridFTP) Other GSI- authenticated remote service requests GIIS: Grid Information Index Server (discovery) MDS-2 (Meta Directory Service) Soft state registration; enquiry

14. EO Grid Middleware Globus Toolkit Structure GRAMMDS GSI GridFTPMDS GSI ??? GSI Reliable invocation Soft state management Notification Compute Resource Data Resource Other Service or Application Job manager Job manager Lots of good mechanisms, but (with the exception of GSI) not that easily incorporated into other systems Service naming

15. EO Grid Middleware NSF Middleware Initiative l NSF Funded Project to build national middleware infrastructure -USC/ISI, SDSC, U. Wisc., ANL, NCSA, I2 l Software Integration (NMI Software Releases) -Interoperability -Testing -Install, Configure, Manage l University Campus Infrastructure Integration -Campus Authentication / GSI -Enterprise Directories / GSI and MDS l Use NMI as Teragrid Baseline -Specialize for Teragrid unique aspects (e.g. Viz resources)

16. EO Grid Middleware NMI-R1 Software Components l Globus Toolkit l Condor-G l Network Weather Service l KX.509 / KCA l Certificate Profile Maker l Pubcookie l Grid Packaging Tools

17. EO Grid Middleware U.S. GRIDS Center l GRIDS = Grid Research, Integration, Deployment, & Support l NSF-funded center to provide -State-of-the-art middleware infrastructure to support national-scale collaborative science and engineering -Integration platform for experimental middleware technologies l ISI, NCSA, SDSC, UC, UW + commercial partners www.grids-center.org

18. EO Grid Middleware Network for Earthquake Eng. Simulation l NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other l On-demand access to experiments, data streams, computing, archives, collaboration www.neesgrid.org: Argonne, Michigan, NCSA, UIUC, USC

19. EO Grid Middleware SCEC Modeling Environment Knowledge Base Ontologies Curated taxonomies, Relations & constraints Pathway Models Pathway templates, Models of simulation codes Code Repositories Data & Simulation Products Data Collections FSM RDM AWM SRM Storage GRID Pathway Execution Policy, Data ingest, Repository access Grid Services Compute & storage management, Security DIGITAL LIBRARIES Navigation & Queries Versioning, Topic maps Mediated Collections Federated access KNOWLEDGE ACQUISITION Acquisition Interfaces Dialog planning, Pathway construction strategies Pathway Assembly Template instantiation, Resource selection, Constraint checking KNOWLEDGE REPRESENTATION & REASONING Knowledge Server Knowledge base access, Inference Translation Services Syntactic & semantic translation Pathway Instantiations Computing Users

20. EO Grid Middleware Data Intensive Physical Sciences l High energy & nuclear physics -Including new experiments at CERN l Gravity wave searches -LIGO, GEO, VIRGO l Time-dependent 3-D systems (simulation, data) -Earth Observation, climate modeling -Geophysics, earthquake modeling -Fluids, aerodynamic design -Pollutant dispersal scenarios l Astronomy: Digital sky surveys

21. EO Grid Middleware National Virtual Observatory Xray (ROSAT) theme Change scale Change theme http://virtualsky.org/ from Caltech CACR Caltech Astronomy Microsoft Research Optical (DPOSS) Coma cluster Virtual Sky has 140,000,000 tiles 140 Gbyte

22. EO Grid Middleware Grid Physics Network (GriPhyN) Enabling R&D for advanced data grid systems, focusing in particular on Virtual Data concept ATLAS CMS LIGO SDSS www.griphyn.org; see also www.ppdg.net, www.eu-datagrid.org

23. EO Grid Middleware Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents Image courtesy Harvey Newman, Caltech

24. EO Grid Middleware Laser Interferometric Gravitational wave Observatory Listening to Collisions of Black Holes and Neutron Stars

25. EO Grid Middleware LIGO Hardware

26. EO Grid Middleware Grid LIGO Architecture Clients eg Web, Script, Agent Clients eg Web, Script, Agent Text request Request Manager Request Manager GriPhyNLDAS Gatekeeper (GRAM) Gatekeeper (GRAM) Science Algorithms Software Collaboratory Parallel Computing GridFTP Local Disk Data HPSS GridFTP Replica Catalog Replica Management Transformation Catalog Virtual Data Catalog Virtual Data Request Data Movement Globus RPC other LDAS Condor jobs

27. EO Grid Middleware iVDGL: A Global Grid Laboratory l International Virtual-Data Grid Laboratory -A global Grid laboratory (US, Europe, Asia, South America, …) -A place to conduct Data Grid tests “at scale” -A mechanism to create common Grid infrastructure -A laboratory for other disciplines to perform Data Grid tests -A focus of outreach efforts to small institutions l U.S. part funded by NSF (2001-2006) -$13.7M (NSF) + $2M (matching) “We propose to create, operate and evaluate, over a sustained period of time, an international research laboratory for data-intensive science.” From NSF proposal, 2001

28. EO Grid Middleware iVDGL Components l Computing resources -2 Tier1 laboratory sites (funded elsewhere) -7 Tier2 university sites software integration -3 Tier3 university sites outreach effort l Networks -USA (TeraGrid, Internet2, ESNET), Europe (Géant, …) -Transatlantic (DataTAG), Transpacific, AMPATH?, … l Grid Operations Center (GOC) -Joint work with TeraGrid on GOC development l Computer Science support teams -Support, test, upgrade GriPhyN Virtual Data Toolkit l Education and Outreach l Coordination, management

29. EO Grid Middleware iVDGL Components (cont.) l High level of coordination with DataTAG -Transatlantic research network (2.5 Gb/s) connecting EU & US l Current partners -TeraGrid, EU DataGrid, EU projects, Japan, Australia l Experiments/labs requesting participation -ALICE, CMS-HI, D0, BaBar, BTEV, PDC (Sweden)

30. EO Grid Middleware Initial US-iVDGL Data Grid Tier1 (FNAL) Proto-Tier2 Tier3 university UCSD Florida Wisconsin Fermilab BNL Indiana BU Other sites to be added in 2002 SKC Brownsville Hampton PSU JHU Caltech

31. EO Grid Middleware iVDGL Map (2002-2003) Tier0/1 facility Tier2 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link Tier3 facility DataTAG Surfnet Later  Brazil  Chile?  Pakistan  Russia  China

32. EO Grid Middleware The TeraGrid: 26 24 8 4 HPSS 5 UniTree External Networks Site Resources NCSA/PACI 8 TF 240 TB SDSC 4.1 TF 225 TB CaltechArgonne

33. EO Grid Middleware Summary l Grid infrastructure is becoming widespread -Major deployment based on common technology -Significant new deployment activities l Consensus building mechanisms in place -Global Grid Forum (www.gridforum.org) l Industrial buy in starting -IBM, Entropia, more to come

34. EO Grid Middleware For More Information l Book (Morgan Kaufman) -www.mkp.com/grids l Globus -www.globus.org -“The Anatomy of the Grid: Enabling Scalable Virtual Organizations” l GRIDS Center -www.grids-center.org l Grid Forum -www.gridforum.org