1. The Grid: The First 50 Years Ian Foster Argonne National Laboratory University of Chicago Carl Kesselman Information Sciences Institute University of Southern California

2. 2 Abstract The Internet and Web have had a major impact on society: by allowing us to discover and access information on a global scale, they have created entirely new businesses and brought new meaning to the term "surf." Yet simply being able to "show you stuff" and "know stuff" are ultimately unsatisfactory: we want to "do stuff"—and increasingly, to "do stuff together" within distributed teams. This need has lead to the creation of the Grid, an infrastructure that enables us to share capabilities, integrating services and resources within and across enterprises, and allowing active collaborations across distributed, multi-organizational collaborations. Powered by on-demand access to computing, seamless access to data, and dynamic composition of distributed services, the Grid promises to enable fundamentally new ways of interacting with our information technology infrastructure, of doing business and practicing science. It represents perhaps the final step in the great disappearing act that will take computing out of our homes and machine rooms and into the fabric of society, where it will stand alongside telephone switches, power generators, and the other invisible technologies that drive the modern world. In this talk, we trace the history of Grid ideas, starting with the first days of the Internet in the early 1970s, proceeding to the work in the mid to late 1990s that created today's Grid, and finally looking some years into the future. We talk about accomplishments, opportunities, obstacles, and challenges--and, we hope, provide a compelling picture of the dynamic and open mix of ideas, technology, and community that are turning the Grid into reality.

3. 3 Licklider (1960): Man-Computer Symbiosis… … is an expected development in cooperative interaction between men and electronic computers. The main aims are 1.to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and 2.to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs.

4. 4 Problem Solving in the 21 st Century l Teams organized around common goals –Communities: “Virtual organizations” l With diverse membership & capabilities –Heterogeneity is a strength not a weakness l And geographic and political distribution –No location/organization possesses all required skills and resources l Must adapt as a function of the situation –Adjust membership, reallocate responsibilities, renegotiate resources

5. 5 Context (1): Revolution in Science l Pre-Internet –Theorize &/or experiment, alone or in small teams; publish paper l Post-Internet –Construct and mine large databases of observational or simulation data –Develop simulations & analyses –Access specialized devices remotely –Exchange information within distributed multidisciplinary teams

6. 6 Context (2): Revolution in Business l Pre-Internet –Central data processing facility l Post-Internet –Enterprise computing is highly distributed, heterogeneous, inter-enterprise (B2B) –Business processes increasingly computing- & data-rich –Outsourcing becomes feasible => service providers of various sorts

7. 7 The (Power) Grid: On-Demand Access to Electricity Time Quality, economies of scale

8. 8 By Analogy, A Computing Grid l Decouple production and consumption –Enable on-demand access –Achieve economies of scale –Enhance consumer flexibility –Enable new devices l On a variety of scales –Department –Campus –Enterprise –Internet

9. 9 Not Exactly a New Idea … l “The time-sharing computer system can unite a group of investigators …. one can conceive of such a facility as an … intellectual public utility.” –Fernando Corbato and Robert Fano, 1966 l “We will perhaps see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country.” –Len Kleinrock, 1967

10. 10 But Things are Different Now …

11. 11 Computing isn’t Really Like Electricity l I import electricity but must export data l “Computing” is not interchangeable but highly heterogeneous: data, sensors, services, … l This complicates things; but also means that the sum can be greater than the parts –Real opportunity: Construct new capabilities dynamically from distributed services l Raises fundamental questions –Achieving economies of scale –Quality of service across distributed services –Applications that exploit synergies

12. 12 New Opportunities Demand New Technology “ Resource sharing & coordinated problem solving in dynamic, multi- institutional virtual organizations” “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder)

13. 13 Taking Sharing to the Next Level l Sharing of communication –Telephones, mailing lists, collaboration tools l Sharing of data and knowledge –Web, semantic web l What about the rest of the infrastructure? –Services, computers, programs, sensors, …

14. 14 Existing Technologies are Helpful, but Not Complete Solutions l Peer-to-peer technologies –Limited scope and mechanisms l Enterprise-level distributed computing –Limited cross-organizational support l Databases –Vertically integrated solutions l Web services –Not dynamic l Semantic web –Limited focus

15. 15 What’s Missing is Support for … l Sharing & integration of resources, via –Discovery –Provisioning –Access (computation, data, …) –Security –Policy –Fault tolerance –Management l In dynamic, scalable, multi-organizational settings

16. 16 Enter the Grid l Infrastructure (“middleware”) for establishing, managing, and evolving multi-organizational federations –Dynamic, autonomous, domain independent –On-demand, ubiquitous access to computing, data, and services l Mechanisms for creating and managing workflow within such federations –New capabilities constructed dynamically and transparently from distributed services –Service-oriented, virtualization

17. 17 Building the Grid l Open source software –Globus Toolkit ®, UK OGSA DAI, Condor, … l Open standards –OGSA, other GGF, IETF, W3C standards, … l Open communities –Global Grid Forum, Globus International, collaborative projects, … l Open infrastructure –UK eScience, NSF Cyberinfrastructure, StarLight, AP-Grid, …

18. 18 Open Source l Encourage adoption of standards by reducing barriers to entry –Overcome new technology Catch-22 l Enable broad Grid technology ecosystem –Key to international cooperation –Key to science-commerce partnership l Jumpstart Grid industry and allow vendors to focus on value-add –E.g., IBM, Avaki use GT3; Platform Globus –Open source is industry friendly!

19. 19 Globus Toolkit ® History DARPA, NSF, and DOE begin funding Grid work NASA begins funding Grid work, DOE adds support The Grid: Blueprint for a New Computing Infrastructure published GT 1.0.0 Released Early Application Successes Reported NSF & European Commission Initiate Many New Grid Projects Anatomy of the Grid Paper Released Significant Commercial Interest in Grids Physiology of the Grid Paper Released GT 2.0 Released Does not include downloads from: NMI, UK eScience, EU Datagrid, IBM, Platform, etc.

20. 20 Increased functionality, standardization Custom solutions 1990199520002005 Open Grid Services Arch Real standards Multiple implementations Web services, etc. Managed shared virtual systems Computer science research Globus Toolkit Defacto standard Single implementation Internet standards The Emergence of Open Grid Standards 2010

21. 21 Grid Communities l Global Grid Forum –Standards, information exchange, advocacy –1000+ participants in tri-annual meetings l Application communities –E.g., physics, earthquake engineering, biomedical, etc. l Software development and support –NSF Middleware Initiative, UK eScience, Globus Toolkit, EGEE, …

22. 22 Data Grids for High Energy Physics l Enable international community of 1000s to access & analyze petabytes of data l Harness computing & storage worldwide l Virtual data concepts: manage programs, data, workflow l Distributed system management

23. 23 myGrid (Goble, De Roure, Shadbolt, et al.) l Imminent data deluge in bioinformatics l Heterogeneous, complex, and inter- related data sources l Integrated, community -wide treatment of data, literature, computational services

24. 24 NEESgrid Earthquake Engineering Collaboratory U.Nevada Reno www.neesgrid.org

25. 25 In flight data Airline Maintenance Centre Ground Station Global Network eg: SITA Internet, e-mail, pager DS&S Engine Health Center Data centre Distributed Aircraft Maintenance Environment (Austin et al.)

26. 26 “Gridified” Infrastructure Industrial Perspective on Grids: A Wide Range of Applications Financial Services Derivatives Analysis Statistical Analysis Portfolio Risk Analysis Derivatives Analysis Statistical Analysis Portfolio Risk Analysis Manufacturing Mechanical/ Electronic Design Process Simulation Finite Element Analysis Failure Analysis Mechanical/ Electronic Design Process Simulation Finite Element Analysis Failure Analysis LS / Bioinformatics Cancer Research Drug Discovery Protein Folding Protein Sequencing Cancer Research Drug Discovery Protein Folding Protein Sequencing Other Web Applications Weather Analysis Code Breaking/ Simulation Academic Web Applications Weather Analysis Code Breaking/ Simulation Academic Sources: IDC, 2000 and Bear Stearns- Internet 3.0 - 5/01 Analysis by SAI Grid Services Market Opportunity 2005 Unique by Industry with Common Characteristics Energy Seismic Analysis Reservoir Analysis Seismic Analysis Reservoir Analysis Entertainment Digital Rendering Massive Multi-Player Games Massive Multi-Player Games Streaming Media

27. 27 Open Infrastructure l Broadly deployed services in support of fundamental collaborative activities –Formation & operation of virtual organizations –Authentication, authorization, discovery, … l Services, software, and policies enabling on- demand access to critical resources –Computers, databases, networks, storage, software services,… l Operational support for 24x7 availability l Integration with campus and commercial infrastructures

28. 28 Tier0/1 facility Tier2 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link Tier3 facility Open Infrastructure Cambridge Newcastle Edinburgh Oxford Glasgow Manchester Cardiff Soton London Belfast DL RAL Hinxton

29. 29 Grid Communities & Technologies l Yesterday –Small, static communities, primarily in science –Focus on sharing of computing resources –Globus Toolkit as technology base l Today –Larger communities in science; early industry –Focused on sharing of data and computing –Open Grid Services Architecture l Tomorrow –Large, dynamic, diverse communities that share a wide variety of services, resources, data –Challenging computer science research issues

30. 30 Summary: The Grid Explained, via the BCS Lovelace Medal “The complexity of the problems facing mankind is growing faster than our ability to solve them. Finding ways to augment our intellect is both a necessary & desirable goal.” Doug Engelbart Linus Torvalds Openness—in software, standards, and community —can be a powerful accelerator of progress New capabilities via resource sharing within distributed virtual organizations … … enabled by open software & standards within an international community.

31. Questions?