1. Copyright Gordon Bell Clusters & Grids The CC – GRID? Era Infinite processing, storage, and bandwidth @ zero cost and latency Gordon Bell (gbell@microsoft.com) Bay Area Research Center Microsoft Corporation

2. Copyright Gordon Bell Clusters & Grids

3. deja’ vu ARPAnet: c1969 – To use remote programs & data – Got FTP & mail. Machines & people overloaded. NREN: c1988 – BW => Faster FTP for images, data – Latency => Got http://www… – Tomorrow => Gbit communication BW, latency <’90 Mainframes, minis, PCs/WSs >’90 very large, dep’t, & personal clusters VAX: c1979 one computer/scientist Beowulf: c1995 one computer/scientist 1960s batch: opti-use allocate, schedule,$ 2000s GRID: opti-use allocate, schedule, $ (… security, management, etc.)

4. Copyright Gordon Bell Clusters & Grids Some observations Clusters are purchased, managed, and used as a single, one room facility. Clusters are the “new” computers. They present unique, interesting, and critical problems… then Grids can exploit them. Clusters & Grids have little to do with one another… Grids use clusters! Clusters should be a good simulation of tomorrow’s Grid. Distributed PCs: Grids or Clusters? Perhaps some clusterable problems can be solved on a Grid… but it’s unlikely. – Lack of understanding clusters & variants – Socio-, political, eco- wrt to Grid.

5. Copyright Gordon Bell Clusters & Grids Some observations GRID was/is an exciting concept … – They can/must work within a community, organization, or project. What binds it? – “Necessity is the mother of invention.” Taxonomy… interesting vs necessity – Cycle scavenging and object evaluation (e.g. seti@home, QCD)seti@home – File distribution/sharing aka IP theft (e.g. Napster, Gnutella) – Databases &/or programs and experiments (astronomy, genome, NCAR, CERN) – Workbenches: web workflow chem, bio… – Single, large problem pipeline… e.g. NASA. – Exchanges… many sites operating together – Transparent web access aka load balancing – Facilities managed PCs operating as cluster!

6. Copyright Gordon Bell Clusters & Grids Grids: Why? The problem or community dictates a Grid Economics… thief or scavenger Research funding… that’s where the problems are

7. In a 5-10 years we can/will have: more powerful personal computers – processing 10-100x; multiprocessors-on-a-chip – 4x resolution (2K x 2K) displays to impact paper – Large, wall-sized and watch-sized displays – low cost, storage of one terabyte for personal use adequate networking? PCs now operate at 1 Gbps – ubiquitous access = today’s fast LANs – Competitive wireless networking One chip, networked platforms e.g. light bulbs, cameras Some well-defined platforms that compete with the PC for mind (time) and market share watch, pocket, body implant, home (media, set-top) Inevitable, continued cyberization… the challenge… interfacing platforms and people.

8. Copyright Gordon Bell Clusters & Grids SNAP … c1995 S calable N etwork A nd P latforms A View of Computing in 2000+ We all missed the impact of WWW! Gordon Bell Jim Gray

9. How Will Future Computers Be Built? Thesis: SNAP: Scalable Networks and Platforms Upsize from desktop to world-scale computer based on a few standard components Because: Moore’s law: exponential progress Standardization & Commoditization Stratification and competition When: Sooner than you think! Massive standardization gives massive use Economic forces are enormous

10. Copyright Gordon Bell Clusters & Grids Performance versus time for various microprocessors

11. Copyright Gordon Bell Clusters & Grids Volume drives simple, cost to standard platforms performanceperformance Stand-alone Desk tops PCs

12. Copyright Gordon Bell Clusters & Grids Computing SNAP built entirely from PCs Wide & Local Area Networks for: terminal, PC, workstation, & servers Centralized & departmental uni- & mP servers (UNIX & NT) Legacy mainframes & minicomputers servers & terms Wide-area global network Legacy mainframe & minicomputer servers & terminals Centralized & departmental servers buit from PCs scalable computers built from PCs TC=TV+PC home... (CATV or ATM or satellite) ??? Portables A space, time (bandwidth), & generation scalable environment Person servers (PCs) Person servers (PCs) Mobile Nets

13. Copyright Gordon Bell Clusters & Grids SNAP Architecture----------

14. Copyright Gordon Bell Clusters & Grids GB plumbing from the baroque: evolving from the 2 dance-hall model Mp — S — Pc : | : |——————-- S.fiber ch. — Ms | : |— S.Cluster |— S.WAN — vs, MpPcMs — S.Lan/Cluster/Wan — :

15. Copyright Gordon Bell Clusters & Grids Modern scalable switches … also hide a supercomputer Scale from <1 to 120 Tbps 1 Gbps ethernet switches scale to 10s of Gbps, scaling upward SP2 scales from 1.2

16. Copyright Gordon Bell Clusters & Grids Interesting “cluster” in a cabinet 366 servers per 44U cabinet – Single processor – 2 - 30 GB/computer (24 TBytes) – 2 - 100 Mbps Ethernets ~10x perf*, power, disk, I/O per cabinet ~3x price/perf Network services… Linux based *42, 2 processors, 84 Ethernet, 3 TBytes

17. Copyright Gordon Bell Clusters & Grids High Performance Computing A 60+ year view

18. Copyright Gordon Bell Clusters & Grids High performance architecture/program timeline 1950.1960.1970.1980.1990.2000 VtubesTrans.MSI(mini) Micro RISCnMicr Sequential programming---->------------------------------ (single execution stream) <SIMD Vector--//--------------- Parallelization--- Parallel programs aka Cluster Computing <--------------- multicomputers <--MPP era------ ultracomputers 10X in size & price!10x MPP “in situ” resources 100x in //sm NOW CC geographically dispersedGrid

19. Copyright Gordon Bell Clusters & Grids Computer types Netwrked Supers… Legion Condor Beowulf NT clusters VPPuni T3E SP2 (mP) NOW NEC mP SGI DSM clusters & SGI DSM NEC super Cray X…T (all mPv) Mainframes Multis WSs PCs -------- Connectivity-------- WAN/LAN SAN DSM SM micros vector Clusters GRID Old World

20. Copyright Gordon Bell Clusters & Grids ISTORE Hardware Vision System-on-a-chip enables computer, memory, without significantly increasing size of disk 5-7 year target: MicroDrive:1.7” x 1.4” x 0.2” 2006: ? 1999: 340 MB, 5400 RPM, 5 MB/s, 15 ms seek 2006: 9 GB, 50 MB/s ? (1.6X/yr capacity, 1.4X/yr BW) Integrated IRAM processor 2x height Connected via crossbar switch growing like Moore’s law 16 Mbytes; ; 1.6 Gflops; 6.4 Gops 10,000+ nodes in one rack! 100/board = 1 TB; 0.16 Tf

21. Copyright Gordon Bell Clusters & Grids The Disk Farm? or a System On a Card? The 500GB disc card An array of discs Can be used as 100 discs 1 striped disc 50 FT discs....etc LOTS of accesses/second of bandwidth A few disks are replaced by 10s of Gbytes of RAM and a processor to run Apps!! 14"

22. Copyright Gordon Bell Clusters & Grids The Network Revolution Networking folks are finally streamlining LAN case (SAN). Offloading protocol to NIC ½ power point is 8KB Min round trip latency is ~50 µs. 3k ins +.1 ins/byte High-Performance Distributed Objects over a System Area Network Li, L. ; Forin, A. ; Hunt, G. ; Wang, Y., MSR-TR-98-68

23. Telnet & FTP EMAIL WWW AudioVideo Voice! Standards Increase Capacity (circuits & bw) Lower response time Create new service Increased Demand The virtuous cycle of bandwidth supply and demand

24. Copyright Gordon Bell Clusters & Grids 744Mbps over 5000 km to transmit 14 GB ~ 4e15 bit meters per second 4 Peta Bmps (“peta bumps”) Single Stream tcp/ip throughput Information Sciences Institute Microsoft QWest University of Washington Pacific Northwest Gigapop HSCC (high speed connectivity consortium) DARPA

25. Copyright Gordon Bell Clusters & Grids Map of Gray Bell Prize results Redmond/Seattle, WA San Francisco, CA New York Arlington, VA 5626 km 10 hops

26. Copyright Gordon Bell Clusters & Grids 1 GBps Ubiquitous 10 GBps SANs in 5 years 1Gbps Ethernet are reality now. – Also FiberChannel,MyriNet, GigaNet, ServerNet,, ATM,… 10 Gbps x4 WDM deployed now (OC192) – 3 Tbps WDM working in lab In 5 years, expect 10x !! 5 MBps 20 MBps 40 MBps 80 MBps 120 MBps (1Gbps)

27. Copyright Gordon Bell Clusters & Grids The Promise of SAN/VIA:10x in 2 years http://www.ViArch.org/ http://www.ViArch.org/ Yesterday: – 10 MBps (100 Mbps Ethernet) – ~20 MBps tcp/ip saturates 2 cpus – round-trip latency ~250 µs Now – Wires are 10x faster Myrinet, Gbps Ethernet, ServerNet,… – Fast user-level communication - tcp/ip ~ 100 MBps 10% cpu - round-trip latency is 15 us 1.6 Gbps demoed on a WAN

28. Copyright Gordon Bell Clusters & Grids Labscape 1 st, 2 nd, 3 rd, or New Paradigm for science?

29. Copyright Gordon Bell Clusters & Grids Labscape

30. Copyright Gordon Bell Clusters & Grids Labscape

31. Copyright Gordon Bell Clusters & Grids Labscape sensors Location tracking of people/samples – multiple resolutions – passive and active tags Manual tasks (e.g., use of reagents, tools) Audio/video records, vision and indexing Networked instruments (e.g., pipettes, refrigerators, etc.)

32. Courtesy of Dr. Thomas Sterling, Caltech

33. Lessons from Beowulf An experiment in parallel computing systems Established vision- low cost high end computing Demonstrated effectiveness of PC clusters for some (not all) classes of applications Provided networking software Provided cluster management tools Conveyed findings to broad community Tutorials and the book Provided design standard to rally community!* Standards beget: books, trained people, software … virtuous cycle* *observations Courtesy, Thomas Sterling, Caltech.

34. Copyright Gordon Bell Clusters & Grids The End How can GRIDs become a non- ad hoc computer structure? Get yourself an application community!