1. IPPS 981 What’s So Different about Cluster Architectures? David E. Culler Computer Science Division U.C. Berkeley http://now.cs.berkeley.edu

2. IPPS 982 High Performance Clusters “happen” Many groups have built them. Many more are using them. Industry is running with it –Virtual Interface Architecture –System Area Networks A powerful, flexible new design technique

3. IPPS 983 Outline Quick “guided tour” of Clusters at Berkeley Three Important Advances => Virtual Networks Alan Mainwaring => Implicit Co-scheduling Andrea Arpaci-Dusseau => Scalable I/O Remzi Arpaci-Dusseau What it means

4. IPPS 984 Stop 1: HP/fddi Prototype FDDI on the HP/735 graphics bus. First fast msg layer on non-reliable network

5. IPPS 985 Stop 2: SparcStation NOW ATM was going to take over the world. The original INKTOMI

6. IPPS 986 Stop 3: Large Ultra/Myrinet NOW

7. IPPS 987 Stop 4: Massive Cheap Storage Basic unit: 2 PCs double-ending four SCSI chains Currently serving Fine Art at http://www.thinker.org/imagebase/

8. IPPS 988 Stop 5: Cluster of SMPs (CLUMPS) Four Sun E5000s –8 processors –3 Myricom NICs Multiprocessor, Multi- NIC, Multi-Protocol –see S. Lumetta IPPS98

9. IPPS 989 Stop 6: Information Servers Basic Storage Unit: – Ultra 2, 300 GB raid, 800 GB tape stacker, ATM –scalable backup/restore Dedicated Info Servers –web, –security, –mail, … VLANs project into dept.

10. IPPS 9810 Stop 7: Millennium PC Clumps Inexpensive, easy to manage Cluster Replicated in many departments Prototype for very large PC cluster

11. IPPS 9811 So What’s So Different? Commodity parts? Communications Packaging? Incremental Scalability? Independent Failure? Intelligent Network Interfaces? Complete System on every node –virtual memory –scheduler –files –...

12. IPPS 9812 Three important system design aspects Virtual Networks Implicit co-scheduling Scalable File Transfer

13. IPPS 9813 Communication Performance  Direct Network Access LogP: Latency, Overhead, and Bandwidth Active Messages: lean layer supporting programming models Latency1/BW

14. IPPS 9814 General purpose requirements Many timeshared processes –each with direct, protected access User and system Client/Server, Parallel clients, parallel servers –they grow, shrink, handle node failures Multiple packages in a process –each may have own internal communication layer Use communication as easily as memory

15. IPPS 9815 Virtual Networks Endpoint abstracts the notion of “attached to the network” Virtual network is a collection of endpoints that can name each other. Many processes on a node can each have many endpoints, each with own protection domain.

16. IPPS 9816 Process 3 How are they managed? How do you get direct hardware access for performance with a large space of logical resources? Just like virtual memory –active portion of large logical space is bound to physical resources Process n Process 2 Process 1 *** Host Memory Processor NIC Mem Network Interface P

17. IPPS 9817 Endpoint Transition Diagram COLD Paged Host Memory WARM R/O Paged Host Memory HOT R/W NIC Memory Read Evict Swap Write Msg Arrival

18. IPPS 9818 Network Interface Support NIC has endpoint frames Services active endpoints Signals misses to driver –using a system endpont Frame 0 Frame 7 Transmit Receive EndPoint Miss

19. IPPS 9819 Solaris System Abstractions Segment Driver manages portions of an address space Device Driver manages I/O device Virtual Network Driver

20. IPPS 9820 LogP Performance Competitive latency Increased NIC processing Difference mostly –ack processing –protection check –data structures –code quality Virtualization cheap

21. IPPS 9821 Bursty Communication among many Client Server Msg burst work

22. IPPS 9822 Multiple VN’s, Single-thread Server

23. IPPS 9823 Multiple VNs, Multithreaded Server

24. IPPS 9824 Perspective on Virtual Networks Networking abstractions are vertical stacks –new function => new layer –poke through for performance Virtual Networks provide a horizontal abstraction –basis for build new, fast services

25. IPPS 9825 Beyond the Personal Supercomputer Able to timeshare parallel programs –with fast, protected communication Mix with sequential and interactive jobs Use fast communication in OS subsystems –parallel file system, network virtual memory, … Nodes have powerful, local OS scheduler Problem: local schedulers do not know to run parallel jobs in parallel

26. IPPS 9826 Local Scheduling Schedulers act independently w/o global control Program waits while trying communicate with its peers that are not running 10 - 100x slowdowns for fine-grain programs! => need coordinated scheduling

27. IPPS 9827 Explicit Coscheduling Global context switch according to precomputed schedule How do you build it? Does it work?

28. IPPS 9828 Typical Cluster Subsystem Structures A LS AA A A A Master A LS A GS A LS GS A LS A GS LS A GS Local service Applications Communication Global Service Communication Master-Slave Peer-to-Peer

29. IPPS 9829 Ideal Cluster Subsystem Structure Obtain coordination without explicit subsystem interaction, only the events in the program –very easy to build –potentially very robust to component failures –inherently “service on-demand” –scalable Local service component can evolve. A LS A GS A LS GS A LS A GS LS A GS

30. IPPS 9830 Three approaches examined in NOW GLUNIX explicit master-slave (user level) –matrix algorithm to pick PP –uses stops & signals to try to force desired PP to run Explicit peer-peer scheduling assist with VNs –co-scheduling daemons decide on PP and kick the solaris scheduler Implicit –modify the parallel run-time library to allow it to get itself co- scheduled with standard scheduler A LS AA A A A M A A GS A LS GS A LS A GS LS A GS A LS A GS A LS GS A LS A GS LS A GS

31. IPPS 9831 Problems with explicit coscheduling Implementation complexity Need to identify parallel programs in advance Interacts poorly with interactive use and load imbalance Introduces new potential faults Scalability

32. IPPS 9832 Why implicit coscheduling might work Active message request-reply model Infer non-local state from local observations; react to maintain coordination observationimplication action fast response partner scheduledspin delayed response partner not scheduledblock WS 1 Job A WS 2 Job BJob A WS 3 Job BJob A WS 4 Job BJob A sleep spin requestresponse

33. IPPS 9833 Obvious Questions Does it work? How long do you spin? What are the requirements on the local scheduler?

34. IPPS 9834 How Long to Spin? Answer: round trip time + 5 x wake-up time –round-trip to stay scheduled together –plus wake-up to get scheduled together –plus wake-up to be competitive with blocking cost –plus 3 x wake-up to meet “pairwise” cost

35. IPPS 9835 Does it work?

36. IPPS 9836 Synthetic Bulk-synchronous Apps Range of granularity and load imbalance –spin wait 10x slowdown

37. IPPS 9837 With mixture of reads Block-immediate 4x slowdown

38. IPPS 9838 Timesharing Split-C Programs

39. IPPS 9839 Many Questions What about –mix of jobs? –sequential jobs? –unbalanced placement? –Fairness? –Scalability? How broadly can implicit coordination be applied in the design of cluster subsystems?

40. IPPS 9840 A look at Serious File I/O Traditional I/O system NOW I/O system Benchmark Problem: sort large number of 100 byte records with 10 byte keys –start on disk, end on disk –accessible as files (use the file system) –Datamation sort: 1 million records –Minute sort: quantity in a minute Proc- Mem P-M

41. IPPS 9841 NOW-Sort Algorithm: 1 pass Read –N/P records from disk -> memory Distribute –send keys to processors holding result buckets Sort –partial radix sort on each bucket Write –gather and write records to disk

42. IPPS 9842 Key Implementation Techniques Performance Isolation: highly tuned local disk- to-disk sort –manage local memory –manage disk striping –memory mapped I/O with m-advise, buffering –manage overlap with threads Efficient Communication –completely hidden under disk I/O –competes for I/O bus bandwidth Self-tuning Software –probe available memory, disk bandwidth, trade-offs

43. IPPS 9843 World-Record Disk-to-Disk Sort Sustain 500 MB/s disk bandwidth and 1,000 MB/s network bandwidth

44. IPPS 9844 Towards a Cluster File System Remote disk system built on a virtual network Client RDlib RD server Active msgs

45. IPPS 9845 Streaming Transfer Experiment

46. IPPS 9846 Results Data distribution affects resource utilization Not delivered bandwidth

47. IPPS 9847 I/O Bus crossings

48. IPPS 9848 Conclusions Complete system on every node makes clusters a very powerful architecture. Extend the system globally –virtual memory systems, –schedulers, –file systems,... Efficient communication enables new solutions to classic systems challenges. Opens a rich set of issues for parallel processing beyond the personal supercomputer.