Cluster Computing: Advantages Better price/performance ratios Better availability Survive node failures with appropriate software support Impressive scaling 100s to 1000s of processors Easy and economical upgrading and technology migration.
Cluster Computing: Communication Parallel computation is communication intensive Dominant contributors for message latency Large messages: Bandwidth Small messages: Software Overhead Cluster communication: small messages Software overhead on traditional networks due to: Multiple data copies through protocol layers OS interrupts and context switching.
SAN Design Goals Minimize latency Avoid operating system by providing user application direct access to the network interface Avoid multiple copies of data High Bandwidth
SAN Design and Examples Key Issues General Architecture Example 1: Virtual Interface Architecture (VIA) Example 2: Memory Channel
Key Issues Design of direct user access to network interface Multiplexing between processes for access to network interface Providing security: avoiding inter-process interference Managing limited communication resources Efficient programming interface
General Architecture Communication model Virtual Interface Architecture (VIA) User-Level Network Interface (U-Net) Virtual Memory Mapped Communication (VMMC) Network Interconnect (NI) Memory Channel 1 & 2 (DIGITAL) ServerNet-II (Compaq)
Virtual Interface Architecture (VIA): An Overview Promoted by Compaq, Intel and Microsoft Defines an industry-standard architecture for communication within clusters Increase scalability Allows a single parallel application to run efficiently over dozens of clustered nodes Speeds up exchange of data between distributed application modules running on different servers
VIA: Goals Protected communication Latencies like memory operation Eliminate intermediate data copies Avoid traps into the OS Eliminate driver running in protected kernel mode Minimize number of instructions for send/receive Remove the reliance on interrupts for communication Keep architecture simple to allow software and hardware implementations Three types of services provided: Unreliable datagram service Reliable delivery service Reliable reception service
A Virtual Interface (VI) Creates illusion that a process has its own network interface. User Proces s Send Queue Receive Queue CNTL VI Network Interface User Proces s Send Queue Receive Queue CNTL VI Kerne l Agent Control and interrupts Per VI control and synchronization
VIA: Architecture Virtual Interface Represents a connection Linked list of descriptors Send, remote-DMA/write and remote-DMA/read on send queue Receive on receive queue Maintains data ordering for a connection using VI Asynchronous I/O between different VIs Kernel agent Unless required by user, not involved in communication Initializes tables and structures for communication Provides memory protection Pins physical memory for each connection. Necessary to avoid delay on receiver’s side due to address translation
Memory Channel 1: An Overview Developed at DIGITAL as a dedicated network interconnect Designed to improve communication performance in a cluster of at most 8 nodes Can support clusters as large as 8 nodes by 12 processors Implements Virtual Shared Memory Supports any computer using industry- standard peripheral-component interconnect (PCI) bus
Memory Channel 1: Goals Minimize cluster communication latencies Maintain the ability to repair hardware while cluster is running Strict message ordering Support full detection and recovery from communication errors Connect to PCI bus for compatibility with all PCI-based Alphas and Intel computers. Support broadcast, multicast and point-to-point communication
Memory Channel 1: Architecture (1) Address Space Mapping Memory-mapped circuit Connection between virtual-address space on sending node and a page of physical memory on receiver’s node. Write-only connection
Memory Channel 1: Architecture (2) Hardware Implementation MC Network I/O Bridge Transmit path Receive Path Processor Memory I/O Device I/O Bridge Transmit path Receive Path Processor Memory I/O Device PCI Bus System Bus System Bus PCI Bus Node 2 Node 1 MC Adaptor
SAN Applications File Server Clusters Database Server Clusters Storage Area Networks (SAN) Web Servers Clusters Scientific Applications Beowulf clusters
Conclusions Introduction to SAN SAN Design and examples VIA Memory Channel 1 SAN Applications