1. 2. Computer Clusters for Scalable Parallel Computing
2.1 Clustering for Massive Parallelism
A computer cluster is a collection of interconnected stand-alone computers which can work together collectively and cooperatively as a single integrated computing resource pool.
Clustering explores massive parallelism at the job level and achieves high availability (HA) through stand-alone operations.
2.1.1 Cluster Development Trends
Milestone Cluster Systems
hot research challenge
fast communication
job scheduling
SSI HA
2.1.2 Design Objectives of Computer Clusters
Ch 2-1. Computer Cluster

2. Dedicated vs. Enterprise Clusters
Scalability
Packaging
Cluster nodes can be packaged in a compact or a slack fashion.
In a compact cluster, the nodes are closely packaged in one or more racks sitting in a rooms.
In a slack cluster, the nodes are attached to their usual peripherals, and they may be located in different rooms, different buildings, or even remote regions.
Packaging directly affects communication wire length, and thus the selection of interconnection technology used.
Control
A cluster can be either controlled or managed in centralized or decentralized fashion.
A compact cluster normally has centralized control, while a slack cluster can be controlled either way.
Homogeneity
Security
Intracluster communication can be either exposed or enclosed.
Dedicated vs. Enterprise Clusters
An enterprise cluster is mainly used to utilize idle resources in the nodes.
Ch 2-1. Computer Cluster

3. 2.1.3 Fundamental Cluster Design Issues
Scalable Performance
Single-System Image (SSI)
Availability Support
Clusters can provide cost-effective HA capability with lots of redundancy in processors, memory, disks, I/O devices, networks, and operating system images.
Cluster Job Management
Job management software is required to provide batching, load balancing, parallel processing, and other functionality.
Internode Communication
Fault Tolerance and Recovery
Cluster Family Classification
Computer cluster
These are clusters designed mainly for collective computation over a single large job.
High-Availability clusters
HA clusters are designed to be fault-tolerant and achieve HA of service.
HA clusters operate with many redundant nodes to sustain faults or failures.
Load-balancing clusters
These clusters shoot for higher resource utilization through load balancing among all participating nodes in the cluster.
Ch 2-1. Computer Cluster

4. 2.2 Computer Cluster and MPP Architecture
2.1.4 Analysis of the Top 500 Supercomputers
Architecture Evaluation
SMP  MPP  cluster systems
Speed Improvement over Time
Pflops (Peta flops): 1,000조 부동소수점 명령/초
Performance plot of the Top 500 supercomputer from 1993 to (Fig. 2.2)
Operating System Trends in the Top 500
Linux (82%)
The Top Five Systems in 2010
Table 2.2
Country Share and Application Share
Major increases of supercomputer applications are in the area of database, research, finance, and information services.
2.2 Computer Cluster and MPP Architecture
2.2.1 Cluster Organization and Resource Sharing
A Basic Cluster Architecture
Ch 2-1. Computer Cluster

5. 2.2.3 Cluster System Interconnects
Figure 2.4 shows the basic architecture of a computer cluster over PCs or workstations.
Resource Sharing in Clusters
Fig, 2.5
The shared-nothing configuration in Part (a) simply connects two or more autonomous computers via a LAN such as Ethernet.
A shared-disk cluster is shown in Part (b). This is what most business clusters desire so that they can enable recovery support in case of node failure.
The shared-memory cluster in Part (c) is much more difficult to realize. The nodes could be connected by a scalable coherence interface (SCI) ring, which is connected to the memory bus of each node through an NIC module.
Node Architecture and MPP Packaging
Cluster nodes are classified into two categories: compute nodes and service nodes
Table 2.3 introduces two example compute node architecture: homogeneous design and hybrid node design.
2.2.3 Cluster System Interconnects
High-Bandwidth Interconnects
Ch 2-1. Computer Cluster

6. 2.2.4 Hardware, Software, and Middleware Support
Crossbar Switch in Google Search Engine Cluster (Fig. 2.7)
Share of System Interconnects over Time
Distribution of large-scale system interconnects (Fig. 2.8).
The InfiniBand system fabric built in a typical high-performance computer cluster (Fig. 2.9).
2.2.4 Hardware, Software, and Middleware Support
The middleware, OS extensions, and hardware support needed to achieve HA in a typical Linux cluster system (Fig. 2.10).
2.2.5 GPU Clusters for Massive Parallelism
GPU Cluster Components
A GPU cluster is often built as a heterogeneous system consisting of three major components: the CPU host nodes, the GPU nodes and the cluster interconnect between them.
Ch 2-1. Computer Cluster

7. Academic Cluster Computing Initiative
Ch 2-1. Computer Cluster