1. Parallel and Distributed Computing: Clusters and Grids Information Session
Subject Code:
WW Grid
Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australia

2. Scalable HPC: Breaking Administrative Barriers & new challenges
2100 ?
PERFORMANCE
2100
Administrative Barriers
Individual
Group
Department
Campus
State
National
Globe
Inter Planet
Galaxy
Desktop
SMPs or SuperComputers
Local
Cluster
Enterprise
Cluster/Grid
Global
Cluster/Grid
Inter Planetary
Grid!

3. Why SC? Large Scale Explorations need them—Killer Applications.
Solving grand challenge applications using modeling, simulation and analysis
Aerospace
Internet &
Ecommerce
Life Sciences
CAD/CAM
Digital Biology
Military Applications
Military Applications
Military Applications

5. PART 2: Cluster Architectures
The promise of supercomputing to the average PC User ? 3

6. HPCC Books, 2 Volumes - Prentice Hall, 1999 Edited by R
HPCC Books, 2 Volumes - Prentice Hall, Edited by R.Buyya with contributions from over 100 leading researchers (

7. Agenda Cluster ? Enabling Tech. & Motivations Cluster Architecture
Cluster Components
Single System Image
Next Section (after break)
Case Studies
Cluster Programming and Application Design
Resources and Conclusions

8. Rise and Fall of Computer Architectures
Vector Computers (VC) - proprietary system:
provided the breakthrough needed for the emergence of computational science, buy they were only a partial answer.
Massively Parallel Processors (MPP) -proprietary systems:
high cost and a low performance/price ratio.
Symmetric Multiprocessors (SMP):
suffers from scalability
Distributed Systems:
difficult to use and hard to extract parallel performance.
Clusters - gaining popularity:
High Performance Computing - Commodity Supercomputing
High Availability Computing - Mission Critical Applications

9. Cluster computing: Past, Present, Future
1960
1990
1995+
1980s
2000+
PDA
Clusters

10. Definition: What is a Cluster?
A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource.
“stand-alone” (whole computer) computer that can be used on its own (full hardware and OS).

11. So What’s So Different about Clusters?
Commodity Parts?
Communications Packaging?
Incremental Scalability?
Independent Failure?
Intelligent Network Interfaces?
Complete System on every node
virtual memory
scheduler
files …
Nodes can be used individually or combined...

12. Cluster Computer Architecture
Parallel Applications
Parallel Applications
Parallel Applications
Sequential Applications
Sequential Applications
Sequential Applications
Parallel Programming Environment
Cluster Middleware
(Single System Image and Availability Infrastructure)
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
Cluster Interconnection Network/Switch

13. A major issues in Cluster design
Enhanced Performance low cost)
Enhanced Availability (failure management)
Single System Image (look-and-feel of one system)
Size Scalability (physical & application)
Fast Communication (networks & protocols)
Load Balancing (CPU, Net, Memory, Disk)
Security and Encryption (clusters of clusters)
Distributed Environment (Social issues)
Manageability (admin. And control)
Programmability (simple API if required)
Applicability (cluster-aware and non-aware app.)

14. Scalability Vs. Single System ImageUP

15. Cluster Applications Numerous Scientific & engineering Apps.
Business Applications:
E-commerce Applications (Amazon, eBay ….);
Database Applications (Oracle on clusters).
Internet Applications:
ASPs (Application Service Providers);
Computing Portals;
E-commerce and E-business.
Mission Critical Applications:
command control systems, banks, nuclear reactor control, star-wars, and handling life threatening situations.

16. Science Portals - e.g., Papia system
Pentiums.
Myrinet.
NetBSD/Linuux.
PM.
Score-D.
MPC++.
RWCP -
Papia PC Cluster

17. Adoption of the Approach

18. Scalable HPC: Breaking Administrative Barriers & new challenges
2100 ?
PERFORMANCE
2100
Administrative Barriers
Individual
Group
Department
Campus
State
National
Globe
Inter Planet
Galaxy
Desktop
SMPs or SuperComputers
Local
Cluster
Enterprise
Cluster/Grid
Global
Cluster/Grid
Inter Planetary
Grid!

19. Towards Grid Computing

20. What is Grid ?
A paradigm/infrastructure that enabling the sharing, selection, & aggregation of geographically distributed resources:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices/instruments – e.g., radio telescope – searching for life in galaxy.
People/collaborators.
[depending on their availability, capability, cost, and user QoS requirements]
for solving large-scale problems/applications.
Thus enabling the creation of “virtual enterprises” (VEs)
Wide area

21. P2P/Grid Applications-Drivers
Distributed HPC (Supercomputing):
Computational science.
High-Capacity/Throughput Computing:
Large scale simulation/chip design & parameter studies.
Content Sharing (free or paid)
Sharing digital contents among peers (e.g., Napster)
Remote software access/renting services:
Application service provides (ASPs) & Web services.
Data-intensive computing:
Drug Design, Particle Physics, Stock Prediction...
On-demand, realtime computing:
Medical instrumentation & Mission Critical.
Collaborative Computing:
Collaborative design, Data exploration, education.
Service Oriented Computing (SOC):
Computing as Competitive Utility: New paradigm, new industries, and new business.

22. A Typical Grid Computing Environment
Grid Information Service
Grid Resource Broker
database
Application R2 R3 R4 R5 RN
Grid Resource Broker R6 R1
Resource Broker
Grid Information Service

23. Need Grid tools for managing
Uniform Access
Security
System Management
Computational Economy
Resource Discovery
Resource Allocation
& Scheduling
Data locality
Network Management
Application Development Tools

24. Grid Computing Approaches
mix-and-match
Object-oriented
Internet/partial-P2P
Grid Computing
Approaches
Network enabled Solvers
NetSolve
Market/Computational Economy
Nimrod-G

25. Many Grid Projects & Initiatives
Australia
Nimrod-G
GridSim
Virtual Lab
Active Sheets
DISCWorld
..new coming up
Europe
UNICORE
MOL
UK eScience
Poland MC Broker
EU Data Grid
EuroGrid
MetaMPI
Dutch DAS
XW, JaWS
Japan
Ninf
DataFarm
Korea...
N*Grid
USA
Globus
Legion
OGSA
Javelin
AppLeS
NASA IPG
Condor-G
Jxta
NetSolve
AccessGrid
and many more...
Cycle Stealing & .com Initiatives
Distributed.net ….
Entropia, UD, Parabon,….
Public Forums
Global Grid Forum
P2P Working Group
IEEE TFCC
Grid & CCGrid conferences

26. Grid Computing Projects
GRIDS Melbourne

27. The Gridbus Vision: To Enable Service Oriented Grid Computing & Bus iness!
WW Grid
Nimrod-G
World Wide Grid!

28. GRIDS Lab @ the U. of Melbourne, The Gridbus Project: www.gridbus.org
Grid Economy & Distributed Scheduling (via Nimrod-G Broker)
GridSim Toolkit: Grid Modeling and Simulation (Java based):
Libra: Economic Cluster Scheduler
Grid Bank: Accounting, Payment, Enforcement Mechanisms
World Wide Grid (WWG) testbed:
Application Enabler Projects:
Virtual Laboratory Toolset for Drug Design
High-Energy Physics and the Grid Network (HEPGrid)
Brain Activity Analysis on the Grid
Cluster and Grid Info Centres: ||

29. Nimrod/G : A Grid Resource Broker
A resource broker for managing, steering, and executing task farming (parameter sweep/SPMD model) applications on Grid based on deadline and computational economy.
Based on users’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability.
Key Features
A single window to manage & control experiment
Persistent and Programmable Task Farming Engine
Resource Discovery
Resource Trading
Scheduling & Predications
Generic Dispatcher & Grid Agents
Transportation of data & results
Steering & data management
Accounting

30. Drug Design: Data Intensive Computing on Grid
Protein
Molecules
Chemical Databases
(legacy, in .MOL2 format)
It involves screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential to serve as drug candidates.

31. [Collaboration with Osaka University, Japan]
MEG(MagnetoEncephaloGraphy) Data Analysis on the Grid: Brain Activity Analysis
64 sensors MEG
Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs 2
Data Generation 3 1
Data Analysis 5
Results
Nimrod-G 4
[deadline, budget, optimization preference]
Life-electronics laboratory,
AIST
World-Wide Grid
Provision of expertise in
the analysis of brain function
Provision of MEG analysis
[Collaboration with Osaka University, Japan]

32. A Glance at Nimrod-G Broker
Nimrod/G Client
Nimrod/G Client
Nimrod/G Client
Nimrod/G Engine
Schedule Advisor
Grid
Store
Trading Manager
Grid Dispatcher
Grid Explorer
Grid Middleware
Globus, Legion, Condor, etc.
TM TS
GE GIS
Grid Information Server(s)
RM & TS
RM & TS
RM & TS G C L G
Legion enabled node.
Globus enabled node. L G C L
See HPCAsia 2000 paper!
RM: Local Resource Manager, TS: Trade Server
Condor enabled node.

33. Active Sheet: Microsoft Excel Spreadsheet Processing on Grid
Nimrod
Proxy
Nimrod-G
World-Wide Grid

35. GridSim Toolkit A Java based tool for Grid Scheduling Simulations
Application, User, Grid Scenario’s Input and Results
Application Configuration
Resource Configuration
User Requirements
Grid
Scenario
. . .
Output
Grid Resource Brokers or Schedulers
GridSim Toolkit
Application Modeling
Resource Entities
Information
Services
Job Management
Resource Allocation
Statistics
Resource Modeling and Simulation (with Time and Space shared schedulers)
Single CPU
SMPs
Clusters
Load Pattern
Network
Reservation
Basic Discrete Event Simulation Infrastructure
SimJava
Distributed SimJava
Virtual Machine (Java, cJVM, RMI)
PCs
Workstations
SMPs
Clusters
Distributed Resources

36. Selected GridSim Users!

37. Fresco by N. Cianfanelli (1841)
Alessandro Volta in Paris in 1801 inside French National Institute shows the battery while in the presence of Napoleon I
Fresco by N. Cianfanelli (1841)
(Zoological Section "La Specula" of National History Museum of Florence University)

38. What ?!?! Oh, mon Dieu ! This is a mad man… ….and in the future,
I imagine a Worldwide
Power (Electrical) Grid …...
Oh, mon
Dieu !
What ?!?!
This is a mad man…

39. = 201 Years
1801
2002

40. Download Software & Information
Nimrod & Parameteric Computing:
Economy Grid & Nimrod/G:
Virtual Laboratory Toolset for Drug Design:
Grid Simulation (GridSim) Toolkit (Java based):
World Wide Grid (WWG) testbed:
Cluster and Grid Info Centres: ||

41. Further Information Books: IEEE Task Force on Cluster Computing
High Performance Cluster Computing, V1, V2, R.Buyya (Ed), Prentice Hall, 1999.
The GRID, I. Foster and C. Kesselman (Eds), Morgan-Kaufmann, 1999.
IEEE Task Force on Cluster Computing
Global Grid Forum
IEEE/ACM CCGrid’xy:
CCGrid 2002, Berlin: ccgrid2002.zib.de
Grid workshop -

42. Further Information Cluster Computing Info Centre:
Grid Computing Info Centre:
IEEE DS Online - Grid Computing area:
Compute Power Market Project

43. Final Word?

44. Backup Slides