1. Architectural Models for Resource Management in the Grid
Rajkumar Buyya
Monash University, Australia
Steve Chapin,
Syracuse University, USA
David DiNucci
Elepar.com, USA

2. Outline Changes in Computing Landscape Resource Management Issues
Architectural Models
Hierarchical Resource Management
Abstract Owner
Market Model
Economy Grid
Nimrod/G Resource Broker
Conclusions

3. Computing Power (HPC) Drivers
Solving grand challenge applications using computer modeling, simulation and analysis
Aerospace
Internet &
Ecommerce
Life Sciences
Digital Biology
CAD/CAM
Military Applications
Military Applications
Military Applications

4. SMPs or SuperComputers
Computing Platforms
2100 ?
PERFORMANCE
2100
Breaking Administrative Barriers
Individual
Group
Department
Campus
State
National
Globe
Inter Planet
Universe
Desktop
SMPs or SuperComputers
Local
Cluster
Enterprise
Cluster/Grid
Global
Cluster/Grid
Inter Planet
Cluster/Grid ??

5. Towards Grid Computing
Unification of geographically distributed resources

6. What is Grid ? An infrastructure that couples:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks,
mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices – e.g., radio telescope – searching for life in galaxy, for pulsars)
People/collaborators.
& offers dependable, consistent, & pervasive access to resources.

7. A Example Grid Infrastructure

8. Sources of Complexity in Grid Resource Management
No single administrative control.
No single ownership policy:
Each resource owner has their own policies or scheduling mechanisms;
Users must honour them (particularly external Grid users).
Heterogeneity of resources.
Dynamic availability – may appear and disappear…

9. Sources of Complexity in Grid Resource Management
Unreliable resource – disappear from view!
No uniform cost model - varies from one user’s resource to another and from time of day.
No single access mechanism – Web, custom interfaces, command line…

10. Grid Resource Management Issues
Authentication (once).
Specify (code, resources, etc.).
Discover resources.
Negotiate authorization, acceptable use, Cost, etc.
Acquire resources.
Schedule Jobs.
Initiate computation.
Steer computation.
Access remote data-sets.
Collaborate with results.
Account for usage.
Discover resources.
Negotiate authorisation,
acceptable use, Cost, etc.
Acquire resources.
Schedule jobs.
Initiate computation.
Steer computation.
Domain 1
Domain 2
Ack: Globus..

11. Architectural Models MODEL REMARKS Systems Hierarchical
It captures model followed in most contemporary systems.
Globus, Legion, CCS, Apples, NetSolve, Ninf.
Abstract Owner (AO)
Order and delivery model and focuses on long term goals.
Expected to emerge and most peer-2-peer computing systems likely to be based on this.
Market Model
It follows economic model for resource discover, sharing, & scheduling.
GRACE, Nimrod/G, JavaMarket, Mariposa.

12. Hierarchical Resource Management
Access/Admission Control Agent
User
Global
Scheduler
Global
Scheduler
Grid Information Service
Persistent Job Control
Agent
Connection Cloud
Global
Scheduler
Monitor
Global
Scheduler
Global
Scheduler
Local
Scheduler
Deployment Agent
Domain
Resource
Manager or Control Agent
Control Domain
- Task
Resource

13. Who owns the GRID? Talk to people Power appliances Use GRID resources
I want to:
My interface is:
I arrange service and payments with a:
(may be many choices)
Abstract
Owner (AO)
Phone co.
Electric co.
HPC
Networks
Instruments
People
Antennae
Cable/fiber
Switches
Generators
Power lines
Transformers
But resources I “get” may belong to others:

14. AO is owner or broker User
User negotiates with AO through “order window”
That AO may own some resources, and/or it may broker with other AOs for those resources
After negotiation, resources are delivered through “pickup window”
Requests
Resources
Order
Window
Pickup
Window AO
Order
Pickup
Order
Pickup
Manager
Resource Manager
Sales
Delivery
AO3
Physical
Resource
AO2
AO1

15. AO Resources Resources are objects Classes are
Instrument
Data source, sink, transform
e.g. programs, people, files, data collection devices
Channel
Moves data among instruments
Complexes of above
Attributes define sizes, times, connections, etc.
Instrument
(File)
Instrument
(Program)
Channels
Instrument
(File)
Instrument
(Program)
Instrument
(Person)
Instrument
(Telescope)

16. (values for variables/attributes + asking price for each)
Negotiating with an AO
Make dummy resource
(with attributes set to constants, variables, or “don’t care”) + bid + delivery plan + variable constraints
Pick one,
Try again,
Or give up
Assign tasks to resource, use, relinquish
User
Perhaps later...
Delivery
Window
Order Window
Resource candidates
(values for variables/attributes + asking price for each) AO
Resource

17. Many Testbeds ? & who pays ?
$grid
GUSTO
EcoGrid
Legion Testbed
NASA IPG

18. Testbeds so far -- observations
Who contributed to resources & why ?
Volunteers: for fun, challenge, fame, public good like & distributed.net projects.
Collaborators: sharing resources while developing new technologies of common interest – Globus, Legion, Ecogrid.
How long ?
Short duration: GUSTO decommissioned.
What do we need ? Grid Marketplace!
Regulates demand and supply, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.

19. Users in Grid Economy & Strategy
Grid Consumers
Execute jobs for solving varying problem size and complexity
Benefit by selecting and aggregating resources wisely
Tradeoff timeframe and cost
Strategy: minimise expenses
Grid Providers
Contribute “idle” resource for executing consumer jobs
Benefit by maximizing resource utilisation
Tradeoff local requirements & market opportunity
Strategy: maximise returns on services

20. Building of a Economy Grid “brokerage” system…..
Who pays for that ???
Users!
Foundation for the Grid Economy

21. Grid Architecture for Computational Economy
Grid Market Services
Information Server(s)
Sign-on
Health
Monitor
Info ?
Grid Explorer
Grid Node N …
Application …
Secure
Job Control
Agent
Schedule Advisor
Grid Node1
QoS
Pricing Algorithms
Trade Server
Trading
Trade Manager
Accounting
Resource
Reservation
Misc. services …
Deployment Agent
JobExec
Resource Allocation
Grid User
Grid Resource Broker
Storage R1 R2 … Rm
Grid Middleware
Services
Grid Service Providers

22. Economic Models for Trading
Commodity Market Model
Posted Prices Models
Bargaining Model
Tendering (Contract Net) Model
Auction Model
English, first-price sealed-bid, second-price sealded-bid (Vickrey), and Dutch.
Proportional Resource Sharing Model
Shareholder Model
Partnership Model

23. Economy Grid = Globus + GRACE
Applications …
Grid
Apps.
Science
Engineering
Commerce
Portals
ActiveSheet
High-level Services and Tools
GlobusView
Grid Status
Grid
Tools
DUROC
MPI-G
MPI-IO
CC++
Nimrod/G
globusrun
Core Services
Heartbeat Monitor
Nexus
GRAM
GRACE-TS
Grid
Middleware
Globus Security Interface
MDS
GASS
DUROC
GARA
GMD
GBank
Grid
Fabric
Local Services
Condor
GRD
QBank
JVM
TCP
UDP
LSF
PBS
eCash
Linux
Irix
Solaris

24. What is Nimrod/G ?
A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy.
Key Features
A single window to manage & control experiment
Resource Discovery
Trade for Resources
Scheduling
Steering & data management
It allows to study the behaviour of some of the output variables against a range of different input scenarios.

25. Nimrod/G Grid Broker Architecture
Nimrod/G Client
Nimrod/G Client
Nimrod/G Client
Nimrod/G Engine
Schedule Advisor
Grid
Store
Trading Manager
Grid Dispatcher
Grid Explorer
Grid Middleware
Globus,Legion, Condor-g,, Ninf,etc.
TM TS
GE GIS
Grid Information Server(s)
RM & TS
RM & TS
RM & TS G C L G
Legion enabled node.
Globus enabled node. L C
RM: Local Resource Manager, TS: Trade Server
Condor enabled node.

26. A Nimrod/G Client Cost Deadline Legion hosts Globus Hosts
Bezek is in both
Globus and Legion Domains

27. User Requirements: Deadline/Budget

28. Global Economy Grid Internet Australia North America Monash Uni.:
ANL: SGI/Sun/SP2
USC-ISI: SGI
UVa: Linux Cluster
Manitoba: Cluster
Nimrod/G
Linux cluster
Globus+Legion
+Condor/G
Solaris WS
Globus/Legion
GRACE_TS
Internet
Europe
ZIB/FUB: T3E/Mosix
Cardiff: Sun E6500
Paderborn: HPCLine
Lecce: Compaq SC
CNR: Cluster
CERN: Cluster
Asia/Japan
Tokyo I-Tech.:
ETL, Tuskuba
Linux cluster
Globus +
GRACE_TS
Globus +
GRACE_TS

30. Conclusions
Proposed three models for Grid resource management architecture
Hierarchical, AO, & Market-model
The future systems are likely follow a model that combines all these models.
The future computing (HPC) infrastructure is going to be a Grid of Clusters.
Peer-to-Peer/Grid has already become a darling of venture capitalists.
The impact of Grid on 21st century economy will be the same as electricity on 20th century economy.