Presentation is loading. Please wait.

Presentation is loading. Please wait.

Architectural Models for Resource Management in the Grid

Similar presentations

Presentation on theme: "Architectural Models for Resource Management in the Grid"— Presentation transcript:

1 Architectural Models for Resource Management in the Grid
Rajkumar Buyya Monash University, Australia Steve Chapin, Syracuse University, USA David DiNucci, USA

2 Outline Changes in Computing Landscape Resource Management Issues
Architectural Models Hierarchical Resource Management Abstract Owner Market Model Economy Grid Nimrod/G Resource Broker Conclusions

3 Computing Power (HPC) Drivers
Solving grand challenge applications using computer modeling, simulation and analysis Aerospace Internet & Ecommerce Life Sciences Digital Biology CAD/CAM Military Applications Military Applications Military Applications

4 SMPs or SuperComputers
Computing Platforms 2100 ? PERFORMANCE 2100 Breaking Administrative Barriers Individual Group Department Campus State National Globe Inter Planet Universe Desktop SMPs or SuperComputers Local Cluster Enterprise Cluster/Grid Global Cluster/Grid Inter Planet Cluster/Grid ??

5 Towards Grid Computing
Unification of geographically distributed resources

6 What is Grid ? An infrastructure that couples:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc; Software – e.g., ASPs renting expensive special purpose applications on demand; Catalogued data and databases – e.g. transparent access to human genome database; Special devices – e.g., radio telescope – searching for life in galaxy, for pulsars) People/collaborators. & offers dependable, consistent, & pervasive access to resources.

7 A Example Grid Infrastructure

8 Sources of Complexity in Grid Resource Management
No single administrative control. No single ownership policy: Each resource owner has their own policies or scheduling mechanisms; Users must honour them (particularly external Grid users). Heterogeneity of resources. Dynamic availability – may appear and disappear…

9 Sources of Complexity in Grid Resource Management
Unreliable resource – disappear from view! No uniform cost model - varies from one user’s resource to another and from time of day. No single access mechanism – Web, custom interfaces, command line…

10 Grid Resource Management Issues
Authentication (once). Specify (code, resources, etc.). Discover resources. Negotiate authorization, acceptable use, Cost, etc. Acquire resources. Schedule Jobs. Initiate computation. Steer computation. Access remote data-sets. Collaborate with results. Account for usage. Discover resources. Negotiate authorisation, acceptable use, Cost, etc. Acquire resources. Schedule jobs. Initiate computation. Steer computation. Domain 1 Domain 2 Ack: Globus..

11 Architectural Models MODEL REMARKS Systems Hierarchical
It captures model followed in most contemporary systems. Globus, Legion, CCS, Apples, NetSolve, Ninf. Abstract Owner (AO) Order and delivery model and focuses on long term goals. Expected to emerge and most peer-2-peer computing systems likely to be based on this. Market Model It follows economic model for resource discover, sharing, & scheduling. GRACE, Nimrod/G, JavaMarket, Mariposa.

12 Hierarchical Resource Management
Access/Admission Control Agent User Global Scheduler Global Scheduler Grid Information Service Persistent Job Control Agent Connection Cloud Global Scheduler Monitor Global Scheduler Global Scheduler Local Scheduler Deployment Agent Domain Resource Manager or Control Agent Control Domain - Task Resource

13 Who owns the GRID? Talk to people Power appliances Use GRID resources
I want to: My interface is: I arrange service and payments with a: (may be many choices) Abstract Owner (AO) Phone co. Electric co. HPC Networks Instruments People Antennae Cable/fiber Switches Generators Power lines Transformers But resources I “get” may belong to others:

14 AO is owner or broker User
User negotiates with AO through “order window” That AO may own some resources, and/or it may broker with other AOs for those resources After negotiation, resources are delivered through “pickup window” Requests Resources Order Window Pickup Window AO Order Pickup Order Pickup Manager Resource Manager Sales Delivery AO3 Physical Resource AO2 AO1

15 AO Resources Resources are objects Classes are
Instrument Data source, sink, transform e.g. programs, people, files, data collection devices Channel Moves data among instruments Complexes of above Attributes define sizes, times, connections, etc. Instrument (File) Instrument (Program) Channels Instrument (File) Instrument (Program) Instrument (Person) Instrument (Telescope)

16 (values for variables/attributes + asking price for each)
Negotiating with an AO Make dummy resource (with attributes set to constants, variables, or “don’t care”) + bid + delivery plan + variable constraints Pick one, Try again, Or give up Assign tasks to resource, use, relinquish User Perhaps later... Delivery Window Order Window Resource candidates (values for variables/attributes + asking price for each) AO Resource

17 Many Testbeds ? & who pays ?
$grid GUSTO EcoGrid Legion Testbed NASA IPG

18 Testbeds so far -- observations
Who contributed to resources & why ? Volunteers: for fun, challenge, fame, public good like & projects. Collaborators: sharing resources while developing new technologies of common interest – Globus, Legion, Ecogrid. How long ? Short duration: GUSTO decommissioned. What do we need ? Grid Marketplace! Regulates demand and supply, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.

19 Users in Grid Economy & Strategy
Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute “idle” resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity Strategy: maximise returns on services

20 Building of a Economy Grid “brokerage” system…..
Who pays for that ??? Users! Foundation for the Grid Economy

21 Grid Architecture for Computational Economy
Grid Market Services Information Server(s) Sign-on Health Monitor Info ? Grid Explorer Grid Node N Application Secure Job Control Agent Schedule Advisor Grid Node1 QoS Pricing Algorithms Trade Server Trading Trade Manager Accounting Resource Reservation Misc. services Deployment Agent JobExec Resource Allocation Grid User Grid Resource Broker Storage R1 R2 Rm Grid Middleware Services Grid Service Providers

22 Economic Models for Trading
Commodity Market Model Posted Prices Models Bargaining Model Tendering (Contract Net) Model Auction Model English, first-price sealed-bid, second-price sealded-bid (Vickrey), and Dutch. Proportional Resource Sharing Model Shareholder Model Partnership Model

23 Economy Grid = Globus + GRACE
Applications Grid Apps. Science Engineering Commerce Portals ActiveSheet High-level Services and Tools GlobusView Grid Status Grid Tools DUROC MPI-G MPI-IO CC++ Nimrod/G globusrun Core Services Heartbeat Monitor Nexus GRAM GRACE-TS Grid Middleware Globus Security Interface MDS GASS DUROC GARA GMD GBank Grid Fabric Local Services Condor GRD QBank JVM TCP UDP LSF PBS eCash Linux Irix Solaris

24 What is Nimrod/G ? A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy. Key Features A single window to manage & control experiment Resource Discovery Trade for Resources Scheduling Steering & data management It allows to study the behaviour of some of the output variables against a range of different input scenarios.

25 Nimrod/G Grid Broker Architecture
Nimrod/G Client Nimrod/G Client Nimrod/G Client Nimrod/G Engine Schedule Advisor Grid Store Trading Manager Grid Dispatcher Grid Explorer Grid Middleware Globus,Legion, Condor-g,, Ninf,etc. TM TS GE GIS Grid Information Server(s) RM & TS RM & TS RM & TS G C L G Legion enabled node. Globus enabled node. L C RM: Local Resource Manager, TS: Trade Server Condor enabled node.

26 A Nimrod/G Client Cost Deadline Legion hosts Globus Hosts
Bezek is in both Globus and Legion Domains

27 User Requirements: Deadline/Budget

28 Global Economy Grid Internet Australia North America Monash Uni.:
ANL: SGI/Sun/SP2 USC-ISI: SGI UVa: Linux Cluster Manitoba: Cluster Nimrod/G Linux cluster Globus+Legion +Condor/G Solaris WS Globus/Legion GRACE_TS Internet Europe ZIB/FUB: T3E/Mosix Cardiff: Sun E6500 Paderborn: HPCLine Lecce: Compaq SC CNR: Cluster CERN: Cluster Asia/Japan Tokyo I-Tech.: ETL, Tuskuba Linux cluster Globus + GRACE_TS Globus + GRACE_TS


30 Conclusions Proposed three models for Grid resource management architecture Hierarchical, AO, & Market-model The future systems are likely follow a model that combines all these models. The future computing (HPC) infrastructure is going to be a Grid of Clusters. Peer-to-Peer/Grid has already become a darling of venture capitalists. The impact of Grid on 21st century economy will be the same as electricity on 20th century economy.

Download ppt "Architectural Models for Resource Management in the Grid"

Similar presentations

Ads by Google