3 Computing Power (HPC) Drivers Solving grand challenge applications using computer modeling, simulation and analysisAerospaceInternet &EcommerceLife SciencesDigital BiologyCAD/CAMMilitary ApplicationsMilitary ApplicationsMilitary Applications
4 SMPs or SuperComputers Computing Platforms2100?PERFORMANCE2100Breaking Administrative BarriersIndividualGroupDepartmentCampusStateNationalGlobeInter PlanetUniverseDesktopSMPs or SuperComputersLocalClusterEnterpriseCluster/GridGlobalCluster/GridInter PlanetCluster/Grid ??
5 Towards Grid Computing Unification of geographically distributed resources
6 What is Grid ? An infrastructure that couples: Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks,mobile devices, PDA, etc;Software – e.g., ASPs renting expensive special purpose applications on demand;Catalogued data and databases – e.g. transparent access to human genome database;Special devices – e.g., radio telescope – searching for life in galaxy, for pulsars)People/collaborators.& offers dependable, consistent, & pervasive access to resources.
8 Sources of Complexity in Grid Resource Management No single administrative control.No single ownership policy:Each resource owner has their own policies or scheduling mechanisms;Users must honour them (particularly external Grid users).Heterogeneity of resources.Dynamic availability – may appear and disappear…
9 Sources of Complexity in Grid Resource Management Unreliable resource – disappear from view!No uniform cost model - varies from one user’s resource to another and from time of day.No single access mechanism – Web, custom interfaces, command line…
11 Architectural Models MODEL REMARKS Systems Hierarchical It captures model followed in most contemporary systems.Globus, Legion, CCS, Apples, NetSolve, Ninf.Abstract Owner (AO)Order and delivery model and focuses on long term goals.Expected to emerge and most peer-2-peer computing systems likely to be based on this.Market ModelIt follows economic model for resource discover, sharing, & scheduling.GRACE, Nimrod/G, JavaMarket, Mariposa.
12 Hierarchical Resource Management Access/Admission Control AgentUserGlobalSchedulerGlobalSchedulerGrid Information ServicePersistent Job ControlAgentConnection CloudGlobalSchedulerMonitorGlobalSchedulerGlobalSchedulerLocalSchedulerDeployment AgentDomainResourceManager or Control AgentControl Domain- TaskResource
13 Who owns the GRID? Talk to people Power appliances Use GRID resources I want to:My interface is:I arrange service and payments with a:(may be many choices)AbstractOwner (AO)Phone co.Electric co.HPCNetworksInstrumentsPeopleAntennaeCable/fiberSwitchesGeneratorsPower linesTransformersBut resources I “get” may belong to others:
14 AO is owner or broker User User negotiates with AO through “order window”That AO may own some resources, and/or it may broker with other AOs for those resourcesAfter negotiation, resources are delivered through “pickup window”RequestsResourcesOrderWindowPickupWindowAOOrderPickupOrderPickupManagerResource ManagerSalesDeliveryAO3PhysicalResourceAO2AO1
15 AO Resources Resources are objects Classes are InstrumentData source, sink, transforme.g. programs, people, files, data collection devicesChannelMoves data among instrumentsComplexes of aboveAttributes define sizes, times, connections, etc.Instrument(File)Instrument(Program)ChannelsInstrument(File)Instrument(Program)Instrument(Person)Instrument(Telescope)
16 (values for variables/attributes + asking price for each) Negotiating with an AOMake dummy resource(with attributes set to constants, variables, or “don’t care”) + bid + delivery plan + variable constraintsPick one,Try again,Or give upAssign tasks to resource, use, relinquishUserPerhaps later...DeliveryWindowOrder WindowResource candidates(values for variables/attributes + asking price for each)AOResource
17 Many Testbeds ? & who pays ? $gridGUSTOEcoGridLegion TestbedNASA IPG
18 Testbeds so far -- observations Who contributed to resources & why ?Volunteers: for fun, challenge, fame, public good like & distributed.net projects.Collaborators: sharing resources while developing new technologies of common interest – Globus, Legion, Ecogrid.How long ?Short duration: GUSTO decommissioned.What do we need ? Grid Marketplace!Regulates demand and supply, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.
19 Users in Grid Economy & Strategy Grid ConsumersExecute jobs for solving varying problem size and complexityBenefit by selecting and aggregating resources wiselyTradeoff timeframe and costStrategy: minimise expensesGrid ProvidersContribute “idle” resource for executing consumer jobsBenefit by maximizing resource utilisationTradeoff local requirements & market opportunityStrategy: maximise returns on services
20 Building of a Economy Grid “brokerage” system….. Who pays for that ???Users!Foundation for the Grid Economy
24 What is Nimrod/G ?A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy.Key FeaturesA single window to manage & control experimentResource DiscoveryTrade for ResourcesSchedulingSteering & data managementIt allows to study the behaviour of some of the output variables against a range of different input scenarios.
28 Global Economy Grid Internet Australia North America Monash Uni.: ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux ClusterManitoba: ClusterNimrod/GLinux clusterGlobus+Legion+Condor/GSolaris WSGlobus/LegionGRACE_TSInternetEuropeZIB/FUB: T3E/MosixCardiff: Sun E6500Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCERN: ClusterAsia/JapanTokyo I-Tech.:ETL, TuskubaLinux clusterGlobus +GRACE_TSGlobus +GRACE_TS
30 ConclusionsProposed three models for Grid resource management architectureHierarchical, AO, & Market-modelThe future systems are likely follow a model that combines all these models.The future computing (HPC) infrastructure is going to be a Grid of Clusters.Peer-to-Peer/Grid has already become a darling of venture capitalists.The impact of Grid on 21st century economy will be the same as electricity on 20th century economy.