1. Nimrod/G and Grid Market A Case for Economy Grid Architecture for Service Oriented Global Grid Computing Rajkumar Buyya, David Abramson, Jon Giddy Monash University, Melbourne, Australia www.buyya.com/ecogrid www.gridcomputing.com

2. 2100 DesktopSMPs or SuperComputers Local Cluster Global Cluster/Grid PERFORMANCEPERFORMANCE Inter Planet Cluster/Grid ?? Individual Group Department Campus State National Globe Inter Planet Universe Administrative Barriers Enterprise Cluster/Grid ? Scalable HPC: Breaking Administrative Barriers

3. Why Grids ? Large Scale Exploration needs themKiller Applications. Solving grand challenge applications using computer modeling, simulation and analysis Life Sciences CAD/CAM Aerospace Military Applications Digital Biology Military Applications Internet & Ecommerce

4. Players in Grid Computing

5. What users want ? Users in Grid Economy & Strategy Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute idle resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity Strategy: maximise returns on services

6. mix-and-match Object-oriented Internet-WWW Problem Solving Approach Market/Computational Economy

7. Grid Node N Grid Architecture for Computational Economy Grid User Application Grid Resource Broker Grid Service Providers Grid Explorer Schedule Advisor Trade Manager Job Control Agent Deployment Agent Trade Server Resource Allocation Resource Reservation R1R1 Misc. services Information Server(s) R2R2 RmRm … Pricing Algorithms Accounting Grid Node1 … Grid Middleware Services … … Health Monitor Grid Market Services JobExec Info ? Secure Trading QoS Storage Sign-on

8. Economy Grid = Globus + GRACE Applications MDS GRAM Globus Security Interface Heartbeat Monitor Nexus Local Services LSF Condor GRDQBank PBS TCP SolarisIrixLinux UDP High-level Services and Tools DUROCglobusrunMPI-G Nimrod/G MPI-IOCC++ GlobusViewGrid Status GASS GRACE-TS GARA Grid Fabric Grid Apps. Grid Middleware Grid Tools GBank GMD eCash JVM DUROC Core Services ScienceEngineeringCommercePortalsActiveSheet … …

9. A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy. Key Features A single window to manage & control experiment Resource Discovery Trade for Resources Resource Composition & Scheduling Steering & data management It allows to study the behaviour of some of the output variables against a range of different input scenarios. Nimrod/G : A Grid Resource Broker

10. A Glance at Nimrod-G Broker Grid Middleware Nimrod/G Client Grid Information Server(s) Schedule Advisor Trading Manager Nimrod/G Engine Grid Store Grid Explorer GE GIS TM TS RM & TS Grid Dispatcher RM: Local Resource Manager, TS: Trade Server Globus,Legion, Condor-g,, Ninf,etc. G G C L Globus enabled node. Legion enabled node. C L Condor enabled node. RM & TS

11. A Nimrod/G Client CostDeadline Legion hosts Globus Hosts Bezek is in both Globus and Legion Domains

12. Discover Resources Distribute Jobs Establish Rates Meet requirements ? Remaining Jobs, Deadline, & Budget ? Evaluate & Reschedule Discover More Resources Adaptive Scheduling algorithms Compose & Schedule

13. Globus+Legion +Condor/G Australia Monash Uni.: Linux cluster Solaris WS Nimrod/G Globus + GRACE_TS Europe ZIB/FUB: T3E/Mosix Cardiff: Sun E6500 Paderborn: HPCLine Lecce: Compaq SC CNR: Cluster Calabria: Cluster CERN: Cluster Pozman: SGI/SP2 Globus + GRACE_TS Asia/Japan Tokyo I-Tech.: ETL, Tuskuba Linux cluster Globus/Legion GRACE_TS North America ANL: SGI/Sun/SP2 USC-ISI: SGI UVa: Linux Cluster Internet Inter-Continental Grid

14. Experimentation on the Grid Workload: 165 jobs, each need 5 minute of cpu time Deadline: 1 hrs. and budget: 800,000 units Strategy: minimise cost and meet deadline Execution Cost with cost optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155 (G$) Resour ce OwnerGrid service s Peak time (G$) Off peak cost Linux cluster (60 nodes) Monash, Australia Globus/ Condor 205 IBM SP2 (80 nodes) ANL, Chicago, US Globus/LL510 Sun (8 nodes) ANL, Chicago, US Globus/ Fork 510 SGI (96 nodes) ANL, Chicago, US Globus/ Condor-G 15 SGI (10 nodes) ISI, LA, US Globus/ Fork 1020

15. Execution @ AU Peak Time

16. Execution @ AU Offpeak Time