1. WW Grid Economic Models for Management of Resources in Peer-to-Peer (P2P) and Grid Computing R. Buyya, H.Stockinger, J.Giddy, D.Abramson Melbourne, Australia Switzerland buyya.com/ecogrid ITCOM 2001, Denver, Aug 19-24, 2001

2. 2

3. 3 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod/G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

4. 4 2100 DesktopSMPs or SuperComputers Local Cluster Global Cluster/Grid PERFORMANCEPERFORMANCE Inter Planetary Grid! Individual Group Department Campus State National Globe Inter Planet Universe Administrative Barriers Enterprise Cluster/Grid ? Scalable HPC: Breaking Administrative Barriers & new challenges

5. 5 Why Grids? Large Scale Explorations need them Killer Applications. Solving grand challenge applications using modeling, simulation and analysis Life Sciences CAD/CAM Aerospace Military Applications Digital Biology Military Applications Internet & Ecommerce

6. 6

7. 7 What is Grid ? An infrastructure that logically couples distributed resources: Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc; Software – e.g., ASPs renting expensive special purpose applications on demand; Catalogued data and databases – e.g. transparent access to human genome database; Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy. People/collaborators. and presents them as an integrated global resource. It enables the creation of virtual enterprises (VEs) for resource sharing. Wide area data archives

8. 8 P2P/Grid Applications-Drivers Distributed HPC (Supercomputing): Computational science. High-Capacity/Throughput Computing: Large scale simulation/chip design & parameter studies. Content Sharing (free or paid) Sharing digital contents among peers (e.g., Napster) Remote software access/renting services: Application service provides (ASPs) & Web services. Data-intensive computing: Virtual Drug Design, Particle Physics, Stock Prediction... On-demand, realtime computing: Medical instrumentation & Mission Critical. Collaborative Computing: Collaborative design, Data exploration, education. Service Oriented Computing (SOC): Computing as Utility: New paradigm and new industries.

9. 9 Building and Using Grids require Services that make our systems Grid Ready! Security mechanisms that permit resources to be accessed only by authorized users. (New) programming tools that make our applications Grid Ready!. Tools that can translate the requirements of an application/user into the requirements of computers, networks, and storage. Tools that perform resource discovery, trading, selection/allocation, scheduling and distribution of jobs and collects results. Globus ?

10. 10 Players in Grid Computing

11. 11 What users want ? Users in Grid Economy & Strategy Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute ( idle ) resources for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements and market opportunity Strategy: maximise return on investment

12. 12 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

13. 13 Sources of Complexity in Resource Management for World Wide Grid Computing Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstations, clusters, and supercomputers, instruments, databases, software) Heterogeneity of fabric management systems (single system image OS, queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of application requirements (CPU, I/O, memory, and/or network intensive) Heterogeneity in resource demand patterns (peak, off-peak,...) Applications need different QoS at different times (time critical results). The utility of experimental results varies from time to time. Geographical distribution of users & located different time zones Differing goals (producers and consumers have different objectives and strategies) Unsecure and Unreliable environment

14. 14 Traditional approaches to resource management & scheduling are NOT useful for Grid ? They use centralised policy that need complete state-information and common fabric management policy or decentralised consensus-based policy. Due to too many heterogenous parameters in the Grid it is impossible to define/get: system-wide performance matrix and common fabric management policy that is acceptable to all. Economics paradigm proved to effective institution in managing decentralization and heterogeneity that is present in human economies! Fall of USSR & Emergence of US as world superpower! (monopoly?) So, we propose/advocate the use of computational economics principles in management of resources and scheduling computations on world wide Grid. Think locally and act globally approach to grid computing!

15. 15 Benefits of Computational Economies It provides a nice paradigm for managing self interested and self- regulating entities (resource owners and consumers) Helps in regulating supply-and-demand for resources. Services can be priced in such a way that equilibrium is maintained. User-centric / Utility driven: Value for money! Scalable: No need of central coordinator (during negotiation) Resources(sellers) and also Users(buyers) can make their own decisions and try to maximize utility and profit. Adaptable It helps in offering different QoS (quality of services) to different applications depending the value users place on them. It improves the utilisation of resources It offers incentive for resource owners for being part of the grid! It offers incentive for resource consumers for being good citizens There is large body of proven Economic principles and techniques available, we can easily leverage it.

16. 16 New challenges of Computational Economy Resource Owners How do I decide service prices ? (economic models?) How do I specify them ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? ….. Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How I trade between timeframe & cost ? …. Any tools, traders & brokers available to automate the process ?

17. 17 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

18. 18 mix-and-match Object-oriented Internet/partial-P2P Network enabled Solvers Market/Computational Economy

19. 19 Many Testbeds ? & who pays ?, who regulates supply and demand ? GUSTO (decommissioned) Legion Testbed NASA IPG World Wide Grid WW Grid

20. 20 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

21. 21 Building an Economy Grid (Next Generation Grid Computing!) To enable the creation and promotion of: Grid Marketplace (competitive) ASP Service Oriented Computing... And let users focus on their own work (science, engineering, or commerce)!

22. 22 Grid Node N GRACE: A Reference Grid Architecture for Computational Economy Grid User Application Grid Resource Broker Grid Service Providers Grid Explorer Schedule Advisor Trade Manager Job Control Agent Deployment Agent Trade Server Resource Allocation Resource Reservation R1R1 Misc. services Information Server(s) R2R2 RmRm … Pricing Algorithms Accounting Grid Node1 … Grid Middleware Services … … Health Monitor Grid Market Services JobExec Info ? Secure Trading QoS Storage Sign-on Grid Bank See PDPTA 2000 paper!

23. 23 Grid Fabric Grid Apps. Grid Middleware Grid Tools Networked Resources across Organisations Computers Clusters Data Sources Scientific Instruments Storage Systems Local Resource Managers Operating Systems Queuing Systems TCP/IP & UDP … Libraries & App Kernels … Distributed Resources Coupling Services SecurityInformation … QoS Process Development Environments and Tools Languages Libraries Debuggers … Web tools Resource BrokersMonitoring Applications and Portals Prob. Solving Env. Scientific … Collaboration Engineering Web enabled Apps Resource Trading Grid Components Market Info

24. 24 Economy Grid = Globus + GRACE Applications MDS GRAM Grid Security Interface Heartbeat Monitor Nexus Local Services LSF Condor GRDQBank PBS TCP SolarisIrixLinux UDP High-level Services and Tools DUROCglobusrunMPI-G Nimrod/G CC++ Grid Status GASS GRACE-TS GARA Grid Fabric Grid Apps. Grid Middleware Grid Tools GBank GMD eCash JVM DUROC Core Services ScienceEngineeringCommercePortalsActiveSheet … … See IPDPS HWC 2001 paper! … …

25. 25 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

26. 26 Economic Models Price-based: Supply,demand,value, wealth of economic system Commodity Market Model Posted Price Model Bargaining Model Tendering (Contract Net) Model Auction Model English, first-price sealed-bid, second-price sealed-bid (Vickrey), and Dutch (consumer:low,high,rate; producer:high, low, rate) Proportional Resource Sharing Model Monopoly (one provider) and Oligopoly (few players) consumers may not have any influence on prices. Bartering Shareholder Model Partnership Model

27. 27 Grid Open Trading Protocols Get Connected Call for Bid(DT) Reply to Bid (DT) Negotiate Deal(DT) Confirm Deal(DT, Y/N) …. Cancel Deal(DT) Change Deal(DT) Get Disconnected Trade Manager Trade Server Pricing Rules DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period API

28. 28 Various Criteria for Judging Effectiveness of Economic Models Social Welfare global good of all Pareto Efficiency global perspective Individual Rationality better off by participating in negotiation Stability mechanisms that cannot be manipulated Computational Efficiency protocols should not consume too much of time Distribution and Communication Efficiency communication overhead to capture a desirable global solution

29. 29 A Commodity Market Model Solve this in 5hrs for $20 Grid Market Directory (GMD) Resource Broker Grid Info. Service GTS (Grid Service Provider) GTS register me as GSP Give me list of GSPs & price? service available? (GTS - Grid Trade Server) (GSP) service available? (RB selects GSPs)

30. 30 How to decide Price ? Fixed price model (like today s Internet) Dynamic/Demand and Supply (like tomorrow s Internet) Usage Period Loyalty of Customers (like Airlines favoring frequent flyers!) Historical data Advance Agreement (high discount for corporations) Usage Timing (peak, off-peak, lunch time) Calendar based (holiday/vacation period) Bulk Purchase (register 100.com domains at once!) Voting -- trade unions decide pricing structure Resource capability as benchmarked in the market! Academic R&D/public-good application users can be offered at cheaper rate compared to commercial use. Customer Type – Quality or price sensitive buyers. Can be Prescribed by Regulating (Govt.) authorities

31. 31 Posted Price Model Solve this by next day for $5 Grid Market Directory (GMD) Resource Broker 2hrs SP2, $5 Grid Info. Service GTS (Grid Service Provider) GTS T3E, $9/hr, Sunday Free for Genome 10% discount today Any SP2/T3E? offers Free or < $2/hr clusters+matlab 5MB free (GTS - Grid Trade Server) (GSP)

32. 32 Bargaining Model Solve this in 5hrs for $20 Grid Market Directory (GMD) Resource Broker Grid Info. Service GTS (Grid Service Provider) GTS register me as GSP Give me list of GSPs access price ?, 2, 3 ? (GTS - Grid Trade Server) (GSP) access price ? (RB negotiates for the best price)

33. 33 Tender/Contract-Net Model Solve this in 15hrs for $10 Grid Market Directory (GMD) Resource Broker Grid Info. Service GTS (Grid Service Provider) GTS Any Ads for service tenders Post: call for tenders (GTS - Grid Trade Server) (GSP) (GSPs bid) gsp1 bid gsp3 bid? gsp2 bid? gspN bid? Closed Reverse Auction ? Buyers name their price and supplies compete to bid the lowest price. Eg: GotFrom.com

34. 34 Auction Model Grid Market Auctioneer (GMA) Resource Broker SP2 time, 9pm-8am Grid Info. Service GTS (Grid Service Provider) GTS Post: auction T3E service Solve this in 20 hrs for $5 (GTS - Grid Trade Server) (GSP) Resource Broker …. $2, gsp1 $4, gsp1 $2, gsp2 Solve this in 1 hrs for $35 (RBs bid)

35. 35 Auction Types English Auction - first-price open-cry each bidder free to raise his bid and highest bidder wins at his bid price. First-price sealed bid each bidder submits one bid and highest bidder wins at his bid price. Vickrey Auction- Second-price sealed bid each bidder submits one bid and highest bidder wins at the price of second highest bid. Dutch (descending) Auction seller continuously lowers the price until one of the bidders bids.

36. 36 Proportional Resource Sharing Model Grid Market Directory (GMD) Resource Broker SP2 time, 9pm-8am Grid Info. Service GTS (Grid Service Provider) GTS Post: auction T3E service Solve this in 20 hrs for $5 (GTS - Grid Trade Server) (GSP) Resource Broker …. RB1: $2 Solve this in 1 hrs for $50 (RBs bid) Give me list of GSPs RBn: $4 Resource Share ? Bid/(sum of all bids). E.g., RB1 share = 1/3 RB n share - 2/3

37. 37 Bartering: Partnership/Shareholder A group of organizations pool in resources (money) together or govt. funded. APAC (Australian Partnership for Advance Computing) VPAC (Victorian PAC) - VIC unis and govt. NPACI in US Allocation proportional to contribution. When in demand: follow proportional resource sharing strategy & QoS. A group of individuals pool in resources (idle cycles) like in Condor pool. Contribute for public good/fame (SETI@Home or distributed.net)

38. 38 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

39. 39 A resource broker for managing, steering, and executing task farming (parametric sweep/SPMD model) applications on Grid based on deadline and computational economy. Based on users QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability. Key Features A single window to manage & control experiment Persistent and Programmable Task Farming Engine Resource Discovery Resource Trading Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & results Steering & data management Accounting Nimrod/G : A Grid Resource Broker

40. 40 Parametric Processing Multiple Runs Same Program Multiple Data Killer Application for the Grid! Parameters Courtesy: Anand Natrajan, University of Virginia Magic Engine See IPDPS 2000 paper!

41. 41 Parameter Processing on the Grid Aggregate Job Submission Aggregate View Submit & Play!

42. 42 A Glance at Nimrod-G Broker Grid Middleware Nimrod/G Client Grid Information Server(s) Schedule Advisor Trading Manager Nimrod/G Engine Grid Store Grid Explorer GE GIS TM TS RM & TS Grid Dispatcher RM: Local Resource Manager, TS: Trade Server Globus, Legion, Condor, etc. G G C L Globus enabled node. Legion enabled node. G L Condor enabled node. RM & TS CL See HPCAsia 2000 paper!

43. 43 A Nimrod/G Monitor CostDeadline Legion hosts Globus Hosts Bezek is in both Globus and Legion Domains

44. 44 User Requirements: Deadline/Budget

45. 45 Discover Resources Distribute Jobs Establish Rates Meet requirements ? Remaining Jobs, Deadline, & Budget ? Evaluate & Reschedule Discover More Resources Adaptive Scheduling Algorithms Compose & Schedule See HPDC AMS 2000 paper!

46. 46 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

47. 47 Experiment Setup Workload (Hypothetical Application): 165 jobs, each need 5 minute of CPU time Deadline: 2 hrs. and budget: 396000 units (G$) Strategy: minimise time / cost Execution Cost with cost optimisation Optimise Cost: 115200 (G$) (finished in 2hrs.) Optimise Time: 237000 (G$) (finished in 1.25 hr.) In this experiment: Time-optimised scheduling run costs double that of Cost-optimised. Users are able to trade-off between Time Vs. Cost depending on QoS requirements.

48. 48 Globus+Legion GRACE_TS Australia Monash Uni.: Linux cluster Solaris WS Nimrod/G Globus + GRACE_TS Europe ZIB/FUB: T3E/Mosix Cardiff: Sun E6500 Paderborn: HPCLine Lecce: Compaq SC CNR: Cluster Calabria: Cluster CERN: Cluster Pozman: SGI/SP2 Globus + GRACE_TS Asia/Japan Tokyo I-Tech.: ETL, Tuskuba Linux cluster Globus/Legion GRACE_TS North America ANL: SGI/Sun/SP2 USC-ISI: SGI UVa: Linux Cluster UD: Linux cluster UTK: Linux cluster Internet World Wide Grid (WWG) Globus + GRACE_TS South America Chile: Cluster WW Grid

49. 49 Resources Selected & Price/CPU-sec. Cost_Opt.Time_Opt 4 5 1 1 1 153 378 Globus, GTS, Fork SGI-ISI, LA, US 42 9 6 7 64 No. of Jobs Executed 7 Globus, GTS, Fork Sun-ANL, Chicago,US 3 Globus, GTS, Fork Solaris/Ultas2 TITech, Tokyo, Japan 4 Globus, GTS, Fork Linux-Barbera-CNR, Pisa, Italy 3 Globus, GTS, Fork Linux-Prosecco-CNR, Pisa, Italy 2 Globus, GTS, Condor Linux Cluster-Monash, Melbourne, Australia Cost/CPU sec.in G$ Grid services & Fabric Resource & Location Total Experiment Cost (G$)237000115200 Time to Complete Exp. (Min.)70119

50. 50 DBC Scheduling for Time Optimization

51. 51 DBC Scheduling for Cost Optimization

52. 52 Agenda A quick glance at today s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions & Pointers SchedulingEconomics Grid Economy Grid

53. 53 Summary and Conclusions P2P and Grid Computing is emerging as a next generation computing platform for solving large scale problems through sharing of geographically distributed resources. Resource management is a complex undertaking as systems need to be adaptive, scalable, competitive, …, and driven by QoS. We proposed a framework based on computational economies and discussed several economic models for resource allocation and for regulating supply-and-demand for resources. Scheduling experiments on World Wide Grid demonstrate our Nimrod-G broker ability to dynamically lease or rent services at runtime based on their quality, cost, and availability depending on consumers QoS requirements. Economics paradigm for QoS driven resource management is essential to push P2P/Grids into mainstream computing!

54. 54 Thank You… Any ??

55. 55 Download Software & Information Nimrod & Parameteric Computing: http://www.csse.monash.edu.au/~davida/nimrod/ Economy Grid & Nimrod/G: http://www.buyya.com/ecogrid/ Virtual Laboratory/Virtual Drug Design: http://www.buyya.com/vlab/ Grid Simulation (GridSim) Toolkit (Java based): http://www.buyya.com/gridsim/ World Wide Grid (WWG) testbed: http://www.buyya.com/ecogrid/wwg/ Looking for new volunteers to grow Please contact me to barter your & our machines! Want to build on our work/collaborate: Talk to me now or email: rajkumar@csse.monash.edu.au