1. Grid Scheduling “ A Distributed Computational Economy and the Nimrod-G Grid Resource Broker ” Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australia www.gridbus.org WW Grid

2. 2 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service- Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed Drug Design Application Case Study GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

3. 3

4. 4 Virtual Lab

5. 5 The Gridbus Vision: To Enable Service Oriented Grid Computing & Bus iness! WW Grid World Wide Grid! Nimrod-G

6. 6 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

7. 7 A Typical Grid Computing Environment Grid Resource Broker Resource Broker Application Grid Information Service Grid Resource Broker database R2R2 R3R3 RNRN R1R1 R4R4 R5R5 R6R6 Grid Information Service

8. 8 Need Grid tools for managing Security Resource Allocation & Scheduling Data locality Network Management System Management Resource Discovery Uniform Access Computational Economy Application Development Tools

9. 9 What users want ? Users in Grid Economy & Strategy Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute ( “ idle ” ) resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity Strategy: maximise return on investment

10. 10 Sources of Complexity in Grid for Resource Management and Scheduling Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstations, clusters, and supercomputers, instruments, databases, software) Heterogeneity of fabric management systems (single system image OS, queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of application requirements (CPU, I/O, memory, and/or network intensive) Heterogeneity in resource demand patterns (peak, off-peak,...) Applications need different QoS at different times (time critical results). The utility of experimental results varies from time to time. Geographical distribution of users & located different time zones Differing goals (producers and consumers have different objectives and strategies) Unsecure and Unreliable environment

11. 11 Traditional approaches to resource management & scheduling are NOT useful for Grid ? They use centralised policy that need complete state-information and common fabric management policy or decentralised consensus-based policy. Due to too many heterogenous parameters in the Grid it is impossible to define/get: system-wide performance matrix and common fabric management policy that is acceptable to all. “ Economic ” paradigm proved as an effective institution in managing decentralization and heterogeneity that is present in human economies! Hence, we propose/advocate the use of “ computational economy ” principles in the management of resources and scheduling computations on the Grid.

12. 12 Benefits of Computational Economies It provides a nice paradigm for managing self interested and self- regulating entities (resource owners and consumers) Helps in regulating supply-and-demand for resources. Services can be priced in such a way that equilibrium is maintained. User-centric / Utility driven: Value for money! Scalable: No need of central coordinator (during negotiation) Resources(sellers) and also Users(buyers) can make their own decisions and try to maximize utility and profit. Adaptable It helps in offering different QoS (quality of services) to different applications depending the value users place on them. It improves the utilisation of resources It offers incentive for resource owners for being part of the grid! It offers incentive for resource consumers for being good citizens There is large body of proven Economic principles and techniques available, we can easily leverage it.

13. 13 New challenges of Computational Economy Resource Owners How do I decide prices ? (economic models?) How do I specify them ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? ….. Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How I trade between timeframe & cost ? …. Any tools, traders & brokers available to automate the process ?

14. 14 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for next generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC) Scheduling Experiments on World Wide Grid testbed Conclusions SchedulingEconomics Grid Economy Grid

15. 15 mix-and-match Object-oriented Internet/partial-P2P Network enabled Solvers Market/Computational Economy

16. 16 Many Grid Projects & Initiatives Australia Nimrod-G GridSim Virtual Lab Active Sheets DISCWorld..new coming up Europe UNICORE MOL UK eScience Poland MC Broker EU Data Grid EuroGrid MetaMPI Dutch DAS XW, JaWS Japan Ninf DataFarm Korea... N*Grid USA Globus Legion OGSA Javelin AppLeS NASA IPG Condor-G Jxta NetSolve AccessGrid and many more... Cycle Stealing &.com Initiatives Distributed.net SETI@Home, …. SETI@Home Entropia, UD, Parabon, …. Public Forums Global Grid Forum P2P Working Group IEEE TFCC Grid & CCGrid conferences http://www.gridcomputing.com

17. 17 Many Testbeds ? & who pays ?, who regulates supply and demand ? GUSTO (decommissioned) Legion Testbed NASA IPG World Wide Grid WW Grid

18. 18 Testbeds so far -- observations Who contributed resources & why ? Volunteers: for fun, challenge, fame, charismatic apps, public good like distributed.net & SETI@Home projects.SETI@Home Collaborators: sharing resources while developing new technologies of common interest – Globus, Legion, Ninf, Gridbus, Nimrod-G, etc. unless you know lab. leaders, it is impossible to get access! How long ? Short term: excitement is lost, too much of admin. Overhead (Globus inst+), no incentive, policy change, … What we need ? Grid Marketplace! Regulates supply-and-demand, offers incentive for being players, simple, scalable solution, quasi- deterministic – proven model in real-world.

19. 19 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

20. 20 Building Grid Economy (Next Generation Grid Computing!) To enable the creation and promotion of: Grid Marketplace (competitive) ASP Service Oriented Computing... And let users focus on their own work (science, engineering, or commerce)!

21. 21 Grid Node N GRACE: A Reference Grid Architecture for Computational Economy Grid Consumer Programming Environments Grid Resource Broker Grid Service Providers Grid Explorer Schedule Advisor Trade Manager Job Control Agent Deployment Agent Trade Server Resource Allocation Resource Reservation R1R1 Misc. services Information Service R2R2 RmRm … Pricing Algorithms Accounting Grid Node1 … Grid Middleware Services … … Health Monitor Grid Market Services JobExec Info ? Secure Trading QoS Storage Sign-on Grid Bank Applications

22. 22 Grid Fabric Grid Apps. Grid Middleware Grid Tools Networked Resources across Organisations Computers Clusters Data Sources Scientific Instruments Storage Systems Local Resource Managers Operating Systems Queuing Systems TCP/IP & UDP … Libraries & App Kernels … Distributed Resources Coupling Services SecurityInformation … QoS Process Development Environments and Tools Languages Libraries Debuggers … Web tools Resource BrokersMonitoring Applications and Portals Prob. Solving Env. Scientific … Collaboration Engineering Web enabled Apps Resource Trading Grid Components Market Info

23. 23 Economy Grid = Globus + GRACE Applications GRAM Globus Security Interface (GSI) Local Services LSF Condor GRDQBank PBS TCP SolarisIrixLinux UDP High-level Services and Tools CactusMPI-G Nimrod-G Broker CC++ GASSGTSGARA Grid Fabric Grid Apps. Grid Middleware Grid Tools GBank GMD eCash JVM DUROC Core Services ScienceEngineeringCommercePortalsActiveSheet … … … … MDS Higher Level Resource Aggregators Nimrod Parametric Language

24. 24 Economic Models Price-based: Supply,demand,value, wealth of economic system Commodity Market Model Posted Price Model Bargaining Model Tendering (Contract Net) Model Auction Model English, first-price sealed-bid, second-price sealed-bid (Vickrey), and Dutch (consumer:low,high,rate; producer:high, low, rate) Proportional Resource Sharing Model Monopoly (one provider) and Oligopoly (few players) consumers may not have any influence on prices. Bartering Shareholder Model Partnership Model See SPIE ITCom 2001 paper!: with Heinz Stockinger, CERN!

25. 25 Grid Open Trading Protocols Get Connected Call for Bid(DT) Reply to Bid (DT) Negotiate Deal(DT) Confirm Deal(DT, Y/N) …. Cancel Deal(DT) Change Deal(DT) Get Disconnected Trade Manager Trade Server Pricing Rules DT - Deal Template: - resource requirements (TM) - resource profile (TS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period API

26. 26 Cost Model Without cost model any shared system becomes un-managable Charge users more for remote facilities than their own Choose cheaper resources before more expensive ones Cost units (G$) may be Dollars Shares in global facility Stored in bank

27. 27 Cost Matrix @ Grid site X Non-uniform costing Encourages use of local resources first Real accounting system can control machine usage 13 21 User 5 Machine 1 User 1 Machine 5 Resource Cost = Function (cpu, memory, disk, network, software, QoS, current demand, etc.) Simple: price based on peaktime, offpeak, discount when less demand,..

28. 28 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

29. 29 A resource broker for managing, steering, and executing task farming (parameter sweep/SPMD model) applications on Grid based on deadline and computational economy. Based on users ’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability. Key Features A single window to manage & control experiment Persistent and Programmable Task Farming Engine Resource Discovery Resource Trading Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & results Steering & data management Accounting Nimrod/G : A Grid Resource Broker

30. 30 Parametric Computing (What Users think of Nimrod Power) Multiple Runs Same Program Multiple Data Killer Application for the Grid! Parameters Courtesy: Anand Natrajan, University of Virginia Magic Engine

31. 31 Sample P-Sweep/Task Farming Applications Bioinformatics: Drug Design / Protein Modelling Bioinformatics: Drug Design / Protein Modelling Sensitivity experiments on smog formation Combinatorial Optimization: Meta-heuristic parameter estimation Ecological Modelling: Control Strategies for Cattle Tick Electronic CAD: Field Programmable Gate Arrays Computer Graphics: Ray Tracing High Energy Physics: Searching for Rare Events Finance: Investment Risk Analysis VLSI Design: SPICE Simulations Aerospace: Wing Design Network Simulation Automobile: Crash Simulation Data Mining Civil Engineering: Building Design astrophysics

32. 32 Drug Design: Data Intensive Computing on Grid It involves screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential to serve as drug candidates. Protein Molecules Chemical Databases (legacy, in.MOL2 format)

33. 33 MEG( MagnetoEncephaloGraphy) Data Analysis on the Grid: Brain Activity Analysis Life-electronics laboratory, AIST Data Analysis Provision of expertise in the analysis of brain function Provision of MEG analysis Data Generation Nimrod-G 64 sensors MEG Results Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs World-Wide Grid [deadline, budget, optimization preference] 1 5 4 3 2 [Collaboration with Osaka University, Japan]

34. 34 P-study Applications -- Characteristics Code (Single Program: sequential or threaded) Long-running Instances Numerous Instances (Multiple Data) High Resource Requirements High Computation-to-Communication Ratio Embarrassingly/Pleasantly Parallel

35. 35 Thesis Perform parameter sweep (bag of tasks) (utilising distributed resources) within “ T ” hours or early and cost not exceeding $M. Three Options/Solutions: Using pure Globus commands Build your own Distributed App & Scheduler Use Nimrod-G (Resource Broker)

36. 36 Remote Execution Steps Choose Resource Transfer Input Files Set Environment Start Process Pass Arguments Monitor Progress Read/Write Intermediate Files Transfer Output Files Summary View Job View Event View +Resource Discovery, Trading, Scheduling, Predictions, Rescheduling,...

37. 37 Using Pure Globus/Legion commands Do all yourself! (manually) Total Cost:$???

38. 38 Build Distributed Application & Scheduler Build App case by case basis Complicated Construction E.g., AppLeS/MPI basedTotal Cost:$???

39. 39 Nimrod-G Broker Automating Distributed Processing Compose, Submit, & Play!

40. 40 Nimrod & Associated Family of Tools P-sweep App. Composition: Nimrod/Enfusion Resource Management and Scheduling: Nimrod-G Broker Design Optimisations: Nimrod-O App. Composition and Online Visualization: Active Sheets Grid Simulation in Java: GridSim Drug Design on Grid: Virtual Lab Remote Execution Server (on demand Nimrod Agent) File Transfer Server

41. 41 A Glance at Nimrod-G Broker Grid Middleware Nimrod/G Client Grid Information Server(s) Schedule Advisor Trading Manager Nimrod/G Engine Grid Store Grid Explorer GE GIS TM TS RM & TS Grid Dispatcher RM: Local Resource Manager, TS: Trade Server Globus, Legion, Condor, etc. G G C L Globus enabled node. Legion enabled node. G L Condor enabled node. RM & TS CL See HPCAsia 2000 paper!

42. 42 GlobusLegion Fabric Nimrod-G Broker Nimrod-G Clients P-Tools (GUI/Scripting) (parameter_modeling) Legacy Applications P2PGTS Farming Engine Dispatcher & Actuators Schedule Advisor Trading Manager Grid Explorer Customised Apps (Active Sheet) Monitoring and Steering Portals Algorithm1 AlgorithmN Middleware... ComputersStorageNetworksInstrumentsLocal Schedulers G-Bank... Agents Resources Programmable Entities Management JobsTasks... AgentSchedulerJobServer PC/WS/ClustersRadio TelescopeCondor/LL/NQS... Database Meta-Scheduler Nimrod/G Grid Broker Architecture Channels... Database CondorGMD IP hourglass! Condor-AGlobus-ALegion-AP2P-A

43. 43 A Nimrod/G Monitor CostDeadline Legion hosts Globus Hosts Bezek is in both Globus and Legion Domains

44. 44 User Requirements: Deadline/Budget

45. 45 Active Sheet: Microsoft Excel Spreadsheet Processing on Grid NimrodProxy Nimrod-G World-Wide Grid

46. 46

47. 47 Nimrod/G Interactions Grid Info Server Process Server User Process File access File Server Grid Node Nimrod Agent Compute Node User Node Grid Dispatcher Grid Trade Server Grid Scheduler Local Resource Manager Nimrod-G Grid Broker Task Farming Engine Grid Tools And Applications Do this in 30 min. for $10?

48. 48 Discover Resources Distribute Jobs Establish Rates Meet requirements ? Remaining Jobs, Deadline, & Budget ? Evaluate & Reschedule Discover More Resources Adaptive Scheduling Steps Compose & Schedule

49. 49 Deadline and Budget Constrained Scheduling Algorithms Algorithm/ Strategy Execution Time (Deadline, D) Execution Cost (Budget, B) Cost OptLimited by DMinimize Cost-Time OptMinimize when possible Minimize Time OptMinimizeLimited by B Conservative-Time Opt MinimizeLimited by B, but all unprocessed jobs have guaranteed minimum budget

50. 50 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

51. 51 The World Wide Grid Sites WW Grid EUROPE: ZIB/Germany PC 2 /Germany AEI/Germany Lecce/Italy CNR/Italy Calabria/Italy Pozman/Poland Lund/Sweden CERN/Swiss CUNI/Czech R. Vrije: Netherlands EUROPE: ZIB/Germany PC 2 /Germany AEI/Germany Lecce/Italy CNR/Italy Calabria/Italy Pozman/Poland Lund/Sweden CERN/Swiss CUNI/Czech R. Vrije: Netherlands ANL/Chicago USC-ISC/LA UTK/Tennessee UVa/Virginia Dartmouth/NH BU/Boston UCSD/San Diego ANL/Chicago USC-ISC/LA UTK/Tennessee UVa/Virginia Dartmouth/NH BU/Boston UCSD/San Diego Monash/Melbourne VPAC/Melbourne Monash/Melbourne VPAC/Melbourne Santiago/Chile TI-Tech/Tokyo ETL/Tsukuba AIST/Tsukuba TI-Tech/Tokyo ETL/Tsukuba AIST/Tsukuba Cardiff/UK Portsmoth/UK Manchester, UK Cardiff/UK Portsmoth/UK Manchester, UK Kasetsart/Bangkok Singapore

52. 52 World Wide Grid (WWG) WW Grid Globus+Legion GRACE_TS Australia Melbourne U. : Cluster VPAC: Alpha Solaris WS Nimrod-G+Gridbus Globus + GRACE_TS Europe ZIB: T3E/Onyx AEI: Onyx Paderborn: HPCLine Lecce: Compaq SC CNR: Cluster Calabria: Cluster CERN: Cluster CUNI/CZ: Onyx Pozman: SGI/SP2 Vrije U: Cluster Cardiff: Sun E6500 Portsmouth: Linux PC Manchester: O3K Globus + GRACE_TS Asia Tokyo I-Tech.: Ultra WS AIST, Japan: Solaris Cluster Kasetsart, Thai: Cluster NUS, Singapore: O2K Globus/Legion GRACE_TS North America ANL: SGI/Sun/SP2 USC-ISI: SGI UVa: Linux Cluster UD: Linux cluster UTK: Linux cluster UCSD: Linux PCs BU: SGI IRIX Internet Globus + GRACE_TS South America Chile: Cluster WW Grid

53. 53 Experiment-1: Peak and Off-peak Workload: 165 jobs, each need 5 minute of cpu time Deadline: 1 hrs. and budget: 800,000 units Strategy: Minimize Cost and meet the deadline Execution Cost with cost optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155 (G$)

54. 54 Application Composition Using Nimrod Parameter Specification Language #Parameters Declaration parameter X integer range from 1 to 165 step 1; parameter Y integer default 5; #Task Definition task main #Copy necessary executables depending on node type copy calc.$OS node:calc #Execute program with parameter values on remote node node:execute./calc $X $Y #Copy results file to use home node with jobname as extension copy node:output./output.$jobname endtask  calc 1 5  output.j1  calc 2 5  output.j2  calc 3 5  output.j3  …  calc 165 5  output.j165

55. 55 Resources Selected & Price/CPU-sec. Resource Type & Size Owner and Location Grid servicesPeaktime Cost (G$) Offpeak cost Linux cluster (60 nodes) Monash, Australia Globus/Condor205 IBM SP2 (80 nodes) ANL, Chicago, US Globus/LL510 Sun (8 nodes)ANL, Chicago, US Globus/Fork510 SGI (96 nodes)ANL, Chicago, US Globus/Condor-G15 SGI (10 nodes)ISI, LA, USGlobus/Fork1020

56. 56 Deadline and Budget-based Cost Minimization Scheduling 1. Sort resources by increasing cost. 2. For each resource in order, assign as many jobs as possible to the resource, without exceeding the deadline. 3. Repeat all steps until all jobs are processed.

57. 57 Execution @ AU Peak Time

58. 58 Execution @ AU Offpeak Time

59. 59 Experiment-2 Setup Workload: 165 jobs, each need 5 minute of CPU time Deadline: 2 hrs. and budget: 396000 G$ Strategies: 1. Minimise cost 2. Minimise time Execution: Optimise Cost: 115200 (G$) (finished in 2hrs.) Optimise Time: 237000 (G$) (finished in 1.25 hr.) In this experiment: Time-optimised scheduling run costs double that of Cost-optimised. Users can now trade-off between Time Vs. Cost.

60. 60 Resources Selected & Price/CPU-sec. Resource & Location Grid services & Fabric Cost/CPU sec.or unit No. of Jobs Executed Time_OptCost_Opt. Linux Cluster-Monash, Melbourne, Australia Globus, GTS, Condor 264153 Linux-Prosecco-CNR, Pisa, Italy Globus, GTS, Fork 371 Linux-Barbera-CNR, Pisa, Italy Globus, GTS, Fork 461 Solaris/Ultas2 TITech, Tokyo, Japan Globus, GTS, Fork 391 SGI-ISI, LA, US Globus, GTS, Fork 8375 Sun-ANL, Chicago,US Globus, GTS, Fork 7424 Total Experiment Cost (G$)237000115200 Time to Complete Exp. (Min.)70119

61. 61 Deadline and Budget Constraint (DBC) Time Minimization Scheduling 1. For each resource, calculate the next completion time for an assigned job, taking into account previously assigned jobs. 2. Sort resources by next completion time. 3. Assign one job to the first resource for which the cost per job is less than the remaining budget per job. 4. Repeat all steps until all jobs are processed. (This is performed periodically or at each scheduling-event.)

62. 62 Resource Scheduling for DBC Time Optimization

63. 63 Resource Scheduling for DBC Cost Optimization

64. Virtual Laboratory Molecular Modeling for Drug Discovery on the World-Wide Grid -- Application Case Study --

65. 65 Drug Design: Data Intensive Computing on Grid It involves screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential to serve as drug candidates. Protein Molecules Chemical Databases (legacy, in.MOL2 format)

66. 66 DataGrid Brokering Nimrod/G Computational Grid Broker Data Replica Catalogue CDB Broker Algorithm1 AlgorithmN... CDB Service “Screen mol.5 please?” GSP1GSP2 GSP4GSP3 (Grid Service Provider) GSPm CDB Service GSPn 1 “advise CDB source? 2 “selection & advise: use GSP4!” 5 Grid Info. Service 3 “Is GSP4 healthy?” 4 “mol.5 please?” 6 “CDB replicas please?” “Screen 2K molecules in 30min. for $10” 7 “process & send results”

67. 67 Software Tools Molecular Modelling Application (DOCK) Parameter Modelling Tools (Nimrod/enFusion) Grid Resource Broker (Nimrod-G) Data Grid Broker Chemical DataBase (CDB) Management and Intelligent Access Tools PDB databse Lookup/Index Table Generation. PDB and associated index-table Replication. PDB Replica Catalogue (that helps in Resource Discovery). PDB Servers (that serve PDB clients requests). PDB Brokering (Replica Selection). PDB Clients for fetching Molecule Record (Data Movement). Grid Middleware (Globus and GrACE) Grid Fabric Management (Fork/LSF/Condor/Codine/ … )

68. 68 The Virtual Lab. – Software Stack Globus [security, information, job submission] [Distributed computers and databases with different Arch, OS, and local resource management systems] Nimrod-G and CDB Data Broker [task farming engine, scheduler, dispatcher, agents, CDB (chemical database) server] Nimrod and Virtual Lab Tools [parametric programming language, GUI tools, and CDB indexer] Molecular Modelling for Drug Design FABRIC APPLICATIONS CORE MIDDLEWARE USER LEVEL MIDDLEWARE PROGRAMMING TOOLS PDBCDB Worldwide Grid

69. 69 V-Lab Components Interaction Grid Info Server Process Server User Process File access File Server Grid Node Nimrod Agent Compute NodeUser Node Grid Dispatcher Grid Trade Server Grid Scheduler Local Resource Manager Nimrod-G Grid Broker Task Farming Engine Grid Tools And Applications Do this in 30 min. for $10? CDB Client Get molecule “n” record from “abc” CDB Docking Process CDB Server Index and CDB1............ CDBm Molecule “n” Location ? Get mol. record CDB Service on Grid

70. 70 DOCK code* (Enhanced by WEHI, U of Melbourne) A program to evaluate the chemical and geometric complementarities between a small molecule and a macromolecular binding site. It explores ways in which two molecules, such as a drug and an enzyme or protein receptor, might fit together. Compounds which dock to each other well, like pieces of a three-dimensional jigsaw puzzle, have the potential to bind. So, why is it important to able to identify small molecules which may bind to a target macromolecule? A compound which binds to a biological macromolecule may inhibit its function, and thus act as a drug. E.g., disabling the ability of (HIV) virus attaching itself to molecule/protein! With system specific code changed, we have been able to compile it for Sun-Solaris, PC Linux, SGI IRIX, Compaq Alpha/OSF1 * Original Code: University of California, San Francisco: http://www.cmpharm.ucsf.edu/kuntz/

71. 71 Dock input file score_ligand yes minimize_ligand yes multiple_ligands no random_seed 7 anchor_search no torsion_drive yes clash_overlap 0.5 conformation_cutoff_factor 3 torsion_minimize yes match_receptor_sites no random_search yes...... maximum_cycles 1 ligand_atom_file S_1.mol2 receptor_site_file ece.sph score_grid_prefix ece vdw_definition_file parameter/vdw.defn chemical_definition_file parameter/chem.defn chemical_score_file parameter/chem_score.tbl flex_definition_file parameter/flex.defn flex_drive_file parameter/flex_drive.tbl ligand_contact_file dock_cnt.mol2 ligand_chemical_file dock_chm.mol2 ligand_energy_file dock_nrg.mol2 Molecule to be screened

72. 72 score_ligand $score_ligand minimize_ligand $minimize_ligand multiple_ligands $multiple_ligands random_seed $random_seed anchor_search $anchor_search torsion_drive $torsion_drive clash_overlap $clash_overlap conformation_cutoff_factor $conformation_cutoff_factor torsion_minimize $torsion_minimize match_receptor_sites $match_receptor_sites random_search $random_search...... maximum_cycles $maximum_cycles ligand_atom_file ${ligand_number}.mol2 receptor_site_file $HOME/dock_inputs/${receptor_site_file} score_grid_prefix $HOME/dock_inputs/${score_grid_prefix} vdw_definition_file vdw.defn chemical_definition_file chem.defn chemical_score_file chem_score.tbl flex_definition_file flex.defn flex_drive_file flex_drive.tbl ligand_contact_file dock_cnt.mol2 ligand_chemical_file dock_chm.mol2 ligand_energy_file dock_nrg.mol2 Parameterize Dock input file (use Nimrod Tools: GUI/language) Molecule to be screened

73. 73 parameter database_name label "database_name" text select oneof "aldrich" "maybridge" "maybridge_300" "asinex_egc" "asinex_epc" "asinex_pre" "available_chemicals_directory" "inter_bioscreen_s" "inter_bioscreen_n" "inter_bioscreen_n_300" "inter_bioscreen_n_500" "biomolecular_research_institute" "molecular_science" "molecular_diversity_preservation" "national_cancer_institute" "IGF_HITS" "aldrich_300" "molecular_science_500" "APP" "ECE" default "aldrich_300"; parameter CDB_SERVER text default "bezek.dstc.monash.edu.au"; parameter CDB_PORT_NO text default "5001";parameter score_ligand text default "yes"; parameter minimize_ligand text default "yes"; parameter multiple_ligands text default "no"; parameter random_seed integer default 7; parameter anchor_search text default "no"; parameter torsion_drive text default "yes"; parameter clash_overlap float default 0.5; parameter conformation_cutoff_factor integer default 5; parameter torsion_minimize text default "yes"; parameter match_receptor_sites text default "no"; parameter random_search text default "yes";...... parameter maximum_cycles integer default 1; parameter receptor_site_file text default "ece.sph"; parameter score_grid_prefix text default "ece"; parameter ligand_number integer range from 1 to 2000 step 1; Create Dock PlanFile 1. Define Variable and their value Molecules to be screened

74. 74 task nodestart copy./parameter/vdw.defn node:. copy./parameter/chem.defn node:. copy./parameter/chem_score.tbl node:. copy./parameter/flex.defn node:. copy./parameter/flex_drive.tbl node:. copy./dock_inputs/get_molecule node:. copy./dock_inputs/dock_base node:. endtask task main node:substitute dock_base dock_run node:substitute get_molecule get_molecule_fetch node:execute sh./get_molecule_fetch node:execute $HOME/bin/dock.$OS -i dock_run -o dock_out copy node:dock_out./results/dock_out.$jobname copy node:dock_cnt.mol2./results/dock_cnt.mol2.$jobname copy node:dock_chm.mol2./results/dock_chm.mol2.$jobname copy node:dock_nrg.mol2./results/dock_nrg.mol2.$jobname endtask Create Dock PlanFile 2. Define Task that jobs need to do

75. 75 Nimrod/TurboLinux enFuzion GUI tools for Parameter Modeling

76. 76 Docking Experiment Preparation Setup PDB DataGrid Index PDB databases Pre-stage (all) Protein Data Bank (PDB) on replica sites Start PDB Server Create Docking GridScore (receptor surface details) for a given receptor on home node. Pre-Staging Large Files required for Docking: Pre-stage Dock executables and PDB access client on Grid nodes, if required (e.g., dock.Linux, dock.SunOS, dock.IRIX64, and dock.OSF1 on Linux, Sun, SGI, and Compaq machines respectively). Use globus- rcp. Pre-stage/Cache all data files (~3-13MB each) representing receptor details on Grid nodes. This can can be done demand by Nimrod/G for each job, but few input files are too large and they are required for all jobs). So, pre- staging/caching at http-cache or broker level is necessary to avoid the overhead of copying the same input files again and again!

77. 77 Chemical DataBase (CDB) Databases consist of small molecules from commercially available organic synthesis libraries, and natural product databases. There is also the ability to screen virtual combinatorial databases, in their entirety. This methodology allows only the required compounds to be subjected to physical screening and/or synthesis reducing both time and expense.

78. 78 Target Testcase The target for the test case: electrocardiogram (ECE) endothelin converting enzyme. This is involved in “ heart stroke ” and other transient ischemia. Is · che · mi · a : A decrease in the blood supply to a bodily organ, tissue, or part caused by constriction or obstruction of the blood vessels.

79. 79 Scheduling Molecular Docking Application on Grid: Experiment Workload – Docking 200 molecules with ECE 200 jobs, each need in the order of 3 minute depending on molecule weight. Deadline: 60 min. and budget: 50, 000 G$/tokens Strategy: minimise time / cost Execution Cost with cost optimisation Optimise Cost: 14, 277(G$) (finished in 59.30 min.) Optimise Time: 17, 702 (G$) (finished in 34 min.) In this experiment: Time-optimised scheduling costs extra 3.5K$ compared to that of Cost-optimised. Users can now trade-off between Time Vs. Cost.

80. 80 Resources Selected & Price/CPU-sec. Resource & Location Grid services & Fabric Cost/CPU sec. or unit No. of Jobs Executed Time_OptCost_Opt Monash, Melbourne, Australia (Sun Ultra01) Globus, Nimrod-G, GTS (master node) -- AIST, Tokyo, Japan, Ultra-4 Globus, GTS, Fork 144102 AIST, Tokyo, Japan, Ultra-4 Globus, GTS, Fork 241 AIST, Tokyo, Japan, Ultra-4 Globus, GTS, Fork 14239 AIST, Tokyo, Japan, Ultra-2 Globus, GTS, Fork 3114 Sun-ANL, Chicago,US, Ulta-8 Globus, GTS, Fork 16214 Total Experiment Cost (G$)17,70214,277 Time to Complete Exp. (Min.)3459.30

81. 81 DBC Time Opt. Scheduling

82. 82 DBC Scheduling for Time Optimization – No. of Jobs in Exec.

83. 83 DBC Scheduling for Time Optimization – No. of Jobs Finished

84. 84 DBC Scheduling for Time Optimization – Budget Spent

85. 85 DBC Cost Opt. Scheduling

86. 86 DBC Scheduling for Cost Optimization – No. of Jobs in Exec.

87. 87 DBC Scheduling for Cost Optimization – No. of Jobs Finished

88. 88 DBC Scheduling for Cost Optimization – Budget Spent

89. 89 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

90. Grid Simulation Using the GridSim Toolkit Grid Resource Modelling and Application Scheduling Simulation

91. 91 Performance Evaluation: With Large Scenarios Varying the number of Resources (1 to 100s..1000s..) Resource capability Cost (Access Price) Users Deadline Budget Workload Different Time (Peak/Off-Peak) We need repeatable and controllable environment Can this be achieved on Real Grid testbed ?

92. 92 Grid Environment Dynamic: Resource Condition/Availability/Load/Users various with time. Experiment cannot be repeated Resources/Users are distributed and owned by different organization It is hard to create controllable environment. Grid testbed size is limited. Also, creating moderate testbed is resource intensive: time consuming + expensive + need to handle many political problems (access permission). Hence, scheduling algorithm developers turn to Simulation.

93. 93 Discrete-Event Simulation A proven technique: Used in modeling and simulation of real world systems: business  factory assembly line  computer systems design. Allows creation of scalable, repeatable, and controllable environment for large-scale evaluation. Language/Library based simulations tools are available. Simscript, parsec Bricks, MicroGrid, Simgrid, GridSim.

94. 94 The GridSim Toolkit A Java based tool for Grid Scheduling Simulations Basic Discrete Event Simulation Infrastructure Virtual Machine (Java, cJVM, RMI) PCs Clusters Workstations... SMPs Distributed Resources GridSim Toolkit Application Modeling Information Services Resource Allocation Grid Resource Brokers or Schedulers Statistics Resource Modeling and Simulation (with Time and Space shared schedulers) Job Management ClustersSingle CPUReservationSMPsLoad Pattern Application Configuration Resource Configuration User Requirements Grid Scenario Network SimJavaDistributed SimJava Resource Entities Output Application, User, Grid Scenario’s Input and Results

95. 95 GridSim Entities Jobs Appli cation Scheduler User #i Broker #iOutput Input Output Input Resource #j Job In Queue Job Out Queue Process Queue Output Input Resource List Information Service Internet Report Writer #i Statistics Recorder #i Shutdown Signal Manager #i

96. 96 EAEA Output_E A Input_E A EBEB Output_E B Input_E B body() Send(output, data, E B ) … body() … … … … Receive(input, data, E A ) … Timed Event Delivery data, t2 (Deliver data @ t2) GridSim Entities Communication Model

97. 97 Time Shared: Multitasking and Multiprocessing PE1 PE2 G1 G2 G3 G1 G2 G3 P1-G2 P1-G1 P3-G2P1-G3P2-G3 Time G1 G1: Gridlet1 Arrives G1FG3 G1F: Gridlet1 Finishes G2G2FG3F Gridlet1 (10 MIPS) Gridlet2 (8.5 MIPS) Gridlet3 (9.5 MIPS) P2-G2: Gridlet2 finishes at the 2 nd prediction time. P1-G2: Gridlet2 didn’t finish at the 1 st prediction time. Tasks on PEs/CPUs 2 6912 16192622 P2-G2

98. 98 Space Shared: Multicomputing G1 G2 G3 G1G3 G2G3 P1-G1 P1-G2P1-G3 Time G1 G1: Gridlet1 Arrives G1FG3 G1F: Gridlet1 Finishes G2G2FG3F Gridlet1 (10 MIPS) Gridlet2 (8.5 MIPS) Gridlet3 (9.5 MIPS) P1-G2: Gridlet2 finishes as per the 1 st Predication Tasks on PEs/CPUs 2 6912 16192622 PE1 PE2

99. 99 Simulating Economic Grid Scheduler R1R1 RmRm.... CT optimizeCost optimize Time optimize None Opt. Resource Discovery and Trading Gridlet Receptor Dispatcher. 1 6 4 2 7 Experiment Interface 3 5 Scheduling Flow Manager R1R1 R2R2 RnRn User Entity (Broker Resource List and Gridlets Q) GIS Broker Entity Grid Resources

100. 100 User1 Grid Broker Entity Grid Resource Entity (Register Resource) Grid Information Service Entity Grid Shutdown Entity (Get Resource List) (Get Resource Characteristics) (Submit Gridlet1) (Gridlet1 Finished) (Submit Gridlet3) (Submit Gridlet2)[ 1 st, 2 nd, 3 rd time predicted completion time of Gridlet1 ] [Gridlet2 completion event] (Gridlet2 Finished) [Gridlet3 completion event] (Gridlet3 Finished) (I am Done) [If all Users are “Done”] (Terminate) (Get Resource List) (Terminate) Grid Statistics Entity (Record My Statistics) Grid User1 Entity (Submit Expt.) (Done Expt.) Report Writer Entity (Create Report) (Get Stat) (Done) (Terminate) (Asynchronous Event) (Synchronous Event) The delivery of the most recently scheduled internal asynchronous event to indicate the Gridlet completion. Internal asynchronous event is ignored since the arrival of other events has changed the resource scenario. Interactions and Events (Time-shared)

101. 101 User1 Grid Broker Entity Grid Resource Entity (Register Resource) Grid Information Service Entity Grid Shutdown Entity (Get Resource List) (Get Resource Characteristics) (Submit Gridlet1) [Gridlet1 completion event] (Gridlet1 Finished) (Submit Gridlet3) (Submit Gridlet2) [Gridlet2 completion event] (Gridlet2 Finished) [Gridlet3 completion event] (Gridlet3 Finished) (I am Done) [If all Users are “Done”] (Terminate) (Get Resource List) (Terminate) Grid Statistics Entity (Record My Statistics) Grid User1 Entity (Submit Expt.) (Done Expt.) Report Writer Entity (Create Report) (Get Stat) (Done) (Terminate)(Asynchronous Event) (Synchronous Event) Internal Asynchronous Event: scheduled and delivered to indicate the completion of Gridlet. Interactions and Events (Space-shared)

102. 102 Experiment-3 Setup: Using GridSim Workload Synthesis: 200 jobs, each job processing requirement = 10K MI or SPEC with random variation from 0-10%. Exploration of many scenarios: Deadline: 100 to 3600 simulation time, step = 500 Budget: 500 to 22000 G$, step = 1000 DBC Strategies: Cost Optimisation Time Optimisation Resources: Simulated WWG resources

103. 103 Simulated WWG Resources

104. 104 Deadline and Budget-based Cost-Time Opt Scheduling It is a combination of Cost and Time Optimisation Algorithm. Create resource groups (RGs) each containing resources with the same cost as. Sort RGs by increasing cost. For each resource in RG in order, assign as many jobs as possible to the resources using the Time opt scheduling, without exceeding the deadline. Repeat all steps until all jobs are processed.

105. 105 DBC Cost Optimisation No. of Jobs: 200 (Heterogeneous) Job Length: 100 SPEC/MIPS on standard CPU with 0-10% of variation randomly.

106. 106 DBC Time Optimisation

107. 107 Comparison: D = 3100, B varied Time Opt Execution Time vs. Budget Execution Cost vs. Budget Cost Opt

108. 108 WWG Resources in Cost Time Opt

109. 109 Cost-Time Opt Scheduling Deadline is High Budget is High

110. 110 CT Opt: Time and Budget Spent

111. 111 DBC Conservative Time Min. Scheduling 1. Split resources by whether cost per job is less than budget per job. 2. For the cheaper resources, assign jobs in inverse proportion to the job completion time (e.g. a resource with completion time = 5 gets twice as many jobs as a resource with completion time = 10). 3. For the dearer resources, repeat all steps (with a recalculated budget per job) until all jobs are assigned. 4. [Schedule/Reschedule] Repeat all steps until all jobs are processed.

112. 112 Selected GridSim Users!

113. 113 Agenda A quick glance at today ’ s Grid computing Resource Management challenges for Service-Oriented Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Nimrod/G -- Grid Resource Broker Scheduling Experiments on World Wide Grid testbed GridSim Toolkit and Simulations Conclusions SchedulingEconomics Grid Grid Economy

114. Conclude with a comparison to the Electrical Grid ……….. Where we are ???? Courtesy: Domenico Laforenza

115. Alessandro Volta in Paris in 1801 inside French National Institute shows the battery while in the presence of Napoleon I Fresco by N. Cianfanelli (1841) (Zoological Section "La Specula" of National History Museum of Florence University)

116. 116 ….and in the future, I imagine a Worldwide Power (Electrical) Grid …... What ?!?! This is a mad man… Oh, mon Dieu !

117. 117 2002 - 1801 = 201 Years

118. 118 Electric Grid Management and Delivery methodology is highly advanced Central Grid Regional Grid Local Grid Production Utility Consumption Whereas, our Computational Grid is in primitive/infancy state?

119. 119 ” I think there is a world market for about five computers.” Thomas J. Watson Sr., IBM Founder, 1943 Can we Predict its Future ?

120. 120 Summary and Conclusion Grid Computing is emerging as a next generation computing platform for solving large scale problems through sharing of geographically distributed resources. Resource management is a complex undertaking as systems need to be adaptive, scalable, competitive, …, and driven by QoS. We proposed a framework based on “ computational economies ” for resource allocation and for regulating supply-and-demand for resources. Scheduling experiments on the World Wide Grid demonstrate our Nimrod-G broker ability to dynamically lease services at runtime based on their quality, cost, and availability depending on consumers QoS requirements. Easy to use tools for creating Grid applications are essential to attracting and getting application community on board. The use of economic paradigm for resource management and scheduling is essential for pushing Grids into mainstream computing and weaving the World-Wide Grid Marketplace!

121. 121 Download Software & Information Nimrod & Parameteric Computing: http://www.csse.monash.edu.au/~davida/nimrod/ Economy Grid & Nimrod/G: http://www.buyya.com/ecogrid/ Virtual Laboratory Toolset for Drug Design: http://www.buyya.com/vlab/ Grid Simulation (GridSim) Toolkit (Java based): http://www.buyya.com/gridsim/ World Wide Grid (WWG) testbed: http://www.buyya.com/ecogrid/wwg/ Cluster and Grid Info Centres: www.buyya.com/cluster/ || www.gridcomputing.com

122. 122 Further Information Books: High Performance Cluster Computing, V1, V2, R.Buyya (Ed), Prentice Hall, 1999. The GRID, I. Foster and C. Kesselman (Eds), Morgan-Kaufmann, 1999. IEEE Task Force on Cluster Computing http://www.ieeetfcc.org Global Grid Forum www.gridforum.org IEEE/ACM CCGrid ’ xy: www.ccgrid.org CCGrid 2002, Berlin: ccgrid2002.zib.de Grid workshop - www.gridcomputing.org

123. 123 Further Information Cluster Computing Info Centre: http://www.buyya.com/cluster/ Grid Computing Info Centre: http://www.gridcomputing.com IEEE DS Online - Grid Computing area: http://computer.org/dsonline/gc Compute Power Market Project http://www.ComputePower.com

124. 124 Final Word?