Presentation is loading. Please wait.

Presentation is loading. Please wait.

© Copyright 2003 The Australian Nimrod Testbed Slavisa Garic (DSTC) Colin Enticott (DSTC) Supervisor: David Abramson (Monash) Utilising GrangeNet Speaker:

Similar presentations


Presentation on theme: "© Copyright 2003 The Australian Nimrod Testbed Slavisa Garic (DSTC) Colin Enticott (DSTC) Supervisor: David Abramson (Monash) Utilising GrangeNet Speaker:"— Presentation transcript:

1 © Copyright 2003 The Australian Nimrod Testbed Slavisa Garic (DSTC) Colin Enticott (DSTC) Supervisor: David Abramson (Monash) Utilising GrangeNet Speaker: Colin Enticott

2 © Copyright 2003 Index DSTC/GrangeNet DSTC/GrangeNet Projects Projects ANT/Enhancement of Nimrod/G V2.0 for GrangeNet ANT/Enhancement of Nimrod/G V2.0 for GrangeNet The Nimrod PortalThe Nimrod Portal Pragma 4 Pragma 4 Resources and resultsResources and results

3 © Copyright 2003 Distributed Systems Technology Centre Distributed Systems Technology Centre The DSTC is a National IT Research and Development Centre focussing on the needs of the Government, Defence, Health, Finance, Telecommunications and Education Sectors. The DSTC is a National IT Research and Development Centre focussing on the needs of the Government, Defence, Health, Finance, Telecommunications and Education Sectors.

4 © Copyright 2003 GrangeNet Grid and next generation Network Grid and next generation Network Develop and operate a multi-gigabit network Develop and operate a multi-gigabit network Support grid and advanced communications services Support grid and advanced communications services 3 year program 3 year program GrangeNet is 'open' to participation of organisations GrangeNet is 'open' to participation of organisations

5 © Copyright 2003 Initial GrangeNet Locations Brisbane Megapop Sydney Megapop Canberra Megapop Melbourne Megapop DSTC QPSF Uni of Queensland Boeing HQ UTS ANU APAC AARNet Uni of Melbourne VPAC I-cubed RMIT eBioinformatics CRCSIT CSIRO BoM SCCN USA link Dual 155M STM-1 link 2.5G PoS link 1 GbE link Monash Uni QUT QMI GriffithU UniSyd UNSW CSIRO IVEC SAPAC TPAC

6 © Copyright 2003 Index DSTC/GrangeNet Projects Projects ANT/Enhancement of Nimrod/G V2.0 for GrangeNet ANT/Enhancement of Nimrod/G V2.0 for GrangeNet The Nimrod PortalThe Nimrod Portal Pragma 4 Pragma 4 Resources and resultsResources and results

7 © Copyright 2003 Nimrod A hunter for computational resources on the grid A hunter for computational resources on the grid Designed to assist parametric problems Designed to assist parametric problems Launching platform Launching platform Performs file copying, execution and maintains parameter sets Performs file copying, execution and maintains parameter sets

8 © Copyright 2003 Nimrod Prof. David Abramson Prof. David Abramson Rok Sosic Rok Sosic Jon Giddy Jon Giddy Slavisa Garic Slavisa Garic

9 © Copyright 2003 Parametric Execution Study the behaviour of some of the output variables against a range of different input scenarios. Study the behaviour of some of the output variables against a range of different input scenarios. Computations are uncoupled (file transfer) Computations are uncoupled (file transfer) Allows real time analysis for many applications Allows real time analysis for many applications More realistic simulations More realistic simulations

10 © Copyright 2003 Dr Dinelli Mather Monash University & MacFarlane Burnett Public Health Policy Genetic Algorithms Dr Shane Dunn, AMRL, DSTO Network Simulation Mahbub Hasan, Monash University Case Studies... Dr Kate Smith, Monash University Neural Network Optimization Cattle Tick

11 © Copyright 2003 Dr Dinelli Mather Monash University & MacFarlane Burnett Public Health Policy Genetic Algorithms Dr Shane Dunn, AMRL, DSTO Network Simulation Mahbub Hasan, Monash University Case Studies... Dr Kate Smith, Monash University Neural Network Optimization Cattle Tick Case Studies... Financial Services Rendering Movies CAD Simulation Parallel MP3 Encoder

12 © Copyright 2003 Dr Dinelli Mather Monash University & MacFarlane Burnett Public Health Policy Genetic Algorithms Dr Shane Dunn, AMRL, DSTO Network Simulation Mahbub Hasan, Monash University Case Studies... Dr Kate Smith, Monash University Neural Network Optimization Cattle Tick Case Studies... Financial Services Rendering Movies CAD Simulation Parallel MP3 Encoder Case Studies... Health Standards Lew Kotler Australian Radiation Protection and Nuclear Safety Agency Factory Simulation John Betts, Monash University Simulated Annealing Optimization Marcus Randall, Bond University Ad-hoc Wireless

13 © Copyright 2003 Nimrod case studies Particles ejecting from a supernova Particles ejecting from a supernova Kuiper belt simulations Kuiper belt simulations Chemical docking 's of combinations Chemical docking 's of combinations GAMESS GAMESS

14 © Copyright 2003 parameter A1 float range from 1 to 4 step 0.1; parameter A2 float range from -1 to 0 step 0.1; parameter B1 float range from 0.5 to 2 step 0.1; parameter B2 float range from 0 to 1 step 0.1; task main copy cart_eth.inp.sub node:. copy spher_eth.inp.sub node:. copy cart_rad.inp.sub node:. copy spher_rad.inp.sub node:. copy pragma4.$OS node:pragma4 copy pragma4.dat node:. substitute cart_eth.inp.sub cart_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute spher_eth.inp.sub spher_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute cart_rad.inp.sub cart_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute spher_rad.inp.sub spher_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp node:execute $HOME/bin/cleanrungms cart_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > cart_eth.out node:execute $HOME/bin/cleanrungms spher_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > spher_eth.out node:execute $HOME/bin/cleanrungms cart_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > cart_rad.out node:execute $HOME/bin/cleanrungms spher_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > spher_rad.out node:execute./pragma4 > pragma4.out node:execute /bin/tar cvf results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar cart_eth.out spher_eth.out cart_rad.out spher_rad.out pragma4.out node:execute /bin/gzip results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar copy node:results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar.gz results/results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar.gz endtask How does a user develop an application using Nimrod? Job 1Job 2Job 3 Job 4Job 5Job 6 Job 7Job 8Job 9 Job 10Job 11Job 12 Job 13Job 14Job 15 Job 16Job 17Job 18 Description of Parameters PLAN FILE

15 © Copyright 2003 Nimrod/G 2.0 Architecture Nimrod/G ClientRsrc. SchedulerNimrod/G Client Grid Information Server(s) RM & TS RM: Local Resource Manager, TS: Trade Server G L Legion enabled node. C Condor enabled node. Nimrod/G GUI Enfuzion API + Database Level 3 Level 2 Level 1 Generator Creator Run File Plan File Job Scheduler Agent Scheduler DB Server Agent Globus Actuator Condor Actuator Legion Actuator Globus enabled node RM & TS Agent Grid Middleware

16 © Copyright 2003 Nimrod/O Performs optimisations Performs optimisations Uses Nimrod or EnFuzion Uses Nimrod or EnFuzion Simplex, Gradient Decent, B.G.F.S., Genetic algorithms, … Simplex, Gradient Decent, B.G.F.S., Genetic algorithms, … Prof. David Abramson Prof. David Abramson Andrew Lewis Andrew Lewis Tom Peachy Tom Peachy

17 © Copyright 2003 Nimrod Dependencies One job cannot start until its dependency jobs have completed One job cannot start until its dependency jobs have completed David Abramson David Abramson Shahaan Ayyub Shahaan Ayyub

18 © Copyright 2003 Active Sheets

19 © Copyright 2003 Active Sheets Prof. David Abramson Prof. David Abramson Paul Roe Paul Roe Gavin Cheuk Gavin Cheuk

20 © Copyright 2003 GriddLeS Legacy Applications are Components Legacy Applications are Components Cannot necessarily modify themCannot necessarily modify them GriddLeS GriddLeS Specification of the interconnections between componentsSpecification of the interconnections between components Interfaces for discovering resources and mapping the computations to themInterfaces for discovering resources and mapping the computations to them Locate data files in the grid and connect the applications to them;Locate data files in the grid and connect the applications to them; Schedule computations on the underlying platforms and making sure the network bandwidth is available andSchedule computations on the underlying platforms and making sure the network bandwidth is available and Monitor the progress of the grid computation and reassign work to other parts of the Grid as necessary.Monitor the progress of the grid computation and reassign work to other parts of the Grid as necessary.

21 © Copyright 2003 Standard Implementation GASS Real time data Data Fusion General Circulation model Regional weather model Photo-chemical pollution modelParticle dispersion model Topography Database Topography Database Vegetation Database Vegetation Database Emissions Inventory Emissions Inventory MPI GASS/GridFTP/GRC MPI Bushfire model GASS

22 © Copyright 2003 GriddLeS Prof. David Abramson Prof. David Abramson Jagan Kommineni Jagan Kommineni

23 © Copyright 2003 Index DSTC/GrangeNetProjects ANT/Enhancement of Nimrod/G V2.0 for GrangeNet ANT/Enhancement of Nimrod/G V2.0 for GrangeNet The Nimrod PortalThe Nimrod Portal Pragma 4 Pragma 4 Resources and resultsResources and results

24 © Copyright 2003 ANT Australian Nimrod Testbed Australian Nimrod Testbed A DSTC project for GrangeNet A DSTC project for GrangeNet Lead by Prof. David Abramson Lead by Prof. David Abramson

25 © Copyright 2003 parameter A1 float range from 1 to 4 step 0.1; parameter A2 float range from -1 to 0 step 0.1; parameter B1 float range from 0.5 to 2 step 0.1; parameter B2 float range from 0 to 1 step 0.1; task main copy cart_eth.inp.sub node:. copy spher_eth.inp.sub node:. copy cart_rad.inp.sub node:. copy spher_rad.inp.sub node:. copy pragma4.$OS node:pragma4 copy pragma4.dat node:. substitute cart_eth.inp.sub cart_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute spher_eth.inp.sub spher_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute cart_rad.inp.sub cart_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp substitute spher_rad.inp.sub spher_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2.inp node:execute $HOME/bin/cleanrungms cart_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > cart_eth.out node:execute $HOME/bin/cleanrungms spher_eth.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > spher_eth.out node:execute $HOME/bin/cleanrungms cart_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > cart_rad.out node:execute $HOME/bin/cleanrungms spher_rad.A1=$A1.A2=$A2.B1=$B1.B2=$B2 > spher_rad.out node:execute./pragma4 > pragma4.out node:execute /bin/tar cvf results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar cart_eth.out spher_eth.out cart_rad.out spher_rad.out pragma4.out node:execute /bin/gzip results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar copy node:results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar.gz results/results.A1=$A1.A2=$A2.B1=$B1.B2=$B2.tar.gz endtask How does a user develop an application using Nimrod? Job 1Job 2Job 3 Job 4Job 5Job 6 Job 7Job 8Job 9 Job 10Job 11Job 12 Job 13Job 14Job 15 Job 16Job 17Job 18 Description of Parameters PLAN FILENimrod Portal

26 © Copyright 2003 Nimrod Portal The portal allows a quick and powerful display of experiments. The portal allows a quick and powerful display of experiments. The status of each experiment can be seen in a glance: The status of each experiment can be seen in a glance: Useable via any web browser that supports secure connections. Shell access not required but is also available. Useable via any web browser that supports secure connections. Shell access not required but is also available. All experiment information on the experiment summary page: All experiment information on the experiment summary page:

27 © Copyright 2003 Nimrod Portal

28 © Copyright 2003 Nimrod Portal

29 © Copyright 2003 Nimrod Portal

30 © Copyright 2003 Plan file editor

31 © Copyright 2003 Nimrod Portal

32 © Copyright 2003 Index DSTC/GrangeNetProjects ANT/Enhancement of Nimrod/G V2.0 for GrangeNet The Nimrod Portal Pragma 4 Pragma 4 Resources and resultsResources and results

33 © Copyright 2003 Pragma 4 Held on the 5 th and 6 th June Held on the 5 th and 6 th June Parameter Scan of an Effective Group Difference Pseudopotential Using Grid Computing Parameter Scan of an Effective Group Difference Pseudopotential Using Grid Computing An experiment by Kim Baldridge and Wibke Sudholt at the University of California, San Diego An experiment by Kim Baldridge and Wibke Sudholt at the University of California, San Diego Using GAMESS Using GAMESS

34 © Copyright 2003 Experiment Resources Machine Name Queue Manager Processors Availa ble OS & Architecture Location hathor.csse.monash.edu.auPBS24Linux x86 Melbourne brecca-2.vpac.orgPBS30Linux x86 Melbourne koume.hpcc.jpGrid Engine4Linux x86 Japan ume.hpcc.jpGrid Engine64Linux x86 Japan amata1.cpe.ku.ac.th SQMS 15Linux x86 Thailand erikson.ucsd.eduPBS74Linux x86 San Diego slic00.sdsc.eduPBS148Linux x86 San Diego hpc420.hpcc.jpPBS14Solaris Sparc Japan tardis.eng.monash.edu.auNQE1Cray SV1 Melbourne sn9280.cray.co.jpNQE1Cray SV1 Japan venus.gridcenter.or.krPBS64Linux x86 Korea jupiter.gridcenter.or.krPBS16Linux x86 Korea apbs.rocksclusters.org PBS 12 Linux x86 San Diego chemcca40.ucsd.edu PBS 36 Linux x86 San Diego

35 © Copyright 2003 World map 5:54am

36 © Copyright 2003 World map 10:35am

37 © Copyright 2003 Number of CPUs

38 © Copyright 2003 Nimrod Viewer

39 © Copyright 2003

40

41 Results 4D input 4D input 1D output 1D output

42 © Copyright 2003

43 Nimrod/O: Local minimas

44 © Copyright 2003 Super Computing 2003 Phoenix Phoenix On now On now

45 © Copyright 2003 New Experiment Resources Machine Name Queue Manager Processors Availa ble OS & Architecture Location lc0.apac.edu.auPBS152Linux x86 Canberra bouscat.cs.cf.ac.ukFork2Linux x86 U.K. bg02n100e00.hpc.cmc.osaka-u.ac.jp PBS156Linux x86 Japan altix16.hpcu.uq.edu.auPBS16Linux ia64 Brisbane globus.is.titech.ac.jp PBS 512Linux x86 Japan sputnik.earth.monash.edu.auPBS16Linux x86 Melbourne cmsdgrid.uohyd.ernet.inFork1Linux x86 India nazgul.qpsf.edu.auSGE7Linux x86 Brisbane wesc9-comsc.grid.cf.ac.ukCondor64IRIX MIPS U.K. grid011.cs.usm.myPBSLinux Malaysia horizon.sdsc.eduSP3 San Diego hypnos.engin.umich.edu254 Michigan

46 © Copyright 2003 Questions?


Download ppt "© Copyright 2003 The Australian Nimrod Testbed Slavisa Garic (DSTC) Colin Enticott (DSTC) Supervisor: David Abramson (Monash) Utilising GrangeNet Speaker:"

Similar presentations


Ads by Google