Presentation is loading. Please wait.

Presentation is loading. Please wait.

Job Submission Using PBSPro and Globus Job Commands.

Similar presentations


Presentation on theme: "Job Submission Using PBSPro and Globus Job Commands."— Presentation transcript:

1 Job Submission Using PBSPro and Globus Job Commands

2 Overview Computational Resources Queuing Systems PBSPro Globus Toolkit

3 Computational Resources Hard Disks (permanent storage) Number of CPUs (processing power) CPU time (processing time) Physical memory (program size) All computers have limited resources These resources may span across multiple processors/machines How to allocate these resources fairly, amongst the many users?

4 Queuing Systems Holding area for pending requests A method for allocating the needed resources based on a user request Processing of requests dynamically, quickly and fairly Many different implementations

5 PBSPro Queuing System used to control the allocation of computational resources to user submitted jobs Allows optimal sharing of all resources Ensures that the limited resources aren’t over-run and exceeded Unattended processing of requests

6 PBS Queues Allows distribution of resources into clearly defined groups, called queues Queues can be defined by:  Maximum CPU time  Number of CPUs available  Memory needed  Concurrently executing jobs Queuing schemes evolve over time as user requests and workload vary

7 Interacting With PBS qsub – Submit a job to the queues qstat – Check the status of a job qdel – Delete a job from the queues qmgr – Create/modify queue settings xpbs – Monitor all queues and jobs

8 qsub – Submit A Job To submit a job to the Queuing System, use “qsub” E.g. qsub test.sub Test.sub is a script file containing the commands to be executed qsub returns your “QueueID”, if the job was submitted successfully: master

9 Submit File For A Serial Job !/bin/bash #PBS -l walltime=2:00:00 #PBS -l mem=5mb #PBS -j oe #PBS –m be cd Serial./test

10 Submit File For A Parallel Job #!/bin/sh #PBS -l nodes=4:ppn=2 #PBS -l walltime=48:00:00 #PBS -j oe cd./MPI/Examples mpiexec./test

11 qstat – Check Job Status To check the status of a job submitted to the Queuing System, use “qstat” Job id Name User Time S Queue master G5D2C ngs :00:40 R cpu master STDIN ngs :58:40 R cpu master STDIN ngs :15:42 R cpu master STDIN ngs :15:02 R cpu master STDIN ngs :13:42 R cpu master STDIN ngs :14:13 R cpu master Co_V ngs :57:53 R cpu master 20_12 ngs :31:17 R cpu master 20_21 ngs :11:59 R cpu master 20_23 ngs :05:51 R cpu master 20_25 ngs :03:09 R cpu1

12 qdel – Delete A Job To delete a job submitted to the Queuing System, use “qdel” E.g. qdel QueueID qdel QueueID1 QueueID2

13 Globus Toolkit An open source toolkit for developing Grid based applications and connectivity Allocating computational resources on remote (Globus aware) machines for the execution of user submitted jobs

14 Globus Job Commands globus-job-run globus-job-submit globus-job-status URL globus-job-get-output URL

15 globus-job-run Allows you to run a job as though it were interactive, on a local or remote machine Don’t actually need to log on to the machine itself Not submitted to the Queuing System Returns the programs output as though you were running interactively

16 globus-job-run Examples globus-job-run grid-data.man.ac.uk /bin/date globus-job-run grid-data.rl.ac.uk./test globus-job-run \ grid-compute.leeds.ac.uk/jobmanager-pbs \ -np 8 -x ‘(jobtype=mpi)(environment= \ (NGSMODULES clusteruser))’./MPI/test globus-job-run grid-data.rl.ac.uk -s./test

17 globus-job-submit To submit a job through a Globus Job Manager Commands returns a URL to the program’s output https://grid-compute.leeds.ac.uk:64167/5291/ / https://grid-compute.leeds.ac.uk:64167/5291/ / The status and output of the job can be tested through this URL

18 globus-job-submit Examples globus-job-submit \ grid-data.rl.ac.uk/jobmanager-pbs./test globus-job-submit \ grid-compute.leeds.ac.uk/jobmanager-pbs \ -x ‘(jobtype=mpi)(directory=/home/bob/mpi) \ (environment=(NGSMODULES clusteruser)) \ (count=8)’./mpi_program globus-job-submit \ grid-data.man.ac.uk/jobmanager-pbs -s./test

19 globus-job-status URL To get the status of a submitted job globus-job-status \ https://gridcompute.leeds.ac.uk:64167/5291/ / https://gridcompute.leeds.ac.uk:64167/5291/ / Returns:  Pending, Active, Done, Failed

20 globus-job-get-output URL To retrieve the output of a job submitted through a Globus Job Manager globus-job-get-output \ https://gridcompute.leeds.ac.uk:64167/5291/ / https://gridcompute.leeds.ac.uk:64167/5291/ / Returns the output of the program to the console


Download ppt "Job Submission Using PBSPro and Globus Job Commands."

Similar presentations


Ads by Google