1. cao@ccrl-nece.deIPDPS 2004 Queue Scheduling and Advance Reservations with COSY Junwei Cao Falk Zimmermann C&C Research Laboratories NEC Europe Ltd. http://www.ccrl-nece.de/

2. cao@ccrl-nece.deIPDPS 2004 Outline Related Work COSY Structure COSY Job Scheduling Scheduling with Advance Reservations Performance Evaluation Experimental Results Ongoing Work

3. cao@ccrl-nece.deIPDPS 2004 Related Work Job Scheduling Systems Catalina, Condor, EASY, LoadLeveler, LSF, Maui, PBS, SGE, STORM, Titan … Scheduling with Advance Reservations W. Smith, I. Foster, and V. Taylor, “Scheduling with Advanced Reservations”, IPDPS 2000. Q. Snell, M. Clement, D. Jackson, and C. Gregory, “The Performance Impact of Advance Reservation Metascheduling”, JSSPP 2000.

4. cao@ccrl-nece.deIPDPS 2004 COSY Structure Operating Systems, MPI Environments (NEC, SCore …), … Users / Applications Resources Generic User Interfaces (submit, release, queue, confirm,…) Job Scheduling Log Management Resource MonitoringDatabase Management Configurations Job Execution Plugin Scripts Specific User Interfaces (mpirun …)

5. cao@ccrl-nece.deIPDPS 2004 COSY Job Scheduling Queue Scheduling: First-come-first-served algorithm Aggressive backfilling Multiple queue support Queue and user priority Resource limitation Access control list Batch and interactive support Advance Reservations: Exact start time Latest end time Two phase commitment

6. cao@ccrl-nece.deIPDPS 2004 Mean queue wait time increases Resource utilization decreases Schedulable with limit percentage of advance reservations Scheduling with Advance Reservations

7. cao@ccrl-nece.deIPDPS 2004 Challenges … How to prevent advance reservations from taking start time advantages over queued jobs? How to satisfy advance reservations as far as possible without sacrificing queue efficiency? Additional scheduling policies have to be defined (in academic/research environments) …

8. cao@ccrl-nece.deIPDPS 2004 COSY Strategies Defining mandatory shortest notice time for advance reservations Using current mean wait time of queued jobs Using times of current mean wait time of queued jobs

9. cao@ccrl-nece.deIPDPS 2004 Performance Evaluation Performance Metrics Queue wait time Resource utilization Rejection rate Workload Modeling Cirne comprehensive model (ANL arrival pattern) 100 requests -> 32 node cluster Time reduced by 180 Experimental Design COSY running with job execution disabled Using different percentages of advance reservations Using different shortest notice time of advance reservations

10. cao@ccrl-nece.deIPDPS 2004 Experimental Results Queue wait time Resource utilization

11. cao@ccrl-nece.deIPDPS 2004 Solutions … Policies applied in COSY COSY takes the current mean wait time of queued jobs as the mandatory AR shortest notice time so that ARs cannot take start time advantages. The mandatory AR shortest notice time is applied in COSY as 1 time of current mean wait time of queued jobs and increases to 4 linearly as the AR percentage increases from 0% (not inclusive) to 15% so that ARs can be satisfied as far as possible without increasing the queue wait time. When the AR percentage is more than 15%, COSY will stop accepting ARs temporarily in order to guarantee a proper queue scheduling. Summary Advantage: adaptive to queue workload. Disadvantage: additional calculation of statistics.

12. cao@ccrl-nece.deIPDPS 2004 Ongoing Work COSY performance evaluation in a larger scale and a longer time period Integration of COSY in GEMSS grid infrastructure to support QoS negotiation Extension of COSY to support standard languages and protocols (e.g. GRAAP and JSDL) COSY-G: job scheduling and workload balancing in grid environments