CSF4 Meta-Scheduler Zhaohui Ding College of Computer Science & Technology Jilin University

2 Agenda What is Meta-Scheduler What is CSF CSF4 Architecture Integrate CSF4 and Gfarm Demo & Practice Future Work

3 Grid Resource Heterogeneous Heterogeneous Dynamic Dynamic Secure Access Secure Access

4 What is Meta-Scheduler CSF4 GRAM

5 Meta-Scheduler VS Local Scheduler Local Scheduler For Cluster Environment For Cluster Environment Same Hardware Architecture, OS Same Hardware Architecture, OS Typically different Resource Management softwares (RMs) like LSF, PBS, and Sun Grid Engine are responsible for load balancing and resource sharing within each local administrative domain. Typically different Resource Management softwares (RMs) like LSF, PBS, and Sun Grid Engine are responsible for load balancing and resource sharing within each local administrative domain.Meta-Scheduler For Grid Environment For Grid Environment Different Hardware, OS, Infrastructure, Local Scheduler, Resource Manager… Different Hardware, OS, Infrastructure, Local Scheduler, Resource Manager… In a Grid-computing environment, there is a common requirement for users to query, negotiate access and manage resources existing within different administrative domains at Grid level. The meta-scheduler is designed to perform such global wide policies. In a Grid-computing environment, there is a common requirement for users to query, negotiate access and manage resources existing within different administrative domains at Grid level. The meta-scheduler is designed to perform such global wide policies.

6 Meta-Scheduler and Local Scheduler in Grid

7 CSF4 Summary What is CSF Community Scheduler Framework Community Scheduler Framework CSF4 is a group of grid services host in GT4 CSF4 is a group of grid services host in GT4 CSF4 is the first GT4-WSRF compliant Meta- Scheduler CSF4 is the first GT4-WSRF compliant Meta- Scheduler Open Source project and can be accessed at Open Source project and can be accessed at The development team of CSF4 is from Jilin University, PRC The development team of CSF4 is from Jilin University, PRC CSF4 has been added to Globus Toolkit 4 as an Execution Component CSF4 has been added to Globus Toolkit 4 as an Execution Component

8 CSF4 in Globus Toolkit 4

9 CSF4 – Architecture

10 CSF4 – Functionalities Submit, control and monitor jobs at a grid level. Queue Service for jobs is provided Choose satisfied cluster for jobs automatically Create and manage advanced reservations for LSF cluster.

11 CSF4 – Functionalities Resource Supported Local Scheduler Supported LSF (support advanced reservation) LSF (support advanced reservation) PBS PBS SGE SGE Condor Condor Infrastructure Supported WS GRAM (GT4 GRAM) WS GRAM (GT4 GRAM) Pre-WS GRAM (GT2 GRAM) Pre-WS GRAM (GT2 GRAM)

12 CSF4 – Functionalities Scheduling Policy Framework A scheduling policy framework is provided so that site and user specific scheduling policies can be implemented regardless of the underlying resource manager. User can configure scheduling policies for queue, and even write a new scheduling policy as a plug-in without modify the source codes of CSF4.

13 Existent Scheduling Policies Plug- in FCFS plugin First Come First Serve First Come First Serve Default policy Default policy always loaded by CSF always loaded by CSF Throttle plugin An optional plugin An optional plugin Constraint the number of jobs lunched in one scheduling cycle Constraint the number of jobs lunched in one scheduling cycle

14 Schedule plugin & scheduling policies Scheduling polices are defined in a queue Scheduling polices are defined in a queue Each policy is implemented inside a scheduling plugin module Each policy is implemented inside a scheduling plugin module A queue can load multiple plugin modules A queue can load multiple plugin modules

15 CSF4 – Functionalities (cont.) Deploy CSF4 in a Grid Community Multiple CSF4s Can Exchange Cluster Information Each Other

16 Deploy CSF4 in a Grid Community Example:

17 Integration of CSF4 & Gfarm A Scalable Grid Infrastructure For Data Intensive Applications

18 Integration of CSF4 & Gfarm Regardless of where the data is locate host1host3host2host5host4 gfsd

19 CSF4 & Gfarm Resources & Jobs Match Match Job Dispatch Job Dispatch Metascheduling Local Scheduling Global Data Access Interface Job execute

20 How to get CSF4? Release Page (Recommended) : roup_id= roup_id= roup_id= roup_id= CVS : Globus Toolkit 4 Package : In “contrib” directory In “contrib” directory

21 Installation Requried Software Requried Software Globus Tookit 4 Globus Tookit 4 $gpt-build csf src.tar.gz $gpt-build csf src.tar.gz $gpt-postinstall $gpt-postinstall

22 Configure Meta-Scheduler Edit $GLOBUS_LOCATION/etc/metascheduler /metascheduler-config.xml Example:

23 Configure Resource Edit $GLOBUS_LOCATION/etc/metascheduler/resou rcemanager-config.xml Example: sge-32 sge-32 GRAM GRAM rocks-32.sdsc.edu/jobmanager-sge rocks-32.sdsc.edu/jobmanager-sge

24 Services started by GT4 container ( cont.) gt4]$ globus-start-container [1]: [2]: [9]: …. [16]: oryService … [23]: [25]: [26]: … [47]: ervice … [49]: [56]:

25 Demo & Practice Before testing $ grid-proxy-init $ cp ~zding/practice/*. $ scp ~zding/practice/* 170:

26 Demo 1 Query Available Cluster $ csf-job-RmInfo

27 Demo 2 Submit WS GRAM job $ csf-job-create –rsl job.xml -name zding_job1 $ csf-job-submit zding_job1 content of job.xml : <job> /bin/echo /bin/echo ${GLOBUS_USER_HOME} ${GLOBUS_USER_HOME} this is a globusrun-ws sample this is a globusrun-ws sample stdout stdout </job> $ csf-job-data zding_job1 $ csf-job-data zding_job1 $ cat ~/stdout $ cat ~/stdout

28 Demo 3 Submit pre-ws gram job $ csf-job-create –rsl job.rsl –name zding_job2 $ csf-job-create –rsl job.rsl –name zding_job2 $ csf-job-submit zding_job2 $ csf-job-submit zding_job2 content of job.rsl &(executable=/bin/echo) &(executable=/bin/echo) (directory=$(HOME)) (directory=$(HOME)) (arguments="this is a globurun example") (arguments="this is a globurun example") (stdout=stdout2) (stdout=stdout2) $ csf-job-data zding_job2 $ csf-job-data zding_job2 $ cat stdout2 $ cat stdout2

29 Demo 4 Submit job with CSF4 & Gfarm Application: MEME353 $ csf-job-create –rsl job3.rsl –name zding_job3 $ csf-job-create –rsl job3.rsl –name zding_job3 $ csf-job-submit zding_job3 $ csf-job-submit zding_job3 $ csf-job-data zding_job3 $ csf-job-data zding_job3 $ gfarmfs –lsfu /gfs/$USER -o kernel_cache $ gfarmfs –lsfu /gfs/$USER -o kernel_cache $ ls –ltr /gfs/$USER/$USER $ ls –ltr /gfs/$USER/$USER

30 Demo 5 Submit multiple jobs to CSF4 $ for i in ; do echo $i; csf-job-create -rsl job4.rsl -name job$i; done $ for i in ; do echo $i; csf-job-submit job$i; done $ for i in ; do echo $i; csf-job-data job$i; done $ qstat

31 Future Work CSF4 Portlet Base on GridSphere Base on GridSphere CSF4 – Opal Integration Opal: Wrapping Scientific Applications as Web Services Opal: Wrapping Scientific Applications as Web Services Implement data-aware scheduling policy in CSF4 Scheduling policy for data intensive application Scheduling policy for data intensive application Improve the performance Improve the performance Workflow supported

32 Thank you