1. CSF4 Meta-Scheduler Zhaohui Ding Zhaohui.ding@email.jlu.edu.cn College of Computer Science & Technology Jilin University

2. 2 Agenda What is Meta-Scheduler What is CSF CSF4 Architecture Integrate CSF4 and Gfarm Demo & Practice Future Work

3. 3 Grid Resource Heterogeneous Heterogeneous Dynamic Dynamic Secure Access Secure Access

4. 4 What is Meta-Scheduler CSF4 GRAM

5. 5 Meta-Scheduler VS Local Scheduler Local Scheduler For Cluster Environment For Cluster Environment Same Hardware Architecture, OS Same Hardware Architecture, OS Typically different Resource Management softwares (RMs) like LSF, PBS, and Sun Grid Engine are responsible for load balancing and resource sharing within each local administrative domain. Typically different Resource Management softwares (RMs) like LSF, PBS, and Sun Grid Engine are responsible for load balancing and resource sharing within each local administrative domain.Meta-Scheduler For Grid Environment For Grid Environment Different Hardware, OS, Infrastructure, Local Scheduler, Resource Manager… Different Hardware, OS, Infrastructure, Local Scheduler, Resource Manager… In a Grid-computing environment, there is a common requirement for users to query, negotiate access and manage resources existing within different administrative domains at Grid level. The meta-scheduler is designed to perform such global wide policies. In a Grid-computing environment, there is a common requirement for users to query, negotiate access and manage resources existing within different administrative domains at Grid level. The meta-scheduler is designed to perform such global wide policies.

6. 6 Meta-Scheduler and Local Scheduler in Grid

7. 7 CSF4 Summary What is CSF Community Scheduler Framework Community Scheduler Framework CSF4 is a group of grid services host in GT4 CSF4 is a group of grid services host in GT4 CSF4 is the first GT4-WSRF compliant Meta- Scheduler CSF4 is the first GT4-WSRF compliant Meta- Scheduler Open Source project and can be accessed at http://sourceforge.net/projects/gcsf Open Source project and can be accessed at http://sourceforge.net/projects/gcsf The development team of CSF4 is from Jilin University, PRC The development team of CSF4 is from Jilin University, PRC CSF4 has been added to Globus Toolkit 4 as an Execution Component CSF4 has been added to Globus Toolkit 4 as an Execution Component

8. 8 CSF4 in Globus Toolkit 4

9. 9 CSF4 – Architecture

10. 10 CSF4 – Functionalities Submit, control and monitor jobs at a grid level. Queue Service for jobs is provided Choose satisfied cluster for jobs automatically Create and manage advanced reservations for LSF cluster.

11. 11 CSF4 – Functionalities Resource Supported Local Scheduler Supported LSF (support advanced reservation) LSF (support advanced reservation) PBS PBS SGE SGE Condor Condor Infrastructure Supported WS GRAM (GT4 GRAM) WS GRAM (GT4 GRAM) Pre-WS GRAM (GT2 GRAM) Pre-WS GRAM (GT2 GRAM)

12. 12 CSF4 – Functionalities Scheduling Policy Framework A scheduling policy framework is provided so that site and user specific scheduling policies can be implemented regardless of the underlying resource manager. User can configure scheduling policies for queue, and even write a new scheduling policy as a plug-in without modify the source codes of CSF4.

13. 13 Existent Scheduling Policies Plug- in FCFS plugin First Come First Serve First Come First Serve Default policy Default policy always loaded by CSF always loaded by CSF Throttle plugin An optional plugin An optional plugin Constraint the number of jobs lunched in one scheduling cycle Constraint the number of jobs lunched in one scheduling cycle

14. 14 Schedule plugin & scheduling policies Scheduling polices are defined in a queue Scheduling polices are defined in a queue Each policy is implemented inside a scheduling plugin module Each policy is implemented inside a scheduling plugin module A queue can load multiple plugin modules A queue can load multiple plugin modules

15. 15 CSF4 – Functionalities (cont.) Deploy CSF4 in a Grid Community Multiple CSF4s Can Exchange Cluster Information Each Other

16. 16 Deploy CSF4 in a Grid Community Example:

17. 17 Integration of CSF4 & Gfarm A Scalable Grid Infrastructure For Data Intensive Applications

18. 18 Integration of CSF4 & Gfarm Regardless of where the data is locate host1host3host2host5host4 gfsd

19. 19 CSF4 & Gfarm Resources & Jobs Match Match Job Dispatch Job Dispatch Metascheduling Local Scheduling Global Data Access Interface Job execute

20. 20 How to get CSF4? Release Page (Recommended) : http://sourceforge.net/project/showfiles.php?g roup_id=103105 http://sourceforge.net/project/showfiles.php?g roup_id=103105 http://sourceforge.net/project/showfiles.php?g roup_id=103105 http://sourceforge.net/project/showfiles.php?g roup_id=103105 CVS : http://sourceforge.net/cvs/?group_id=103105 http://sourceforge.net/cvs/?group_id=103105 http://sourceforge.net/cvs/?group_id=103105 Globus Toolkit 4 Package : In “contrib” directory In “contrib” directory

21. 21 Installation Requried Software Requried Software Globus Tookit 4 Globus Tookit 4 $gpt-build csf-4.0.1-src.tar.gz $gpt-build csf-4.0.1-src.tar.gz $gpt-postinstall $gpt-postinstall

22. 22 Configure Meta-Scheduler Edit $GLOBUS_LOCATION/etc/metascheduler /metascheduler-config.xml Example:

23. 23 Configure Resource Edit $GLOBUS_LOCATION/etc/metascheduler/resou rcemanager-config.xml Example: sge-32 sge-32 GRAM GRAM rocks-32.sdsc.edu/jobmanager-sge rocks-32.sdsc.edu/jobmanager-sge 2119 2119 2.4 2.4

24. 24 Services started by GT4 container ( cont.) [globus@rocks-32 gt4]$ globus-start-container [1]: https://198.202.88.32:8443/wsrf/services/TriggerFactoryService [2]: https://198.202.88.32:8443/wsrf/services/DelegationTestService [9]: https://198.202.88.32:8443/wsrf/services/metascheduler/QueuingService …. [16]: https://198.202.88.32:8443/wsrf/services/metascheduler/ResourceManagerFact oryService … [23]: https://198.202.88.32:8443/wsrf/services/metascheduler/ReservationService [25]: https://198.202.88.32:8443/wsrf/services/IndexFactoryService [26]: https://198.202.88.32:8443/wsrf/services/ReliableFileTransferService… … [47]: https://198.202.88.32:8443/wsrf/services/metascheduler/ResourceManagerLsfS ervice … [49]: https://198.202.88.32:8443/wsrf/services/metascheduler/JobService [56]: https://198.202.88.32:8443/wsrf/services/ManagedJobFactoryService

25. 25 Demo & Practice Before testing $ grid-proxy-init $ cp ~zding/practice/*. $ scp ~zding/practice/* $USER@rocks- 170:

26. 26 Demo 1 Query Available Cluster $ csf-job-RmInfo

27. 27 Demo 2 Submit WS GRAM job $ csf-job-create –rsl job.xml -name zding_job1 $ csf-job-submit zding_job1 content of job.xml : <job> /bin/echo /bin/echo ${GLOBUS_USER_HOME} ${GLOBUS_USER_HOME} this is a globusrun-ws sample this is a globusrun-ws sample stdout stdout </job> $ csf-job-data zding_job1 $ csf-job-data zding_job1 $ cat ~/stdout $ cat ~/stdout

28. 28 Demo 3 Submit pre-ws gram job $ csf-job-create –rsl job.rsl –name zding_job2 $ csf-job-create –rsl job.rsl –name zding_job2 $ csf-job-submit zding_job2 $ csf-job-submit zding_job2 content of job.rsl &(executable=/bin/echo) &(executable=/bin/echo) (directory=$(HOME)) (directory=$(HOME)) (arguments="this is a globurun example") (arguments="this is a globurun example") (stdout=stdout2) (stdout=stdout2) $ csf-job-data zding_job2 $ csf-job-data zding_job2 $ cat stdout2 $ cat stdout2

29. 29 Demo 4 Submit job with CSF4 & Gfarm Application: MEME353 $ csf-job-create –rsl job3.rsl –name zding_job3 $ csf-job-create –rsl job3.rsl –name zding_job3 $ csf-job-submit zding_job3 $ csf-job-submit zding_job3 $ csf-job-data zding_job3 $ csf-job-data zding_job3 $ gfarmfs –lsfu /gfs/$USER -o kernel_cache $ gfarmfs –lsfu /gfs/$USER -o kernel_cache $ ls –ltr /gfs/$USER/$USER $ ls –ltr /gfs/$USER/$USER

30. 30 Demo 5 Submit multiple jobs to CSF4 $ for i in 1 2 3 4; do echo $i; csf-job-create -rsl job4.rsl -name job$i; done $ for i in 1 2 3 4; do echo $i; csf-job-submit job$i; done $ for i in 1 2 3 4; do echo $i; csf-job-data job$i; done $ qstat

31. 31 Future Work CSF4 Portlet Base on GridSphere Base on GridSphere CSF4 – Opal Integration Opal: Wrapping Scientific Applications as Web Services Opal: Wrapping Scientific Applications as Web Services Implement data-aware scheduling policy in CSF4 Scheduling policy for data intensive application Scheduling policy for data intensive application Improve the performance Improve the performance Workflow supported

32. 32 Thank you