1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org CHARON System  http://egee.cesnet.cz/en/voce/Charon.html Petr Kulhánek, 1,2 Jan Kmuníček, 1,3 and Martin Petřek 1,2 1. CESNET, Czech Republic 2. National Centre for Biomolecular Research, Masaryk University in Brno, Czech Republic 3.Institute of Computer Science, Masaryk University in Brno, Czech Republic

2. EGEE project is funded by European Commission (contract number IST-2003-508833) 2 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Contents  GRID Vocabulary  Summary of User Authentication/Authorization in GRID  Job Submission in Native LCG Middleware  Charon Extension Layer  Overview  Module System  Job Submission and Management  Configuration  Conclusions  Examples are in: skurut4.cesnet.cz:/home/kulhanek/cesnet_seminar/  input directory contains fresh input data  test directory contains results of test run

3. EGEE project is funded by European Commission (contract number IST-2003-508833) 3 Enabling Grids for E-sciencE INFSO-RI-508833 GRID Vocabulary VOCE– Virtual Organization for Central Europe UI – user interface SE – storage element CE – computing element WN – worker node CHARON System

4. EGEE project is funded by European Commission (contract number IST-2003-508833) 4 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Summary of User Authentication/Authorization in GRID grid-proxy-init or grid-proxy-init-sc grid-proxy-info grid-proxy-destroy myproxy-init or myproxy-init-sc myproxy-info myproxy-get-delegation myproxy-destroy NOTE: grid-proxy-init-sc and myproxy-init-sc are not a regular part of LCG distribution. They serve for proxy certificate initialization using secure cards (HW Tokens).

5. EGEE project is funded by European Commission (contract number IST-2003-508833) 5 Enabling Grids for E-sciencE INFSO-RI-508833 Job Submission in Native LCG Middleware Example #01: 01_lcg_test edg-job-submit lcg_job.jdl edg-job-statushttps://skurut3.cesnet.cz:9000/AXyE-vQ04qcFfM-kQFe6QQ edg-job-get-output–dir./ https://skurut3.cesnet.cz:9000/AXyE-vQ04qcFfM-kQFe6QQ CHARON System # LCG Test Job (lcg_job.jdl) Type = "Job"; JobType = "Normal"; Executable = "/bin/hostname"; StdOutput = "lcg_job.stdout"; StdError = "lcg_job.stdout"; OutputSandbox = {"lcg_job.stdout"}; Arguments = ""; RetryCount = 2;

6. EGEE project is funded by European Commission (contract number IST-2003-508833) 6 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System What is Charon?  uniform and modular approach for (complex) computational jobs submission and management  generic system for use of application programs in the Grid environment (LCG middleware, …) Why Charon?  many various batch systems & scheduling components used in grid environment  each batch system has unique tools and different philosophy of its utilization  LCG provided tools are quite raw and simple  many additional tasks to use computer resources properly

7. EGEE project is funded by European Commission (contract number IST-2003-508833) 7 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Scheme of Charon (CEL - Charon Extension Layer)

8. EGEE project is funded by European Commission (contract number IST-2003-508833) 8 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Activation of CEL on UI of VOCE 1. login to skurut4.cesnet.cz 2. type following command: /afs/ics.muni.cz/software/ncbr/voce/migrating/install_charon 3. logout from skurut4.cesnet.cz 4. login to skurut4.cesnet.cz again NOTE: CEL installer modify.profile and.bashrc files. Deactivation of CEL on UI of VOCE 1. login to skurut4.cesnet.cz 2. type following command: /afs/ics.muni.cz/software/ncbr/voce/migrating/uninstall_charon

9. EGEE project is funded by European Commission (contract number IST-2003-508833) 9 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Module system –manages application software  each software package is described by a specific module  configuration information is internally stored in XML format  find the architecture and parallel mode that best fit available computational resources  solve conflicts or dependencies between individual modules  list available modules sorted into categories  use pre-installed modules on WNs or install them on the fly if they are missing

10. EGEE project is funded by European Commission (contract number IST-2003-508833) 10 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Commands of Module System –module command: module [action] [module1 [module2] …] Most Important Actions: add (load) remove (unload) avail list* * list is default action NOTE: Type module -h for short summary info about module command. –modview command  menu driven configuration of visualization of modules

11. EGEE project is funded by European Commission (contract number IST-2003-508833) 11 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Module Name Completion Example: amber  amber:8.0m:auto:auto  amber:8.0m:pn3:single user  default values  resolved final name pn3 – module is optimised for Pentium III architecture single – module is built for sequential execution

12. EGEE project is funded by European Commission (contract number IST-2003-508833) 12 Enabling Grids for E-sciencE INFSO-RI-508833 Module Activation $ module add amber Module specification: amber (add action) =========================================================== Architecture : pn3 Number of CPUs : 1 Max CPUs per node : 1 Exported module : amber:8.0m Complete module : amber:8.0m:pn3:single INFO: Module is taken from local software repository. CHARON System

13. EGEE project is funded by European Commission (contract number IST-2003-508833) 13 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Charon System – Overview of Commands Job Submission and Management psubmit pinfo psync pgo(does not work in GRID environment) Charon Setup pconfigure palias (deprecated, it will be substituted with pconfigure) pprofile (deprecated, it will be substituted with pconfigure) psum

14. EGEE project is funded by European Commission (contract number IST-2003-508833) 14 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Charon Job Restrictions each job has to be in separate directory job directories must not overlap job is described by script only relative paths to job directory contents have to be used in job script software should be activated by Module System

15. EGEE project is funded by European Commission (contract number IST-2003-508833) 15 Enabling Grids for E-sciencE INFSO-RI-508833 Example of Charon Job (Example #02: 02_isomaltose) CHARON System

16. EGEE project is funded by European Commission (contract number IST-2003-508833) 16 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System psubmit [NCPU] [syncmode]

17. EGEE project is funded by European Commission (contract number IST-2003-508833) 17 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System pinfo

18. EGEE project is funded by European Commission (contract number IST-2003-508833) 18 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System psync

19. EGEE project is funded by European Commission (contract number IST-2003-508833) 19 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Results

20. EGEE project is funded by European Commission (contract number IST-2003-508833) 20 Enabling Grids for E-sciencE INFSO-RI-508833 CHARON System Other Examples Example #03: 03_complex_job psubmit skurut get_pdbs Example #04: 04_parallel_sander psubmit skurut md_test 2

21. EGEE project is funded by European Commission (contract number IST-2003-508833) 21 Enabling Grids for E-sciencE INFSO-RI-508833 Charon Configuration Sync Mode – option for data transfer between UI and WN gridcopy – all data within job directory as input – all data within job directory as result stdout – all data within job directory as input – only standard output as result (other data are discarded) Resources – identification of particular CE Properties – fine grained selection of computational resources (throught Requirements item in JDL) Alias -uniform combination of above setup in a single word CHARON System

22. EGEE project is funded by European Commission (contract number IST-2003-508833) 22 Enabling Grids for E-sciencE INFSO-RI-508833 palias CHARON System $ palias Alias Queue/VO Syncmode Properties Resources -------------------------------------------------------------------------------------- stdout voce stdout - - tuke voce gridcopy - ce.grid.tuke.sk:2119/jobmanager-pbs-voce polgrid voce gridcopy - ce.polgrid.pl:2119/jobmanager-lcgpbs-voce poznan voce gridcopy - ce.egee.man.poznan.pl:2119/jobmanager-lcgpbs-voce skurut voce gridcopy - skurut17.cesnet.cz:2119/jobmanager-lcgpbs-voce zeus voce gridcopy - zeus02.cyf-kr.edu.pl:2119/jobmanager-lcgpbs-voce

23. EGEE project is funded by European Commission (contract number IST-2003-508833) 23 Enabling Grids for E-sciencE INFSO-RI-508833 Next Release of Charon System (2006) gLite support more batch systems support within single Charon installation simplification of configuration (e.g. only pconfigure will remain) better handling of error states documentation in WIKI What Do We Need ? What type of application software is required by users? What data (type and size) does this software produce or operate with? Do you require parallel execution of application software? suggestions and criticism of Charon System

24. EGEE project is funded by European Commission (contract number IST-2003-508833) 24 Enabling Grids for E-sciencE INFSO-RI-508833 Conclusions –Single job management  encapsulation of a single computational job  minimization of overhead resulting from direct middleware usage (JDL file preparation, etc.)  easy submission and navigation during job lifetime –Application programs management  powerful software management and administration  comfortable enlargement of available application portfolio  Acknowledgements  European Commission (contract number IST-2003-508833)  Ministry of Education, Youth, and Physical Training of the Czech Republic (contract number MSM0021622413) CHARON System