1. 3 rd EGEE Conference Athens 18th-22nd April EGEE is a project funded by the European Union under contract IST-2003-508833 Geant4 Production in the LCG Environment Patricia Méndez Lorenzo Patricia.Mendez@cern.ch CERN/INFN IT-GD-ED/CNAF EGEE is a project funded by the European Union under contract IST-2003-508833

2. 3 rd EGEE Conference – 18-22 April, 2005 - 2 Summary ≈ The Geant4 Package ≈ Geant4 Physics tests in LCG2 ≈ Framework development for Geant4 ≈ Update of the Framework ≈ Summary and Conclusions Thanks to Alberto Ribon, Jakub Moscicki, Simone Campana and Roberto Santinelli

3. 3 rd EGEE Conference – 18-22 April, 2005 - 3 Geant4 Toolkit Generic Toolkit for Monte Carlo simulation of particle interactions with the matter (i.e. detectors) ◘ Application domains: ➙ High-Energy Physics: ATLAS, CMS and LHCb (LHC), BaBar (SLAC), etc ➙ Space Radiation: ESA ➙ Medical Physics: Proton and brachy therapies, etc ◘ Object-Oriented (C++) project, modular and extensible. Significant improved with respect its predecessor, Geant3, not only from the software structure, but mainly for the physics coverage ◘ Electromagnetic physics of Geant4 and even more Hadronic physics are complex fields. It is fundamental to test their models covering the widest possible range of particles, materials and energies Here appears the Grid Contribution

4. 3 rd EGEE Conference – 18-22 April, 2005 - 4 Geant4 Physics Tests on LCG2 ◘ Electromagnetic and Hadronic physics are fundamental features to be properly simulated in High-Energy Physics and medical applications. However they are extremely CPU demanding ▪ Number of events and energy depending: 1 event of 1GeV ~ 0.03 sec (2.4 GHz) 1 event of 300 Gev ~ 9-10 sec ◘ Goal: Comparison some shower observables between the two different Geant4 versions (7.0.cand01 vs 6.2.p02 ) and check statistical significant changes ◘ Samplings: ▪ 7 simplified detectors (sampling calorimeters) ▪ 7 different beam energies ▪ 23 different beam energies ▪ 5 physics list ◘ First application on LCG resources last December (2004). ▪ A total amount of about 3 years of CPU time (1GHz machine) ▪ Less than 15 GByte of output per Geant4 version 5635 jobs per Geant4 version

5. 3 rd EGEE Conference – 18-22 April, 2005 - 5 Geant4 Production in LCG ◘ Stages: 1. Software installation: Installation of the Geant4 packages (with all the required external additional packages: PI, AIDA, etc) ▪ Software provided via a tar file ▪ Installation through jobs using specific LCG tools ▪ Fundamental request for the sites: Shared area between WNs and perfectly definition of the software installation region 2. Events production: 5635 * 2 jobs (production of 5000 events in each job) 3. Analysis: Statistical tests to perform the comparison between the two Geant4 versions ◘ General Characteristics: ▪ VO: dteam (6 certificates, one as dteamsgm) ▪ Sites and middleware operating system: RedHat7.3 ▪ Own RB + BDII + UI: lxb2006 at CERN ▪ All output (~ 30 GByte) stored at CERN (lxn1183)

6. 3 rd EGEE Conference – 18-22 April, 2005 - 6 Framework Development for Geant4 Generation of a general framework consisting of 3 major tools: ◘ Tool for general and automatic job submission ◘ Tool for events generation in all those sites where the software has been installed ◘ Tool for data analysis ● First Part: Tool for job submissionMethodology: ▪ Copy and registry of the Geant4 package - A file containing the TURL is created and is passed to the WN ▪ Follow up of candidates able to admit Geant4 jobs ▪ Selection of long queues only ▪ Automatic built of the.jdl files for each long queue - Built taking as base those proposed by the user joining the name of the queue where to submit the job - Software Installation tools are used to perform the installation ▪ Submission of these files to each queue

7. 3 rd EGEE Conference – 18-22 April, 2005 - 7 Framework Development for Geant4 Software Installation tool (Tool submitted in the first step to all sites to install the software) ◘ First step: - The tar file is copied from the SE at CERN to the WN - It is untar and copied to the VO_DTEAM_SW_DIR area ◘ Second Step: Software Installation tool - A small production (small number of events) is performed, together with some statistical tests - If succeeded a tag is published in the Information System Results: √ The software installation was tried in 63 sites 8 sites were having submission problems 27 sites did not have defined the VO_DTEAM_SW_DIR area or did not have shared area among WNs In 28 sites the installation succeeded

8. 3 rd EGEE Conference – 18-22 April, 2005 - 8 Framework Development for Geant4 Second Step: Tool for the ProductionStrategy: ◘ Only long queues will be used to run the production ◘ All outputs (hbook files) will be stored at CERN: lxn1183.cern.ch ◘ Submission in bunches of 1127 jobs; one bunch per physics list ◘ Two RBs (lxn1177 and lxb2006 at CERN) have been usedMethodology: Geant4 provides their own code to perform the events production Python Script for each type of particle, energy, physics list and calorimeter is created by the framework from one template provided by Geant4 Generation of one jdl per job containing the code provided by Geant4 (the same for all jobs) + that script generated by the framework and changing for each job Submission of all jdl files to all sites containing the Geant4 installation

9. 3 rd EGEE Conference – 18-22 April, 2005 - 9 Framework Development for Geant4 Results: A hbook file containing 5000 event is created in the case the production succeeded The name of the file is created by the framework containing the type of particles, the energy, the physics list and the calorimeter within the name (important to perform later the comparison) The hbook file is copied and registered to a disk at CERN Around 4508 jobs (two physics list for both Geant4 versions) were run in lest than 2 weeks in 28 sites with a efficiency of about 87%

10. 3 rd EGEE Conference – 18-22 April, 2005 - 10 Framework Development for Geant4 Before the analysis stage the outputs should be checked: Dealing with about 5000 outputs is not an easy task Tool able to print in a file all the LFNs in the case the efficiency was 100% (reference file) Output retrieve (only std.out files). If succeeded the file will contain the name of the LFN A 2 nd tool will check all std.out files looking for the succeeded jobs - The corresponding GUID and LFN will be stored (test file) and compared with the information included in the reference file At this point it was more important for us to analyze the successful jobs than to understand the cause of the unsuccessful ones

11. 3 rd EGEE Conference – 18-22 April, 2005 - 11 Framework Development for Geant4 Third Step: Tool for the AnalysisMethodology: Search of common successful outputs in both Geant4 Versions Each couple of successful outputs are copied into a local area and analyzed with their own tools Finally the copy is removed from the local area

12. 3 rd EGEE Conference – 18-22 April, 2005 - 12 Update of the Framework ◘ This framework covered the Geant4 requirements for its first production ◘ It’s not useful for larger production - Difficult to deal with the output and visualize the results ◘ A new complete tool has been developed for large production - Flexible enough to be used for any VO and any user application - Most of the improvements mostly relative to the outputs handle Documentation: “LCG2 User Guide” http://grid-deployment.web.cern.ch/grid-deployment/cgi- bin/index.cgi?var=eis/docs Download: http://goc.grid.sinica.edu.tw/gocwiki/User_tools

13. 3 rd EGEE Conference – 18-22 April, 2005 - 13 Update of the Framework The new framework consists mainly of two tools: 1. Tool to perform the automatic job submission 2. Tool to retrieve and handle the corresponding output 1. Automatic job submission Overview: Given an user’s jdl this tool performs the following actions: It lists all sites able to run the jdl provided by the user It creates automatically a jdl file based on that provided by the user It submits the just created jdl containing the user application(s) Moreover it creates a subdirectory (defined by the user) containing a list of the sites where the jobs have been submitted, the corresponding jdls and the jobs IDs

14. 3 rd EGEE Conference – 18-22 April, 2005 - 14 Update of the Framework Additional Features: The user can define the queues where the jobs are submitted. These queues are checked to see whether it fixes the job requirements. Requested LFN files can be included. The corresponding TURLs are searched and included in a file passed in the InputSandbox to the WN 2. Retrieve and handle of the outputs The 2 nd tool checks the status of the jobs from the job IDs included in the directory given by the user It provides the following output: The job run in ramses.dcic.ups.es:2119/jobmanager-torque-dteam is in status: Scheduled The job run in grid01.phy.ncu.edu.tw:2119/jobmanager-torque-dteam is in status: running The job run in scaic10.scai.frauhofer.de:2119/jobmanager-torque-dteam is in status: over The user is queried to retrieve the output to the destination he has previously decided

15. 3 rd EGEE Conference – 18-22 April, 2005 - 15 Update of the Framework Additional Features: It is possible to visualize the outputs on the web A html report is provided showing the files decided by the user

16. 3 rd EGEE Conference – 18-22 April, 2005 - 16 Summary and Conclusions In lest than 4 weeks Geant4 was able to run a large number of events in 28 sites (including software installation, production and data analysis) Efficiency = 87% It was developed a general framework fixing those Geant4 requirements This framework has been extended to be used by any VO for any general job submission - General job submission tool - Easy output handle Already available for the user Geant4 is regularly updated in LCG. The Biomed Community can always count on any Geant4 version(s) in the LCG Sites if needed.

17. Experiment Software Installation Tank & Spark It consists of three different components: Tank : =multithread (gSOAP based) service (running on the CE) listening for GSI-authenticated (and non) connections Spark :=client application running on each WN (through a cronjob and/or through a normal “grid-job” from lcg- ManageSoftware) and contacting tank for retrieve/insert/delete software informations. R-sync server running on another machine (a SE for instance) and acting as central repository of the software.

18. Experiment Software Installation SE TANK CE WN ESM 1 JDL-installation job from ESM arrives on CE 2 SPARK ESM requests ends up on WN that becomes SPARK The software (here labeled as “c”) is installed locally through the middle layer lcg- ManageSoftware. A pre-validation is highly recommended before triggering the propagation. The Information System is upgraded 3 Spark-client program is called. Delegated credentials of the ESM are checked in TANK. SPARK ask for a sw tag registration in TANK central DB. 4 TANK registers the new tag and synchronize through R-SYNC the new directory created in SPARK in a central repository 5 TANK is contacted by all WNs one at the time External conditions are checked. Special site policies can be taken into account. Local installation on WNs is triggered. No authentication is required : each WN trusts TANK. 7 At the end of the whole process TANK will e-mail the ESM indicating the result of the installation; the Information System is upgraded accordingly to the result of the process Site Firewall ab abc “c” 6