1. Steve LloydGridPP13 Durham July 2005 Slide 1 Using the Grid Steve Lloyd Queen Mary, University of London

2. Steve LloydGridPP13 Durham July 2005 Slide 2 Getting Started http://ca.grid-support.ac.uk/ 1. Get a digital certificate 2. Join a Virtual Organisation (VO) For LHC join LCG and choose a VO 3. Get access to a local User Interface Machine (UI) and copy your files and certificate there Authentication – who you are http://lcg-registrar.cern.ch/ Authorisation – what you are allowed to do > source /afs/cern.ch/project/gd/LCG-share/2.4.0/sl3/etc/profile.d/grid_env.sh Lxplus:

3. Steve LloydGridPP13 Durham July 2005 Slide 3 Overview Job Description UI Machine Resource Broker Input Sandbox Script you want to run Other files (Job Options, Source...) Storage Element Compute Element Storage Element Grid Proxy Certificate Output Sandbox Output files (Ntuples, Logs...) Input Data Output Data Job Output Sandbox

4. Steve LloydGridPP13 Durham July 2005 Slide 4 Job Preparation ############# athena.jdl ################# Executable = "athena.sh"; StdOutput = "athena.out"; StdError = "athena.err"; InputSandbox = {"athena.sh", "MyJobOptions.py", "MyAlg.cxx", "MyAlg.h", "MyAlg_entries.cxx", "MyAlg_load.cxx", "login_requirements", "requirements", "Makefile"}; OutputSandbox = {"athena.out","athena.err", "ntuple.root", "histo.root", "CLIDDBout.txt"}; Requirements = Member("VO-atlas-release-10.0.1", other.GlueHostApplicationSoftwareRunTimeEnvironment); ################################################ Input files Output Files Choose ATLAS Software Version Prepare a file of Job Description Language (JDL): My C++ CodeJob OptionsScript to run #############Hello World################# Executable = "/bin/echo"; Arguments = "Hello Steve welcome to the Grid "; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; ######################################### Simplest Case? Realistic Case:

5. Steve LloydGridPP13 Durham July 2005 Slide 5 Job Preparation #!/bin/bash # Script to run Athena Analysis on the Grid # Steve Lloyd source $VO_ATLAS_SW_DIR/software/10.0.1/setup.sh source $SITEROOT/dist/10.0.1/Control/AthenaRunTime/*/cmt/setup.sh # Set up the correct directory structure for my files (?) cmt create UserAnalysis UserAnalysis-00-02-02 # Fails to create these for some reason mkdir UserAnalysis/UserAnalysis-00-02-02/UserAnalysis mkdir UserAnalysis/UserAnalysis-00-02-02/src/components mv AnalysisSkeleton.cxx UserAnalysis/UserAnalysis-00-02-02/src mv AnalysisSkeleton.h UserAnalysis/UserAnalysis-00-02-02/UserAnalysis mv UserAnalysis_load.cxx UserAnalysis/UserAnalysis-00-02-02/src/components mv UserAnalysis_entries.cxx UserAnalysis/UserAnalysis-00-02-02/src/components mv requirements Makefile UserAnalysis/UserAnalysis-00-02-02/cmt A Job Script (ATLAS Specific): # Build my code export CMTPATH=`pwd`:${CMTPATH} cd UserAnalysis/UserAnalysis-00-02-02/cmt cmt config source setup.sh gmake # Back to base cd../../.. # Botch required to find PDGTABLE.MeV: #cp $SITEROOT/dist/10.0.1/InstallArea/share/PDGTABLE.MeV. # Run the job: athena.py AnalysisSkeleton_jobOptions.py

6. Steve LloydGridPP13 Durham July 2005 Slide 6 Job Submission [lloyd@lcgui ~/atlas]$ grid-proxy-init Your identity: /C=UK/O=eScience/OU=QueenMaryLondon/L=Physics/CN=steve lloyd Enter GRID pass phrase for this identity: Creating proxy.............................. Done Your proxy is valid until: Thu Mar 17 03:25:06 2005 [lloyd@lcgui ~/atlas]$ Make a copy of your certificate to send out (~ once a day): [lloyd@lcgui ~/atlas]$ edg-job-submit --vo atlas -o jobIDfile athena.jdl Selected Virtual Organisation name (from --vo option): atlas Connecting to host lxn1188.cern.ch, port 7772 Logging to host lxn1188.cern.ch, port 9002 ================================ edg-job-submit Success ==================================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ The edg_jobId has been saved in the following file: /home/lloyd/atlas/jobIDfile ============================================================================================ [lloyd@lcgui ~/atlas]$ Submit the Job: VOJDLFile to hold job IDs

7. Steve LloydGridPP13 Durham July 2005 Slide 7 [lloyd@lcgui ~/atlas]$ edg-job-status -i jobIDfile ------------------------------------------------------------------ 1 : https://lxn1188.cern.ch:9000/tKlZHxqEhuroJUhuhEBtSA 2 : https://lxn1188.cern.ch:9000/IJhkSObaAN5XDKBHPQLQyA 3 : https://lxn1188.cern.ch:9000/BMEOq90zqALvkriHdVeN7A 4 : https://lxn1188.cern.ch:9000/l6wist7SMq6jVePwQjHofg 5 : https://lxn1188.cern.ch:9000/wHl9Yl_puz9hZDMe1OYRyQ 6 : https://lxn1188.cern.ch:9000/PciXGNuAu7vZfcuWiGS3zQ 7 : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ a : all q : quit ------------------------------------------------------------------ Choose one or more edg_jobId(s) in the list - [1-7]all:7 ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-short reached on: Wed Mar 16 17:45:41 2005 ************************************************************* [lloyd@lcgui ~/atlas]$ RAL Valencia CERN Taiwan Job Status Taiwan Find out its status: Ran at:

8. Steve LloydGridPP13 Durham July 2005 Slide 8 Job Retrieval [lloyd@lcgui ~/atlas]$ edg-job-get-output -dir. -i jobIDfile Retrieving files from host: lxn1188.cern.ch ( for https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ have been successfully retrieved and stored in the directory: /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ ********************************************************************************* [lloyd@lcgui ~/atlas]$ ls -lt /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ total 11024 -rw-r--r-- 1 lloyd hep 224 Mar 17 10:47 CLIDDBout.txt -rw-r--r-- 1 lloyd hep 69536 Mar 17 10:47 ntuple.root -rw-r--r-- 1 lloyd hep 5372 Mar 17 10:47 athena.err -rw-r--r-- 1 lloyd hep 11185282 Mar 17 10:47 athena.out Retrieve the Output:

9. Steve LloydGridPP13 Durham July 2005 Slide 9 Reading Data ############# athena.jdl #################... InputData = {"lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root", "lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root"}; DataAccessProtocol = {"gsiftp"};... ################################################ Either specify in the JDL (then resource broker will try and find a CE 'close' to the SE): Logical file names "rome" stands for ATLAS Rome Workshop Production not the location of the file! Or pull the files from inside the job script: #!/bin/bash # Get the data lcg-cp -v --vo atlas lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root file://`pwd`/Zee_18.pool.root lcg-cp -v --vo atlas lfn:rome.004201.recov10.ZeeJimmy._00019.AOD.pool.root file://`pwd`/Zee_19.pool.root...

10. Steve LloydGridPP13 Durham July 2005 Slide 10 Conclusions The Grid is here – it works! Currently difficult to install and maintain the middleware and the experiment’s software It is VERY straightforward to use if all you have is a binary executable It is straightforward to use if someone has installed your experiment's software somewhere There are huge resources available: On 29 June 5789 out of 14079 CPUs free (UK: 1548 out of 2966) Use it!