1. Ganga 4 Basics - Tutorial Jakub T. Moscicki ARDA/LHCb Ganga Tutorial, September 2006

2. Ganga Tutorial, April 2005 2 Contents What is Ganga  7 things to know about Ganga from K.Harrison Setup and running for the first time Quick introduction to Python Simple Operations in Interactive Text Shell Copying jobs and submitting to other backends Scripting interface Shipping files Splitting

3. Ganga Tutorial, April 2005 3 What is Ganga? AtlasPROD DIAL DIRAC LCG2 gLite localhost LSF submit, kill get output update status store & retrieve job definition prepare, configure Ganga4 Job scripts Gaudi Athena AtlasPROD DIAL DIRAC LCG2 gLite localhost LSF

4. Ganga Tutorial, April 2005 4 Ganga Web Site:  http://cern.ch/ganga http://cern.ch/ganga Ganga Tutorial Bug reports Ganga is based on Python 2.2  Python Website: http://www.python.orgwww.python.org  Excellent Python Tutorial: http://docs.python.org/tut/tut.html Other resources:  IPython page: http://ipython.scipy.org/ Documentation

5. Ganga Tutorial, April 2005 5 Setting up Ganga ATLAS tutorial instructions! Note: we are using a beta version – new release by the end of this month % ganga *** Welcome to Ganga *** Version: Ganga-4-2-0-beta7 Documentation and support: http://cern.ch/ganga Type help() or help('index') for online help. You will be asked few questions. Answer “yes” by pressing ENTER. GRID certificate: type your password or press Control-D several times. To quit press Control-D at the interactive prompt In[1]: ^D Do you really want to exit ([y]/n)? %

6. Ganga Tutorial, April 2005 6 Ganga Text Prompt Ganga is based on python 2.2 and has an enhanced python prompt (IPython):  python programming/scripting var = 5 print var*10  easy access to the shell commands !less ~/.gangarc # personal config file !pwd  history  TAB completion it works on keywords, variables, objects, try: –va  many more features

7. Ganga Tutorial, April 2005 7 x = 2 print x*3 # you may also skip print if x==2: print "yes, x==2" # NOTE INITIAL SPACES! alist = [1,2,3] for y in alist: print y print len(alist) # built-in function len help(len) Python: Statements, Variables

8. Ganga Tutorial, April 2005 8 print range(11) help(range) def square(v): return v*v print square(x) alist = [1,2,3] for i in range(len(alist)): alist[i] = square(alist[i]) blist = [square(x) for x in alist] Python: Functions

9. Ganga Tutorial, April 2005 9 import sys print sys.argv import os print os.environ['HOME'] from os import environ from os import * import math, cmath print math.cos(math.pi) v =(1+1j) print v*v math.log(-1) # exception Python: Modules

10. Ganga Tutorial, April 2005 10 # standard library & user classes import random r = random.Random() r.choice([1,2,3,4]) r.randint(30,40) #built-in classes: for example strings s = "hello at the tutorial" # double quotes s.split() s.count('a') s.upper() dir(s) if 'tut' in s: print 'tut is in ',s print '-'*20 # single quotes Python: Classes and Objects

11. Ganga Tutorial, April 2005 11 # file objects f = file('/etc/password') lines = f.readlines() print len(lines) # writing files f2 = file('testfile','w') print >> f2, “this is a test”,20*'-' f2.close() !cat testfile NOTE: Ganga defines a special class File. Do not confuse File and file! Python: files

12. Ganga Tutorial, April 2005 12 Job Objects and Job Registry [In:] jobs [In:] Job() [In:] jobs [In:] j = jobs[1] [In:] print j.application [In:] print j.backend # job registry: a list of all your jobs [In:] jobs [In:] jobs[1] Ganga Scripting

13. Ganga Tutorial, April 2005 13 Jobs Job ( status = 'new', inputdir = '/afs/cern.ch/user/k/kuba/gangadir/workspace/Local/2/input/', name = '', outputdir = '/afs/cern.ch/user/k/kuba/gangadir/workspace/Local/2/output/', outputsandbox = [], id = 2, application = Executable ( exe = '/bin/hostname', env = {}, args = [] ), inputdata = None, inputsandbox = [ ], backend = Local ( exitcode = None, id = None )

14. Ganga Tutorial, April 2005 14 [In:] j = Job() [In:] j.application.exe = '/bin/hostname' [In:] j.name = "MyTest" [In:] print j [In:] j.submit() # wait until job is completed and look # at the output directory [In:] print j.status [In:] print j.outputdir [In:] !ls $j.outputdir [In:] !cat $j.outputdir/stdout # the ! and $ syntax is IPython – you # cannot do it in normal python # it is useful for interactive work # instead of mouse copy-paste Basic Job Submission

15. Ganga Tutorial, April 2005 15 # once a job is submitted you cannot modify # it. if you want to submit a new job you # should create a new job object [In:] j2 = j.copy() [In:] j2.backend = LSF() [In:] j2.submit() # if you have GRID certificate you can try [In:] j3 = j.copy() [In:] j3.backend = LCG() [In:] j3.submit() # print jobs to see all your jobs More submission

16. Ganga Tutorial, April 2005 16 # see predefined plugins # (only some combinations supported) list_plugins() help('index') # submission in a loop for i in range(5): j = Job() j.application.exe="/bin/echo" j.application.args=["hello ",str(i)]) print j.id, j.inputdir print j.submit() More ideas

17. Ganga Tutorial, April 2005 17 # For frequently used jobs – use templates! t = JobTemplate(j) print templates j = Job(t) # Create a new job from t # Other useful methods j.kill() j.remove() # Removes the output as well! for j in jobs: j.kill() jobs.clean() # Faster! Deletes ALL jobs. More Ideas (2)

18. Ganga Tutorial, April 2005 18 export(j,'myjob') # press ^D to quit ganga cat myjob ganga submit myjob ganga query *restart ganga* load('myjob') Scripting possibilities

19. Ganga Tutorial, April 2005 19 j = Job() j.application.exe=File('spy') j.outputsandbox = ['*.txt'] j.submit() Do not confuse File with file! Shipping files

20. Ganga Tutorial, April 2005 20 Prime number crunching:  cryptography (also Grid certificates)  you break the key by factoring large integers into prime numbers Application setup:  we have 15 tables with 1M primes each the files are downloaded from a web server  we have a “prime_factor.py” script which scans the table and returns the factors it finds: [(p1,n1),(p2,n2),...,(pk,nk)] so that product of pow(pi,ni) = the number Splitting Demo

21. Ganga Tutorial, April 2005 21 Job 1:  find all primes factors of 4643871513989286 among the first million primes j = Job() j.application.exe=File('prime_factor.py') j.application.args =['4643871513989286', 'http://cern.ch/diane/download/primes/primes1.zip'] j.outputsandbox=['*.dat'] j.submit() cat $j.outputdir/*.dat execfile('primes.py') # load helper functions for primes check_prime_job(j) Splitting Demo

22. Ganga Tutorial, April 2005 22 Job 2: find more factors using 5 tables j = j.copy() split_prime_job(j,5) Exercise:  find all primes of: 118020903911855744138963610 Notes:  you cannot not crack real codes this way ;-)  instead of the Web server one could/should use a Grid SE Splitting Demo

23. Ganga Tutorial, April 2005 23 Try: ganga -h to see options Have a look also in the config file: ~/.gangarc Hints and Pitfalls

24. Ganga Tutorial, April 2005 24 Questions? Practical exercises Questions?