GANGA PANDA Dietrich Liko

Motivation Access to OSG resources by GANGA Collaboration with US colleagues Possibly an alternative way of submitting jobs to (some) LCG sites

Does it make sense ? GANGA has been designed as a backend neutral system, PANDA is just another option We have to stay close to pathena capabilities and add our strength – GUI – Job management – GANGA Robot – AMI etc Other aspects like incomplete datasets are a problem for GANGA and for pathena Aim: GANGA has to be an attractive interface also for PANDA users.

How does pathena work It is surprisingly similar to GANGA Small details are different The idea is to run standard pathena jobs via PANDA – We just provide the parameters – The maintenance of the run script stays with PANDA

Some aspects PANDA Build jobs PANDA supports a workflow and use a kind of DAG to compile just once PANDA backend extended to include an optional PANDABuildJob

PANDA Job backend = Panda ( status = None, actualCE = None, site = 'ANALY_BNL_ATLAS_1', id = , buildjob = PandaBuildJob ( status = 'finished', id = ) Single Jobs – Two backend jobids in one Job Subjobs: – PandaBuildJob part of the master job

How to define a Panda Job Definition of the Job is part of the RunTimeHandler Upload tar file to WebDAV server Create a job definition, that will be pickeled and uploaded to the PANDA server

JobSpec jspec = JobSpec() jspec.jobDefinitionID = job.id jspec.jobName = commands.getoutput('uuidgen') jspec.AtlasRelease = 'Atlas-%s' % app.atlas_release jspec.homepackage = 'AnalysisTransforms' jspec.transformation = '%s/runAthena10' % Client.baseURLSUB if job.inputdata: jspec.prodDBlock = job.inputdata.dataset else: jspec.prodDBlock = 'NULL' jspec.destinationDBlock = self.outputdataset jspec.destinationSE = job.backend.site jspec.prodSourceLabel = 'user' jspec.assignedPriority = 1000 jspec.computingSite = job.backend.site

FileSpec finp = FileSpec() finp.lfn = lfn finp.GUID = guid #finp.fsize = #finp.md5sum = finp.dataset = job.inputdata.dataset finp.prodDBlock = job.inputdata.dataset finp.dispatchDBlock = job.inputdata.dataset finp.type = 'input' finp.status = 'ready‘ <- data is already present. One can also wait for a file to arrive jspec.addFile(finp) Similar for output and log file

Issues Some pyton issues Datetime and decimal required – At least python 2.3 outputdata

Outputdata pathena extracts the outputdata from the option file – Many possibilities – ntuple, hist, ESD, AOD, TAG, AANT, THIST, iRoot, EXT, Stream1 – A fixed schema is used to overload the names Ntuple -> dataset name, ntuple name, jobid user.DietrichLiko.ganga.25.PIPPO._00001.root Manual extraction of these parameters turns out to be not an option – Last missing part of the handler Can only run in athena environment

Immediate work Finish the last piece Testing Documentation Support for most pathena options Afterwards: Iteration with Johannes on further support