K.Harrison CERN, 21st November 2002 GANGA: GAUDI/ATHENA AND GRID ALLIANCE - Background and scope - Project organisation - Technology survey - Design - Work towards first release - Conclusions

21st November Background and scope - Ganga is being developed jointly by ATLAS and LHCb to provide an interface for running Gaudi/Athena applications on the Grid  Deal with all phases of a job life cycle: configuration, submission, monitoring, error recovery, output collection, bookkeeping  Hide Grid technicalities from user - First ideas for Ganga put forward in summer 2001 by P.Mato (LHCb) and C.Tull (ATLAS) - Ganga project supported in UK by GridPP  Cambridge and Oxford each have one joint ATLAS/LHCb post funded until December 2004

21st November Ganga: schematic representation Gaudi/Athena application Ganga GUI/CLI JobOptions Algorithms Collective & Resource Grid Services Histograms Monitoring Results

21st November Project organisation - Current main contributors to Ganga development are:  Design team: K.Harrison, W.Lavrijsen, P.Mato, A.Soroko, C.Tull  Project coordinators for GridPP: N.Brook, R.W.L.Jones - Project information is available on the web: - A mailing list has been set up: - Design team nominally has a telephone meeting once every two weeks (next meeting 3rd December) - Presentations of Ganga status and plans given at various other meetings of ATLAS, LHCb and GridPP

21st November Technology survey (1) - Looked at two systems for distributed production:  AliEnAliEn - Developed and used by ALICE - Has its own Grid middleware, but plan is to incorporate EDG tools in the future - Foresee possibility of distributed analysis using PROOF  SLICESLICE - Developed and used by LHCb - Job control currently based on PVSS, but testing EDG middleware as it becomes available

21st November Technology survey (2) - Looked at two web-based user-Grid interfaces:  GENIUSGENIUS - Developed by NICE srl (Italy) and INFN - Implements all EDG middleware commands, and provides an easy-to-use interface to the EDG Testbed  GRAPPAGRAPPA - Developed by GriPhyN/ATLAS group at Indiana University - Prototype allows submission of Athena Atlfast jobs to US-ATLAS Testbed - Plan evolution to a Virtual-Data Browser: act on Virtual Data, with functionality analogous to a web browser

21st November Design considerations -Technology survey has given us insight into how Ganga should be implemented - Ganga should not reproduce what already exists, but should make use of, and complement, work from other projects, including AthASK, DIAL and Grappa in ATLASAthASKDIALGrappa - The design should be modular, and the different modules should be accessed via a thin interface layer implemented using a scripting language, with Python the current choice - Ganga should provide a set of tools that can be accessed from the command line (may be used in scripts), together with a local GUI and/or a web-based GUI that simplifies the use of these tools - Ganga should allow access to local resources as well as to the Grid

21st November Possible Ganga architecture Server EDG UI PYTHON SW BUS XML RPC server XML RPC module GANGA Core Module OS Module Gaudi /Athena GaudiPython PythonROOT PYTHON SW BUS GUI DB Remote user (client) LAN/WAN GRID LRMS Local Job DB Production DB Bookkeeping DB Job Configuration DB

21st November Ganga components - Core Ganga components, each corresponding to a Python base class, will include:  Application – set of packages, parameters, inputs, outputs  ApplicationHandler – deals with an Application  Job – specification of required Application and batch/Grid resource requests  JobHandler – deals with a Job  Each base class has specialisations for different types of Application/Job - Idea is that Ganga will have functionality analogous to a mail system, with jobs having a role similar to mails  Make configuring a Gaudi/Athena job and running it on the Grid as easy as sending a mail

21st November Example specialisations of base classes ApplicationHandler AnotherApplicationHandler AthenaApplicationHandle r AtlfastApplicationHandler Address different types of applications JobHandler AnotherJobHandler GridJobHandler PBSJobHandlerLSFJobHandler LocalJobHandler Address different types of batch systems

21st November Plans for first release - Aim to have a first release of Ganga in January Minimum functionality expected of first release is as follows:  It should run on the CERN Lxplus machines  Implementation choices known to be incompatible with other platforms will be avoided  It should handle a single Gaudi/Athena job, and one job type per experiment  Target Atlfast for ATLAS  Target DaVinci (Gaudi-based analysis) for LHCb  Other types of jobs, and chained jobs, will come later  Job submission will be to the CERN Lxbatch system  Submission to other batch systems and to Grid will be added with subsequent releases (Grid submission by March 2003?)

21st November Current status - First versions of Application, ApplicationHandler, Job, JobHandler base classes have been coded  Working on specialisations - Have implemented an interface to the LHCb JobOptions database  Use xmlrpclib module - Prototype GUI has been created  Probably needs some improvement - Working on environment configuration using cmt from Python  Take AthASK as a starting point, but need to separate experiment-independent parts from workarounds specific to ATLAS

21st November Prototype GUI Embedded Python interpreter Tree of user jobs Job options for selected job

21st November Conclusions - Ganga project has been successfully set up as an ATLAS/LHCb joint venture - A technology survey has been carried out, and has given insight into how Ganga should be implemented - A basic design for Ganga has been formulated  Component based, and will evolve with time - Making good progress towards a first release (January 2003), with job submission to a local batch system - Tentatively aim to have a Ganga release with submission to the Grid around March 2003