Seven things you should know about Ganga K. Harrison (University of Cambridge) Distributed Analysis Tutorial ATLAS Software & Computing Workshop, CERN, September

11 September 20062/8 1) Ganga is a user tool for job definition and management Allows simple switching between testing on a local batch system and large-scale data processing on distributed resources (Grid) Developed in the context of ATLAS and LHCb –For ATLAS, have built-in support for applications based on Athena framework, for JobTransforms, and for DQ2 data-management system Component architecture readily allows extension Implemented in Python Strong development team, meaning strong user support –F.Brochu (Cambridge), U.Egede (Imperial), J.Elmsheuser (München), K.Harrison (Cambridge), H.C.Lee (ASCC), D.Liko (CERN), A.Maier (CERN), J.T.Moscicki (CERN), A.Muraru (Bucharest), V.Romanovsky (IHEP), A.Soroko (Oxford), C.L.Tan (Birmingham) Contributions past and present from many others

11 September 20063/8 2) Ganga is based on a simple, but flexible, job abstraction A job is constructed from a set of building blocks, not all required for every job Merger Application Backend Input Dataset Output Dataset Splitter Data read by application Data written by application Rule for dividing into subjobs Rule for combining outputs Where to run What to run Job

11 September 20064/8 3) Ganga provides a framework for handling different types of Application, Backend, Dataset, Splitter and Merger, implemented as plugin classes Athena GangaObject IApplication IBackendIDataset ISplitterIMerger LCG -CE -requirements -middleware -id -status -reason -actualCE -exitcode -atlas_release -max_events -options -option_file -output -user_setup -user_area User System Plugin Interfaces Example plugins and schemas Each plugin class has its own schema

11 September 20065/8 4) Ganga simplifies running of ATLAS (and LHCb) applications on a variety of Grid and non-Grid backends PBSOSGNorduGridLocalLSFPANDA US-ATLAS WMS LHCb WMS Executable Athena (Simulation/Digitisation/ Reconstruction/Analysis) AthenaMC (Production) Gauss/Boole/Brunel/DaVinci (Simulation/Digitisation/ Reconstruction/Analysis) LHCbExperiment neutralATLAS Implemented Work in progress Running of a particular Application on a given Backend is enabled by implementing an appropriate adapter component or Runtime Handler

11 September 20066/8 5) Ganga provides a selection of interfaces, offering users choice in the way they work Ganga’s Command Line Interface in Python (CLIP) provides interactive job definition and submission from an enhanced Python shell (IPython) –Especially good for trying things out, and seeing how the system works Ganga scripts, which may contain any Python/IPython or CLIP commands, allow automation of repetitive tasks Scripts included in distribution enable kind of approach traditionally used when submitting jobs to a local batch system ganga submit --backend=LCG athenaJob.py Ganga’s Graphical User Interface (GUI) allows job management based on mouse selections and field completion –Lots of configuration possibilities

11 September 20067/8 6) Ganga helps users keep track of what they’ve done Job progress is monitored, with status changes reported automatically Retrieval and/or storage of job outputs is dealt with Job-related files and information are kept organised in the Ganga work directory (~/gangadir) gangadir repository input Local templates output workspace Remote gui jobs6667

11 September 20068/8 7) Ganga is a work in progress Key Ganga functionality is demonstrated in this tutorial Documentation can be found in User Guides section of Ganga web site: Tutorial participants strongly encouraged to try using Ganga themselves, and to get in touch if they run into problems or have suggestions for improvement –Send to any of the Ganga developers –Send to Must sign up to list to be able to use it –Submit a report via Ganga’s bug-submission page in Savannah: Should either login to Savannah first, or give address  Development ongoing, and user feedback essential to ensure Ganga has all functionality required for distributed analysis in ATLAS