Production Manager Tools (New Architecture) LHCb Data Management Meeting 30 September 2003 Gennady Kuznetsov

Lessons Learnt © Andrei Tsaregorodtsev DataGRID instability problems persisting : MDS information system failures ; Site miss configuration ; Outbound IP connectivity is not available on all sites : Needed for the LHCb software installation on per job basis ; Needed for jobs exchanging messages with production services . Data transfer : bbftp file transport replaced by the replica manager tools ; Next run of tests will hopefully be more successful : better understanding of the system ; Use other schema for job delivery to a worker node . © Andrei Tsaregorodtsev Presenter Name: Gennady Kuznetsov Facility Name: PPD

Other Schema Other schema Flexible component architecture Full forward and backward compatibility of jobs. No needs for installation of the control or monitoring software on the node or site Workflows, Steps, Modules stored on server or in Production Manager Library as XML file. Job is Python code generated from XML during submission. Monitoring code got embedded into Jobs automatically Agent (for development) Agents can be run on any computer local or remote Single agent can control one or many sites simultaneously Agent implements Andrey’s pool job submission schema. Presenter Name: Facility Name:

Components Step Workflow Production Module Job Step We add Module as additional component of the architecture Module Step Levels of usage: Module – Programmer Step – Production manager Workflow – User Step Workflow Job Production This module structure allow as to construct any algorithm as combination of modules. Each step generates job as a Python program. Presenter Name: Facility Name: Presenter Name: Gennady Kuznetsov Facility Name: PPD

Python code of single module. Can be many classes. Module Editor Python code of single module. Can be many classes. Module Name Stored as XML file Description Module variables. Presenter Name: Facility Name:

Step Editor Stored as XML file, where all modules are embedded Graphical representation of Module relationship (not ready) Stored as XML file, where all modules are embedded Step Name Description Variables of currently selected instance Definitions of Modules Instances of Modules Selected instance Presenter Name: Facility Name:

Workflow Editor Not ready yet. It will links all jobs in a workflow instance. The simplest workflow is a single step. It will combine workflow and “Vector” and generate as many instances of workflow as many members in a vector. For example it can be list of numbers (job number or name) or it can be a list of files for reconstructions or analysis. Presenter Name: Facility Name:

Architecture Central Services DIRAC Personal Desktop Node IDL+Python scoreboard Architecture Production Manager Agent on the Node replaced by automatically generated Python program No any software required except PYTHON Central Services DIRAC Personal Desktop Node Node Grid Front End IDL+Python via SSH Node DIRAC Personal Agent Node Node Batch Front End shell+Python via SSH Node Presenter Name: Gennady Kuznetsov Facility Name: PPD

DIRAC Personal Desktop Language C++ with QT Provides tools to create Module Library for the experiment (for Programmer). Provides tools to construct Steps and Workflows from Module Library. (for Production Manager) Saves all objects as XML files. Generates batch of Python jobs for the submission into GRID or Batch System. DIRAC Personal Desktop Module Editor Step Editor Workflow Editor Production Manager Monitoring Manager Code Generator Presenter Name: Gennady Kuznetsov Facility Name: PPD

Personal Agent Language Java. Represent User or Production Manager in the network on a permanent basis (jobs does not crash without agent). Provides means to submit and monitor jobs to any number of Batch or GRID front ends via SSH of local calls. Provides local disk space for data retrieval. Allow job dependency. DIRAC Personal Agent Job Dependency Service Job Submission Service Internal Monitoring Srv. Data Retrieval Service Local Disk Job pool New Not ready Ready Submitted Running Output ready Done Job states: Presenter Name: Gennady Kuznetsov Facility Name: PPD

Central Services (CERN) Job Pool Prepackaged Applications archive Central Services Production Service Monitoring Service Bookkeeping Service Storage Service Application distr. ser. Data base with web front-end Oracle at CERN CASTOR or Replica Manager Presenter Name: Gennady Kuznetsov Facility Name: PPD