M. Gheata ALICE offline week, October 2009

Current train wagons GroupAOD producersWork on ESD input Work on AOD input PWG PWG31 (vertexing)2 (+ others to come from muon) 0 (7 to come D2H vertexing) PWG43 (jetan+2partcor) 43 (same) Total4+2 (filters)2414 (+7 to come) Grand total: 35 to 40 tasks to be included ( 28 already in) ! The central analysis train is meant to optimize running all these modules on existing and future data. It is maintained and run centrally but all analysis users are encouraged to participate, comply to the specific requirements and operation conditions.

Train monitoring page Data accessibility problems were the main reason preventing 100% success rate

Alitrain – “training” phase New analysis production user called alitrain Analysis files are produced by the macro ANALYSIS/macros/AnalysisTrainNew.C and submitted centrally by LPM manager So far only standard AOD’s produced for the most recent MC productions (~70 million AOD events available!) To be able to exercise AOD-based analysis To learn how to optimize analysis jobs in grid A lot of new wagons were included and tested

Alitrain – production phase Train ready for production now – will run weekly (unless some real blocker appears) All wagons comply with the framework and with the new requirements for organized mode. The transition process to reach this point was long, but necessary Scaled with the number of developing tasks To run the train efficiently, we need to establish and agree on a certain operation mode To do this, everyone should follow some well-defined rules Some of these already announced on the analysis mailing list, others to be discussed this week

Reminder: condition for including new wagons Any new wagon request should follow the conditions described in the analysis web pageweb page Providing an AddTask macro, its calling sequence, libraries to be loaded… The request must specify in addition the supported input types and type of outputs We will test the wagon in all modes Including memory and CPU profiling from now on – first results to be discussed on Friday Very small chances that a task included on Friday will be able to run on the tag next week !

Train operation: composition Input data: ESD or AOD. MC available in both. Produced data: AOD’s, deltaAOD’s, histograms Not all combinations can (or make sense) to run in the same train Scheduling of future trains and their types and composition should be decided on regular basis (analysis meetings)

Train operation: conditions The train cannot run weekly with the last minute changes of 30+ tasks There are delays related to trunk compilation, build system We need to do some tests before launching on tens of millions events A new train will run weekly (typically Wednesday) New branch on Friday – last commit allowed until 14:00 Tag done on Monday or Tuesday Train tested locally before the new version is deployed, and on a small scale after New requests for next train and new tasks will be considered during Wednesday meeting. Users are responsible for testing their changes before Friday. Not working wagons will be disabled from the train and a Savannah bug will be posted. Please test new code and notify us in case this affects the way your wagon should be run in the train macro !

Train operation: future plans Memory and CPU profiles for each wagon using sysinfo via the analysis manager (see Friday) Reduce verbosity of wagons and correlate with verbosity of analysis manager Some outputs quite big now Introduce a policy in output file names Histos.root not acceptable anymore Extract some systematics based on train runs Run performance, failure conditions, try to spot the weak points in the whole chain Grid jdl and macro to assemble the train in a grid job rather than on a local machine

Conclusions Analysis train entering production mode at full speed Train operations more strict now Operation mode to be respected by everyone Actions are taken to spot and address remaining weak points We hope a large participation in debugging: users, developers, grid experts