Status of the Analysis Task Force J.F. Grosse-Oetringhaus ALICE Week, Bologna 21.06.06

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Initiated by Yves Schutz in May The analysis task force’s objective is to assure that a robust and efficient analysis framework is available By stressing the analysis tools on the Grid with PROOF Share the experience with ALICE users Submit requirements to the Offline team Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Members Federico Carminati - PH/AIP <Federico.Carminati@cern.ch> Panos Christakoglou - PH/AIP <Panos.Christakoglou@cern.ch> Rafael Diaz Valdes - PH/UAI <Rafael.Diaz.Valdes@cern.ch> Andrei Gheata - PH/UAT <Andrei.Gheata@cern.ch> Jan Fiete Grosse-Oetringhaus - PH/AIP <Jan.Fiete.Grosse-Oetringhaus@cern.ch> Peter Hristov - PH/AIP <Peter.Hristov@cern.ch> Claus Jorgensen - PH/AIO <Claus.Jorgensen@cern.ch> Mercedes Lopez Noriega - PH/AIP <Mercedes.Lopez.Noriega@cern.ch> Andreas Morsch - PH/AIP <Andreas.Morsch@cern.ch> Tapan Nayak - PH/AIP <tapan.nayak@cern.ch> Yves Schutz - PH/AIP <Yves.Schutz@cern.ch> Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Outlook Summary of 4 talks from the Offline Week Preparation of new Analysis page in the Offline Web (Diaz, R.) File and event level metadata - Queries to the file catalog (Christakoglou, P.) The CERN Analysis Facility: PROOF for Day-1-Physics and Calibration (Grosse-Oetringhaus, J.) A skeleton of analysis framework (Gheata, A.) All found here http://indico.cern.ch/conferenceDisplay.py?confId=a056303#3 More information about analysis on the Grid (A. Peters) http://indico.cern.ch/getFile.py/access?contribId=4&sessionId=4&resId=0&materialId=slides&confId=1148 Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Analysis web page http://aliceinfo.cern.ch/Offline/Activities/Analysis/ Linked on Offline page Information about analysis Dedicated pages to come Analysis framework The CERN Analysis Facility (CAF) Analysis on the Grid Mailing list: alice-project-analysis-task-force@cern.ch Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Run and event level metadata You want to analyse events from/with e.g. ESD files pp collisions the start and stop time of the run between 19/03/2008 and 20/03/2008 a properly reconstructed vertex a Vz value between -1 cm and 1 cm Run (File) level metadata Event level metadata Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Run level metadata Internal note by Markus Oldenburg run comment run type physics, laser, pulser, pedestal, simulation run start time run stop time run stop reason Normal, beam loss, detector failure, … magnetic field setting FullField, ReversedField, ZeroField, HalfField collision system PbPb, pp, pPb, … collision energy trigger class detectors present in this run # of events in this run run sanity for reconstructed events production tag production software library version calibration & alignment setting for simulation Simulation config tag ... Internal note by Markus Oldenburg http://cern.ch/Oldenburg/MetaData/MetaData.doc Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN The event tag system After selecting a set of files to be analyzed, more selection criteria can be applied at the event level in order to analyze only the events that fulfil them, thus reducing the analysis time. Criteria: e.g. Multiplicity range Mean pt range NChargedAbove10GeV range ... The classes that perform this step are AliEventTagCuts and AliTagAnalysis Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN The event tag system (2) For further information on the event tag system (structure, fields and use): http://indico.cern.ch/materialDisplay.py?subContId=2&contribId =s5t2&sessionId=s5&materialId=0&confId=a056302 A note on the architecture and usage of the event tag system is almost ready and will be submitted soon (P. Christakoglou) Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN PROOF Parallel ROOT Facility Interactive parallel analysis on a local cluster PROOF itself is not related to Grid (can be used in the Grid) It is aimed that the use of PROOF is fully transparent The same code can be run locally and in a PROOF system (certain rules have to be followed) Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

PROOF Schema Local PC root $ root root [0] tree->Process(“ana.C”) ESD $ root root [0] tree->Process(“ana.C”) $ root

PROOF Schema Local PC Remote PROOF Cluster root ESD ESD ESD $ root node1 ESD ana.C ESD node2 $ root root [0] tree->Process(“ana.C”) $ root ESD node3 ESD node4

PROOF Schema Local PC Remote PROOF Cluster root proof PROOF master ESD node1 ESD ana.C ESD node2 $ root root [0] tree->Process(“ana.C”) $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) $ root ESD node3 ESD node4

PROOF Schema Local PC Remote PROOF Cluster root proof proof proof PROOF master node1 ESD ana.C ESD proof node2 $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) $ root root [0] tree->Process(“ana.C”) $ root ESD proof node3 ESD proof node4

PROOF Schema Local PC Remote PROOF Cluster root proof proof proof stdout/result root proof PROOF master ana.C node1 ESD ana.C ESD proof node2 $ root $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) $ root root [0] tree->Process(“ana.C”) root [1] gROOT->Proof(“remote”) root [2] chain->Process(“ana.C”) $ root root [0] tree->Process(“ana.C”) ESD proof node3 ESD proof node4

CERN Analysis Facility The CERN Analysis Facility (CAF) will run PROOF for ALICE Prompt analysis of pp data Pilot analysis of PbPb data Calibration & Alignment Available to the whole collaboration but the number of users will be limited for efficiency reasons Design goals 500 CPUs 100 TB of selected data locally available Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

CAF Schema ... Tier-1 data export Tape storage Experiment Disk Buffer Sub set (moderated) CAF computing cluster Proof node local disk Proof node local disk Proof node local disk ... Proof node local disk Proof node local disk Proof node local disk

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Analysis Framework Which constraints on the analysis are there? Big events Maximize the profit from having one event in memory Parallelize an analysis task for subsets of events (PROOF) If possible, serialize current event for several analysis chains Complex analysis chains Split a complex analysis into smaller modules Reuse functionality provided by modules, create a pool of useful/tested/agreed ones Assembly a big analysis from smaller independent pieces Easily locate/debug possible problems Something that every user wants Simple to use and robust Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Prototype A data-driven model controlled by a top-level TSelector mechanism AliAnalysisTask – a basic task depending only on input data AliAnalysisDataSlot – representing a slot where data of a defined type may be connected to a task (A task can define several input/output slots) AliAnalysisDataContainer – a data placeholder Top level containers publishing initial input data for several client tasks Result containers are connected to the output slot of a producer task and also to several client tasks AliAnalysisManager – a selector talking to a list of top-level tasks Task execution serialized event by event (TSelector) The analysis developer will not have to take care of internal mechanisms of TSelector/TTask or implementation details of this framework Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

The analysis tasks and data containers Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Analysis manager Allows definition of the set of tasks and the data containers to be used Provides methods for connecting tasks to containers to create an analysis tree Deriving from TSelector Allows initializing/connecting input data to top level containers via Init(), Begin() methods Executes the top level tasks during Process() Assemblies the output during Terminate() Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN

Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN Summary Analysis Task Force exists to make sure that all needed tools for analysis will be in place. We have to try, test, stress the tools now  Join! Analysis web page for all relevant information Run & Event tags for selection of files on the Grid  Try it when you use events from PDC06! The CERN Analysis Facility is being set up, a PROOF cluster for analysis  Try it when you analyze PDC06 events! A prototype of the analysis framework is nearly ready  Try it! Status of the Analysis Task Force, J.F. Grosse-Oetringhaus, CERN