Analysis framework - status

Slides:



Advertisements
Similar presentations
5/2/  Online  Offline 5/2/20072  Online  Raw data : within the DAQ monitoring framework  Reconstructed data : with the HLT monitoring framework.
Advertisements

The LEGO Train Framework
Trains status&tests M. Gheata. Train types run centrally FILTERING – Default trains for p-p and Pb-Pb, data and MC (4) Special configuration need to be.
NET+OS Bootloader Overview Requirements How to Build How to Customize Changes to Applications Configuration Settings.
Chiara Zampolli in collaboration with C. Cheshkov, A. Dainese ALICE Offline Week Feb 2009C. Zampolli 1.
The ALICE Analysis Framework A.Gheata for ALICE Offline Collaboration 11/3/2008 ACAT'081A.Gheata – ALICE Analysis Framework.
Experience with analysis of TPC data Marian Ivanov.
Costin Grigoras ALICE Offline. In the period of steady LHC operation, The Grid usage is constant and high and, as foreseen, is used for massive RAW and.
Analysis infrastructure/framework A collection of questions, observations, suggestions concerning analysis infrastructure and framework Compiled by Marco.
V4-20-Release P. Hristov 08/08/ Changes: v4-20-Rev-38 #85151 Memory leak in T0 DQM agent. From rev #85276 AliGRPPreprocessor.cxx: Port to.
Offline report – 7TeV data taking period (Mar.30 – Apr.6) ALICE SRC April 6, 2010.
Infrastructure for QA and automatic trending F. Bellini, M. Germain ALICE Offline Week, 19 th November 2014.
PWG3 Analysis: status, experience, requests Andrea Dainese on behalf of PWG3 ALICE Offline Week, CERN, Andrea Dainese 1.
Andrei Gheata, Mihaela Gheata, Andreas Morsch ALICE offline week, 5-9 July 2010.
Analysis trains – Status & experience from operation Mihaela Gheata.
5/2/  Online  Offline 5/2/20072  Online  Raw data : within the DAQ monitoring framework  Reconstructed data : with the HLT monitoring framework.
Planning and status of the Full Dress Rehearsal Latchezar Betev ALICE Offline week, Oct.12, 2007.
ALICE analysis framework References for Analysis Tools used to the ALICE simulated data.
Debugging of # P. Hristov 04/03/2013. Introduction Difficult problem – The behavior is “random” and depends on the “history” – The debugger doesn’t.
ALICE Operations short summary ALICE Offline week June 15, 2012.
PROOF and ALICE Analysis Facilities Arsen Hayrapetyan Yerevan Physics Institute, CERN.
1 Checks on SDD Data Piergiorgio Cerello, Francesco Prino, Melinda Siciliano.
Alberto Colla - CERN ALICE off-line week 1 Alberto Colla ALICE off-line week Cern, May 31, 2005 Table of contents: ● Summary of requirements ● Description.
Computing for Alice at GSI (Proposal) (Marian Ivanov)
A. Gheata, ALICE offline week March 09 Status of the analysis framework.
AliRoot survey: Analysis P.Hristov 11/06/2013. Are you involved in analysis activities?(85.1% Yes, 14.9% No) 2 Involved since 4.5±2.4 years Dedicated.
Gustavo Conesa ALICE offline week Gamma and Jet correlations analysis framework Short description, Status, HOW TO use and TO DO list 1/9.
1 Offline Week, October 28 th 2009 PWG3-Muon: Analysis Status From ESD to AOD:  inclusion of MC branch in the AOD  standard AOD creation for PDC09 files.
Data processing Offline review Feb 2, Productions, tools and results Three basic types of processing RAW MC Trains/AODs I will go through these.
M. Gheata ALICE offline week, October Current train wagons GroupAOD producersWork on ESD input Work on AOD input PWG PWG31 (vertexing)2 (+
PWG3 analysis (barrel)
Analysis experience at GSIAF Marian Ivanov. HEP data analysis ● Typical HEP data analysis (physic analysis, calibration, alignment) and any statistical.
Analysis train M.Gheata ALICE offline week, 17 March '09.
M. Gheata ALICE offline week, 24 June  A new analysis train macro was designed for production  /ANALYSIS/macros/AnalysisTrainNew.C /ANALYSIS/macros/AnalysisTrainNew.C.
V5-01-Release & v5-02-Release Peter Hristov 20/02/2012.
Offline Weekly Meeting, 24th April 2009 C. Cheshkov & C. Zampolli.
AliRoot Classes for access to Calibration and Alignment objects Magali Gruwé CERN PH/AIP ALICE Offline Meeting February 17 th 2005 To be presented to detector.
Analysis efficiency Andrei Gheata ALICE offline week 03 October 2012.
V4-20-Release P. Hristov 21/02/ Changes: v4-20-Rev-14 #78385 Please port AliTPCPreprocessor.cxx Rev to Release #78344 Request to port changes.
Analysis framework plans A.Gheata Offline week 13 July 2011.
Some topics for discussion 31/03/2016 P. Hristov 1.
AliRoot survey: Calibration P.Hristov 11/06/2013.
V4-19-Release P. Hristov 11/10/ Not ready (27/09/10) #73618 Problems in the minimum bias PbPb MC production at 2.76 TeV #72642 EMCAL: Modifications.
ANALYSIS TRAIN ON THE GRID Mihaela Gheata. AOD production train ◦ AOD production will be organized in a ‘train’ of tasks ◦ To maximize efficiency of full.
HYDRA Framework. Setup of software environment Setup of software environment Using the documentation Using the documentation How to compile a program.
Monthly video-conference, 18/12/2003 P.Hristov1 Preparation for physics data challenge'04 P.Hristov Alice monthly off-line video-conference December 18,
CALIBRATION: PREPARATION FOR RUN2 ALICE Offline Week, 25 June 2014 C. Zampolli.
V4-18-Release P. Hristov 21/06/2010.
Jan Fiete Grosse-Oetringhaus
Data Formats and Impact on Federated Access
ALICE experience with ROOT I/O
Analysis trains – Status & experience from operation
PWG2 Analysis status Adam Kisiel, CERN for the PWG2 group.
Analysis tools in ALICE
Developments of the PWG3 muon analysis code
DPG Activities DPG Session, ALICE Monthly Mini Week
Status of the Analysis Task Force
Work report Xianghu Zhao Nov 11, 2014.
ALICE analysis preservation
News on the CDB Framework
v4-19-Release & v4-20-Release
v4-18-Release: really the last revision!
New calibration strategy – follow-up from the production review
Experience in ALICE – Analysis Framework and Train
Analysis Trains - Reloaded
AliRoot status and PDC’04
Framework for the acceptance and efficiency corrections
2 Getting Started.
Use Of GAUDI framework in Online Environment
Offline framework for conditions data
Presentation transcript:

Analysis framework - status Andrei Gheata, ALICE offline week, Analysis framework - status

Analysis output files Outputs validation improved Triggered by the ‘unzip error’ . Output files that can be just partially recovered are not good for AOD analysis. The analysis manager checks if the files were can be properly opened. It writes a 0-length file that can be used for validation of jobs Folders in the output files mgr->CreateContainer(“comparison”, TList::Class(), AliAnalysisManager::kOutputContainer, “myAnalysisHistos.root#comparison”) Will make the folder “comparison” in the output file and put results there Available soon Added support for creating both standard AOD and several deltaAOD's in the same ESD-based analysis. Producing a deltaAOD using AliAODHandler::SetNonStandardAOD() in an analysis on input AOD is still possible. To use the new feature: - use a standard AOD handler (no call to SetNonStandardAOD) - in the AliAnalysisTaskSE-derived task, call AddAODBranch(cname, addobj, fname) where the newly added fname is the file name where the branch(es) is(are) to be written. Added AliAnalysisTaskSE::IsStandardAOD() to check from the user code if the AOD handler is configured for standard AOD or not. Check first via AODEvent() if there is any AOD handler at all. One has just to supply a valid path to an Alien location to the method: AliAnalysisManager->SetSpecialOutputLocation(path); where: path = "alien:///alice/cern.ch/user/.../myOutput/" has to be prefixed by alien:// and must correspond to the user token that started the proof job. The common output files (AOD) will be copied to this location as AliAOD_n.root from each slave (n) while the rest of the outputs (histograms) will be first merged then copied to the same location.

Additions to the AliEn plugin Names of files generated by the alien plugin can (and should) be changed to avoid clashes Output directory can be an absolute alien path New method added to support raw run numbers (like 0000876543) AddRunNumber(const char *number) New method allowing to customize the command executed to run the job Default: ‘root –b –q’ ‘aliroot –b –q’ needed by PWG1 QA train

Bottlenecks in file access, memory problems We noticed that most failures in grid analysis for IO intensive jobs is due to timeouts No simple tools to make some systematic on the conditions that trigger this behavior Started to make some based on xrootd environment variables for timeouts Memory usage plots per task will be produced on demand in “test” mode So that everybody can debug leaks

Analysis tender Agreed proposal discussed last offline week A set of detector specific algorithms attached to a tender wagon Read-only access to OCDB, to get the ‘latest and best’ set of calibration data and correct the current ESD event in memory with updated info (TPC and TRD PID, primary vertex, …) Do NOT create a new ESD, but just make the updated info directly available to the analysis cars after the tender Probably produce a new set of AOD’s to avoid running the tender too often

Workflow Only the tender accesses OCDB ESD’s (first pass) ESD event Header ESD event Header ANALYSIS MANAGER OCDB Calibration info Calibration corrections Vertex TPC TRD ... PWG2 wagons PWG3 wagons PWG4 wagons TENDER ESD filters AOD ∆AODs Only the tender accesses OCDB Info used by detector expert algorithm updating the current ESD event in memory The other wagons normally running (as without tender) If writing AOD’s, they will automatically contain the corrections

Tender implementation Basic. Two classes. AliAnalysisTender: public AliAnalysisTask Has pointers to: current ESD event, CDB manager, CDB key (!), list of supplies (!) ConnectInputData(): Unlocks CDB manager, set default storage and run number, allow supplies (!) to set the specific storages and read from CDB, lock CDB Exec(): Call ProcessEvent() for all tender supplies AliAnalysisTenderSupply : public TNamed: the detector algorithm to run in the tender 2 methodes to implement: Init() to initialize CDB access and to read relevant CDB info ProcessEvent() to run the algorithm and patch the ESD event in memory