Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, 24-26 Nov. 2006 - p. 1 The CMS Computing Software and Analysis Challenge 2006 Department.

Slides:



Advertisements
Similar presentations
Nicola De Filippis Integration meeting, 28 th September p. 1 MC production for CSA06 Department of Physics and INFN Bari N. De Filippis S. My and.
Advertisements

1 14 Feb 2007 CMS Italia – Napoli A. Fanfani Univ. Bologna A. Fanfani University of Bologna MC Production System & DM catalogue.
11 Application of CSF4 in Avian Flu Grid: Meta-scheduler CSF4. Lab of Grid Computing and Network Security Jilin University, Changchun, China Hongliang.
1 ALICE Grid Status David Evans The University of Birmingham GridPP 16 th Collaboration Meeting QMUL June 2006.
Your university or experiment logo here BaBar Status Report Chris Brew GridPP16 QMUL 28/06/2006.
B A B AR and the GRID Roger Barlow for Fergus Wilson GridPP 13 5 th July 2005, Durham.
Cosners House Forum on LHC startup April 07 1 Minimum bias studies etc…. Craig Buttar University of Glasgow.
Chapter 5 Input/Output 5.1 Principles of I/O hardware
31/03/00 CMS(UK)Glenn Patrick What is the CMS(UK) Data Model? Assume that CMS software is available at every UK institute connected by some infrastructure.
Experimental Particle Physics PHYS6011 Joel Goldstein, RAL 1.Introduction & Accelerators 2.Particle Interactions and Detectors (2) 3.Collider Experiments.
1 Click here to End Presentation Software: Installation and Updates Internet Download CD release NACIS Updates.
CS 6143 COMPUTER ARCHITECTURE II SPRING 2014 ACM Principles and Practice of Parallel Programming, PPoPP, 2006 Panel Presentations Parallel Processing is.
Chapter 13 Web Page Design Studio
Quark Compositeness Search with γ +Jet Final State at the LHC Satyaki Bhattacharya, Sushil S. Chauhan, Brajesh Choudhary, Debajyoti Choudhury Department.
CMS Alignment and Calibration Yuriy Pakhotin on behalf of CMS Collaboration.
Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.
1 CMS Tracker Alignment and Implications for Physics Performance Nhan Tran Johns Hopkins University CMS Collaboration SPLIT
WLCG/8 July 2010/MCSawley WAN area transfers and networking: a predictive model for CMS WLCG Workshop, July 7-9, 2010 Marie-Christine Sawley, ETH Zurich.
Stefano Belforte INFN Trieste 1 CMS SC4 etc. July 5, 2006 CMS Service Challenge 4 and beyond.
L3 Filtering: status and plans D  Computing Review Meeting: 9 th May 2002 Terry Wyatt, on behalf of the L3 Algorithms group. For more details of current.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
Zhiling Chen (IPP-ETHZ) Doktorandenseminar June, 4 th, 2009.
Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.
Claudio Grandi INFN Bologna CMS Operations Update Ian Fisk, Claudio Grandi 1.
Preparation of KIPT (Kharkov) computing facilities for CMS data analysis L. Levchuk Kharkov Institute of Physics and Technology (KIPT), Kharkov, Ukraine.
Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.
Operational Experience with CMS Tier-2 Sites I. González Caballero (Universidad de Oviedo) for the CMS Collaboration.
2004 Xmas MeetingSarah Allwood WW Scattering at ATLAS.
Nicola De Filippis CMS Italia, Napoli, Feb p. 1 Produzioni MC ai Tiers CMS nel 2007: prospettive CMS-wide e contributo italiano Università,
1 M. Paganoni, HCP2007 Computing tools and analysis architectures: the CMS computing strategy M. Paganoni HCP2007 La Biodola, 23/5/2007.
Status of the LHCb MC production system Andrei Tsaregorodtsev, CPPM, Marseille DataGRID France workshop, Marseille, 24 September 2002.
November SC06 Tampa F.Fanzago CRAB a user-friendly tool for CMS distributed analysis Federica Fanzago INFN-PADOVA for CRAB team.
CHEP06, Mumbai-India, Feb 2006V. Daniel Elvira 1 The CMS Simulation Validation Suite V. Daniel Elvira (Fermilab) for the CMS Collaboration.
EGEE is a project funded by the European Union under contract IST HEP Use Cases for Grid Computing J. A. Templon Undecided (NIKHEF) Grid Tutorial,
ATLAS Data Challenges US ATLAS Physics & Computing ANL October 30th 2001 Gilbert Poulard CERN EP-ATC.
Stefano Belforte INFN Trieste 1 Middleware February 14, 2007 Resource Broker, gLite etc. CMS vs. middleware.
Stefano Belforte INFN Trieste 1 CMS Simulation at Tier2 June 12, 2006 Simulation (Monte Carlo) Production for CMS Stefano Belforte WLCG-Tier2 workshop.
25 sep Reconstruction and Identification of Hadronic Decays of Taus using the CMS Detector Michele Pioppi – CERN On behalf.
The CMS Simulation Software Julia Yarba, Fermilab on behalf of CMS Collaboration 22 m long, 15 m in diameter Over a million geometrical volumes Many complex.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
Post-DC2/Rome Production Kaushik De, Mark Sosebee University of Texas at Arlington U.S. Grid Phone Meeting July 13, 2005.
The CMS CERN Analysis Facility (CAF) Peter Kreuzer (RWTH Aachen) - Stephen Gowdy (CERN), Jose Afonso Sanches (UERJ Brazil) on behalf.
Status of RPC trigger analysis and Muon Trigger efficiencies for W-> μν study By Archana Sharma, Suman B. Beri Panjab University Chandigarh India-CMS Meeting.
CMS Week Sept '07Leonard Apanasevich (UIC) Pedrame Bargassa (Rice) 1 Physics Priorities for Trigger Development Leonard Apanasevich (UIC) Pedrame Bargessa.
Claudio Grandi INFN Bologna CERN - WLCG Workshop 13 November 2008 CMS - Plan for shutdown and data-taking preparation Claudio Grandi Outline: Global Runs.
ATLAS Trigger Development
Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.
The CMS Computing System: getting ready for Data Analysis Matthias Kasemann CERN/DESY.
LHCb report to LHCC and C-RSG Philippe Charpentier CERN on behalf of LHCb.
Daniele Spiga PerugiaCMS Italia 14 Feb ’07 Napoli1 CRAB status and next evolution Daniele Spiga University & INFN Perugia On behalf of CRAB Team.
1 Andrea Sciabà CERN The commissioning of CMS computing centres in the WLCG Grid ACAT November 2008 Erice, Italy Andrea Sciabà S. Belforte, A.
04/09/2007 Reconstruction of LHC events at CMS Tommaso Boccali - INFN Pisa Shahram Rahatlou - Roma University Lucia Silvestris - INFN Bari On behalf of.
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
HI July Exercise and Muon DQM preparation Mihee Jo Mihee Jo / Lab meeting.
GDB, 07/06/06 CMS Centre Roles à CMS data hierarchy: n RAW (1.5/2MB) -> RECO (0.2/0.4MB) -> AOD (50kB)-> TAG à Tier-0 role: n First-pass.
1 Reconstruction tasks R.Shahoyan, 25/06/ Including TRD into track fit (JIRA PWGPP-1))  JIRA PWGPP-2: Code is in the release, need to switch setting.
ATLAS Distributed Analysis DISTRIBUTED ANALYSIS JOBS WITH THE ATLAS PRODUCTION SYSTEM S. González D. Liko
Monitoring the Readiness and Utilization of the Distributed CMS Computing Facilities XVIII International Conference on Computing in High Energy and Nuclear.
1 June 11/Ian Fisk CMS Model and the Network Ian Fisk.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
Muon Reconstruction and Vertex Constraint Giovanni Abbiendi Bologna CMS meeting, 20 November 2007.
David Lange Lawrence Livermore National Laboratory
Univ. Milano-Bicocca e INFN
ALICE Physics Data Challenge 3
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
Philippe Charpentier CERN – LHCb On behalf of the LHCb Computing Group
Workshop Summary Dirk Duellmann.
US ATLAS Physics & Computing
N. De Filippis - LLR-Ecole Polytechnique
The LHCb Computing Data Challenge DC06
Presentation transcript:

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 1 The CMS Computing Software and Analysis Challenge 2006 Department of Physics and INFN Bari N. De Filippis On behalf of the CMS collaboration

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 2 Contributors Tommaso Boccali Andrea Sciaba' Luca Lista Sergio Fantinel Daniele Bonacorsi Marco Paganoni Giacinto Donvito Alessandra Fanfani Giorgio Maggi Stefano Belforte Giuseppe Bagliesi Francesco Safai Tehrani Giuseppe Codispoti Salvatore My Marcello Abbrescia Antonio Pierro Paolo Capiluppi William Bacchi Livio Fanò Carlos Kavka Ugo Gasparini Paolo.Meridiani Frederic Ronga Federico CALZOLARI Guido Cuscela Massimo Biasotto Federica Fanzago Maria.Damato Marco Corvo HAJDU Csaba Simon Metson, StuartWakefield Mona Aggarwal Olivier van der Aa Giuseppe Mazza Alex Martin Dave Newbold David Colling Paolo.Bartalini Filippo Ambroglini Giuseppe.Cerati Patrizia Azzi Ezio Torrassa Martino.Margoni Laura Edera

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 3 What was CSA06? A 50 million event exercise to test the workflow and dataflow as defined in the CMS computing model A test at 25% of the capacity needed in 2008 Main components: Preparation of large MC simulated datasets (some with HLT-tags) Prompt reconstruction at Tier-0: –Reconstruction at 40 Hz (over 150 Hz) using CMSSW –Application of calibration constants from offline DB –Generation of Reco, AOD, and AlCaReco datasets –Splitting of an HLT-tagged sample into 10 streams Distribution of all AOD & some FEVT to all participating Tier-1s Calibration jobs on AlCaReco datasets at some Tier-1s and CAF Re-reconstruction performed at Tier-1s Skim jobs at some Tier-1s with data propagated to Tier-2s Physics jobs at Tier-2s and Tier-1s on AOD and Reco Italian contribution

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 4 – June 1 June 14: First Version of Detector and Physics reconstruction SW for CSA06 – June 1: Computing systems ready for Service Challenge SC4 – June 15: physics simulation validation complete – July 1: start MC production – Aug.15: Calibration, alignment, HLT (and first version L1 simulation), reconstruction, and analysis tools ready – Aug.30: 50 Mevt produced, 5M with HLT pre-processing – Sep. 1: Computing systems ready for CSA – Sep 15: Start CSA06 – Oct 1: Smooth operation for CSA06 – Oct 30: End smooth operation for CSA06 – Nov 15: Finish CSA06 Official Timeline

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 5 Success metrics Most of performance metrics of the CSA06 are: Number of participating Tier-1 - Goal: 7 - Threshold: 5; Number of participating Tier-2 - Goal: 20 - Threshold 15; Weeks of running at sustained rate - Goal: 4 - Threshold: 2; Tier-0 Efficiency - Goal: 80 % - Threshold: 30 %, measured as unattended uptime fraction over 2 best weeks of the running period; Running grid jobs (Tier-1+Tier-2) per day (2h jobs typ.) - Goal: 50K - Threshold: 30K; Grid job efficiency - Goal: 90 % - Threshold: 70 %; Data serving capability at each participating site from the disk storage to CPU: Goal 1MB/s/execution slot - Threshold : 400 MB/s (Tier-1) or 100 MB/sec (Tier-2) Data transfer Tier-0 to Tier-1 to tape - Individual goals (threshold at 50 % of goal); for CNAF it was: 25 MB/s; Data transfer Tier-1 to Tier-2 - Goal: 20 MB/s into each Tier-2 - Threshold: 5 MB/s; Overall "success" is to have 50 % of participant at or above goal and 90 % above threshold.

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 6 Tier-0 (CERN): –1.4M SI2K (~1400 CPUs at CERN) –240 TB Tier-1 (7 sites): –2500 CPUs in total –70 TB disk + tape as minimum to participate Tier-2 (25 sites): –2400 CPUs in total –Average 10 TB disk at participating Tier-2 Computing resources

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 7 CSA06 MC production

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 8 ProdAgent tool used to automatise the production: consists of many agents running in parallel: JobCreator, JobSubmitter, JobTracking, MergeSensor…. ouput files are registered in Data bookeeping service (DBS); blocks of files are registered in Data Location System (DLS) which takes care of mapping of file blocks and storage elements where they exist Files are merged for optimum size before transfer to CERN CMS software (CMSSW) installed via grid tools or directly by site admins in remote sites. A local catalogue used to map LFNs to local PFNs via a set of rules Storage technologies deployed: CASTOR, dCache, DPM MC production software and tools

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 9 MC pre-production 4 production teams active: 1 for OSG with contact person: -- Ajit Mohapatra – Wisconsin (taking care of 7 OSG CMS Tier2) 3 for LCG: -- LCG(1) with contact person Jose Hernandez – Madrid (Spain, France, Belgium, CERN) -- LCG(2) with contact person Carsten Hof – Aachen (Germany, Estonia, Taiwan, Russia, Switzerland, FNAL) -- LCG(3) with contact person Nicola De Filippis – Bari (Italy, UK, Hungary) Large partecipation of CMS T1s and T2s involved

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 10 Monitoring of MinBias (1) Maximum rate per day: 1.15 M

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 11 Monitoring of MinBias (2) T1 -CNAF Most of the failures at CNAF were related to stageout and stagein problems with CASTOR2 Bari PisaLNL

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 12 Dataset statistics 1. Minimum bias (40M) 2. Z µµ (2M) 3. T-Tbar (6M) –All decays 4. W e (4M) –events selected in narrow range to illuminate 2 SMs 5. Electroweak soup (5M) –W l nu + Drell-Yan (m>15 GeV) + WW +H WW 6. HLT soup (5M): 10 effective MC HLT triggers (no taus pass) –W (leptons) + Drell-Yan (leptons) + t-tbar (all modes) + dijets 7. Jet calibration soup (1M) –dijet + Z+jet, various pt-hat ranges 8. Soft Muon Soup (2M) –Inclusive muons in minbias + J/Psi production 9. Exotics Soup (1M) –LM1 SUSY, Z (700 GeV), and excited quark (2000 GeV) [all decays] 12 M of events produced by the LCG(3) team Total: ~ 66 M events Total FEVT: O(150) TB

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 13 Efficiency and problems Efficiency: Overall efficiency: 88% –Probability for a job to end successfully once it is submitted Grid efficiency: 95% –Aborted jobs: jobs not submitted because requirements not met (merge jobs) or jobs once submitted fail due to Grid infrastructure reason Problems: stage out was the main cause of job failures. More robust checking were implemented, more attempts to stage, a fallback strategy etc.. merge jobs caused tipically an overload of the storage system because of the high rate of read access; CASTOR2 at CNAF was tuned to cope with the needs of the production (D. Bonacorsi and CNAF admins) site validation: storage, software tag, software mount points, matching of CE consistency between fileblock/files in DBS/DLS and the reality at sites. Support of Italian Tier-1 and Tier-2 very effective also in August

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 14 CSA06 reconstruction, calibration/alignment

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 15 Reconstruction with CMSSW_1_0_x (x 6) –All main reconstruction components included Detector-specific local reconstruction and clustering Tracking (only 1 algo used), vertexing, standalone, jets Global (with tracker), electrons, photons, b&tau tagging –Reconstruction time small (no p/u!): 4.5s/ev MB, 20s/ev TTbar Computing model assumes 25 s/ev Calibration/Alignment –Ability to pull in constants from Offline DB included for ECAL, Tracker, and Muon reconstruction –Direct access to Oracle or via Frontier cache Tier-0 tasks in CSA06

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 16 Processing for CSA officially launched October 2 –First week mostly minbias (with some EWK) using CMSSW102 while bugs fixed to improve robustness on signal samples Second week processing included signal samples at rates generally matched to T1 bandwidth metrics and using CMSSW103 After having run for about 23 days, 120M events at 100% uptime, decided to increase scale for last days Reprocessed all signal samples in ~5 days using CMSSW106 and maximum CPU usage –Useful to re-do some samples (FEVT, Reco, AOD, AlCaReco) because of some problems/mistakes in earlier generation (missing files, missing muon objects) Performance: –160 Hz processing rate, peaking at 300 Hz signals, minbias, and HLT split samples –1250 CPUs for prompt reconstruction –150 CPUs for AOD and AlCaReco production (separate step) –All constants pulled from Frontier i.e. full complexity of CSA exercise 4 weeks uptime (goal), 207M events processed Tier-0 operations

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 17 Calibration/alignment tasks: –Specialized tasks to align/calibrate subsystems using start-up miscalibrated samples, e.g. Align a portion of Tracker with HIP algorithm by using Z sample on the central analysis facility (CAF) for prompt calibration/alignment Intercalibrate ECAL crystals by phi symmetry in minbias events, 0 /, or by isolated electrons from W/Z –Specialized reduced RECO data format (AlCaReco) to be used for calibration/alignment stream from Tier-0 –Mechanism to write constants back into offline DB to be used –Re-reconstruction at Tier-1 required to test new constants –Propose that miscalibration is applied at RECO Datasets for alignment exercise: Z µµ Calibration/Alignment exercise at Tier-0 CAF

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 18 Tracker Alignment exercise CSA06 misalignment scenario: TIB dets and TOB rods misaligned by applying: random shifts, drawing from a flat distribution of witdth +/-100 m in (x,y,z) for the double sided modules and in x (sensitive coordinate) for the single sided ones random rotations, drawing from a flat distribution of witdth +/-10 mrad, in (alpha,beta,gamma) for all the modules TIB double sided dets positions Alignment exercise: to read the object in the DB, to apply the initial misalignment; to run the iterative HIP algorithm and to determine alignment constants; 1M events used and 10 iterations. jobs running in parallel on 20 CPUs on a dedicated queue at Tier-0; new costants inserted into the DB

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 19 Tomcat and squids (caching servers) in place and tested before CSA DB populated with some sets of constants –No miscalib., start-up miscalib. (4%), etc… But multiple failures on first tests –Crashes (needed CORAL patch) –Logging of 28K queries/job kills servers (disabled) Successfully in CSA by ~Oct.24 In CSAGood Tests Failed tests Access to DB via Frontier

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 20 All 7 Tier-1 centers participated in the challenge performing very well –some storage element software or hardware problems at individual sites –but all have recovered and rapidly cleared any accumulated backlogs The longest down time at any site has been about 18 hours Files are injected into the CMS data transfer system PhEDEx and transferred using FTS –One central service failures Recovery has been rapid –Highest rate from CERN was 550MB/s First 3 Week Average SiteRate ASGC14.3MB/s CNAF18.0MB/s FNAL47.8MB/s GridKa21.7MB/s IN2P314.6MB/s PIC14.4MB/s RAL16.4MB/s Total147MB/s Transfer Tier-0/Tier-1s

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 21 Transfer Tier-0/Tier-1s Transfer to Tier1 CNAF overall successfull …..after the prompt reconstruction at Tier-0:

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 22 To fit data at T2, and to reduce primary datasets to manageable sizes, it was needed to run skim jobs at T1s to select events according to the analyses Skim configuration files prepared according to the RECO and AOD format (also including some MC truth information) Organized skim jobs ran with ProdAgent Different skim procedures prepared by the users for running on the same dataset were unified in a single skim job producing different streams 10 filters prepared by the Italian people to cope with the analyses prepared 4 teams for running skim jobs at tier-1s –N. De Filippis: Electroweak soup (RAL, CNAF, ASGC, IN2P3) –D. Mason: Jets (FNAL) –C. Hof: TTbar ( FZK and FNAL) –J. Hernandez: Zmumu (PIC and CNAF) Skim job output files shipped to Tier-2s for end-user analyses 9 Oct. – T1 Skim jobs started Skimming data at Tier-1s

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 23 First RECO/AOD definition completed for CSA06 production RECO Content: –Tracker Clusters Rec-hits skipped for disk space reasons Can be recomputed from clusters –Ecal/HCal/Muon RecHits –Track core plus extra + attached RecHits Refitting is straightforward from attached hits –Vertices, Ecal Clusters, Calo Towers –High Level Objects Photons, Electrons (links with tracks missing…), Muons, Jets, Met (from Calo Towers and Generator) Tau tagging –HLT output summary Trigger bits + links to High Level Objects (as candidates…) –HepMC Generator –Geant 4 Tracks/Vertices AOD Content: a proper subset of RECO –Clusters, Hits are dropped –Track core only saved Cant refit a track on AOD Only muon tracks have RecHits attached in AOD –Vertices, Ecal Clusters, Calo Towers –High Level Objects, HepMC Generator RECO/AOD data formats

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 24 Monitoring of skim jobs at Tier-1s

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 25 Transfer of skim outputs from Tier-1s to Tier-2s Problems related to: wrong config. of Tier-2 sites wrong setup of download agents with FTS CNAF related problems (FTS server, CASTOR)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 26 Exceeded 1PB in 1 month! Total transfer Tier-0 to Tier-1s and Tier-2s

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 27 Analyses at Tier-2s (1) P. Govoni

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 28 Analyses at Tier-2s (2) All INFN Tier2s took part to the last step of the CSA06: the physics analyses starting from the output of skim procedures Legnaro/ Padua (W selection ) Rome (electron reco) Pisa (tau validation) Bari (tracker misalignment) (Study of minimum bias/underlyin g event)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 29 1)Electron and Z mass reconstruction using the hybrid supercluster energy (barrel only): Analysis at Rome Three analyses with goal: 1. to study of the electron reconstruction in Z ee events (Meridiani) 2. to measure the W mass in W e events (Tabarelli De Fatis, Malberti, CMS NOTE ) 3. to run a simple calibration with W e events (Govoni) Eff vs pT Eff vs mZmZ

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 30 Analysis at Pisa (1) The general idea is to simulate a "early data taking" activity of the group: 1. the goal is to study the tau tag efficiency from the Z events (like described in CMS/AN 2006/074) 2.the goal is to study the misidentification with the recoiling jet with Z+jet, Z events 3.In addition: run validation package on skimmed events 3) The validation package has been run on pure di-tau sample and on skimmed ttbar sample (S. Gennai, G. Bagliesi). Isolation efficiency vs Isolation Cone : pT of the jet

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 31 Monte Carlo tuning for LHC Pileup undestanding UE contribution measurements in MB events Analysis at Pisa (2) Study of minimum bias/underlying event (Fanò, Ambroglini, Bartalini): MinBiasUE

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 32 Analysis at Legnaro Goal: to study the W preselection with different Monte Carlo data samples Two data samples were considered (Torassa, Margoni, Gasparini): (1) the electroweak soup (3.4 M evts, 50% W and 50% DY) (2) the soft muons (1.8 M evts, 50% minimum bias and 50% J/, pT > 4 GeV) The transverse momentum, the efficiency vs and vs pT as obtained with the GlobalMuon reconstructor (to be compared with standalone…) EWK soup

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 33 Analysis at Bari Goals: to study the effect of tracker misalignment on track reconstruction performances (De Filippis) : a)with the perfect tracker geometry; b)in the short term and in the long term misalignment scenario by reading misalignment position and errors via frontier/squid from the offline database ORCAOFF. c)by using the tracker module position and errors as obtained by the output of the alignment process that will be run at CERN T0. Data samples used: Z and TTbar (the second for computing the fake rate)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 34 Analysis jobs at Bari CRAB_1_4_0 used to submit 1.8 k jobs grid efficiency = 99 %, appl. eff = 94 % Bunch of 150 jobs run in different time slots max 45 jobs run in parallel the configuration of squid tuned to ensure that the alignment data were read by the local cache of squid via the frontier client rather than from CERN (blue histo). frontier/squid works as expected at tier-2 Bari when accessing alignment data

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 35 The last step of CSA06: Re-reconstruction at Tier-1s Goals: to demonstrate re-reconstruction from some RAW data at Tier-1s as part of the calibration exercise Status: access of Offline database via frontier working re-reconstruction demonstrated at ASGC, FNAL, IN2P3, PIC and CNAF Running at RAL and further tests at CNAF PIC

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 36 Problems with CMSSW: - the "reasonability" of the code was not too much taken into account. Operations were driven by computing, and the feeing was: "whatever you run we do not care. It is enough it is not crashing". -as it often happens in this case, the release schedule was crazy. Also the initial milestones were somehow crazy, and it meant a really hard work to cope with them. -CSA06 meant blocking developments for some time, to make sure we were maintaining the backward-compatibility. But it also meant a lot of code had to live either in the head, or in pre releases for some time. It would be better to have specifically two releases ongoing at a time: a production one, and a development one. - Framework proved to be usable for T0 reconstruction. HLT was not attempted at CSA06 and so no conclusions on that. What should have to work better (1)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 37 What should have to work better (2) Storage system: CASTOR and DPM support (in general rfio access ) for CMS application had a lot of problems ( libdpm patched, > 2 GB files required a patch) CASTOR updates too much critical for the operation during the CSA06 operations: that caused a lot of problems and an emergency status for CNAF Integration issues: all the pieces of the CSA06 worked (example: CMSSW releases, PA, skim jobs, DBS/DLS interactions) but a lot of effort of operation teams to make them integrated each other; PA: tool that required a lot of distributed expertise, dedicated hw/sw setup (at least three machines), real–time monitoring the CMS SW installation in remote sites was problematic LCG/OSG performances very good

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 38 Conclusions/suggestions CSA06 was successful at INFN (all the steps were executed) but thanks to the 100 % work of few experts and to the coordinated effort of many people at Tier-1 and Tier-2 sites. CSA06 was supposed to be a challenge to commission the computing/software/analysis system but in some cases it required also development/deployment of the tools CSA06 analysis exercises could be as the ramp-up for the physics program/organization in Italy A new CSA would be the best for 2007 with simulated and real data; focus on start-up operations (calibration and alignment) and analysis preparation

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 39 Backup slides

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 40 pccms30 ProdAgent UI Test and backup setup + PhEDEx injection ProdAgent UI I pccms6 DB mirror pccms28 Apache 2.0 php MySQL ProdAgent UI II pccms27 Apache 2.0 php MySQL pccms29 PhEDEx server PA_035, PA_041 PA_045, PA_047 various productions monitored Managed By different PA versions Production setup at Bari

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 41 Monitoring of production via web interface First prototype of monitoring was developed by Bari team:

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 42 CTF tracks pixel clus. island basic custers ecal hits island super clusters It. cone, 5 calo-towers HB/HE hits Tracker/VerticesE/GammaHCal/Jets/Met AOD RECO Muons strip clus. preshower hits HF hits HO hits DT 1D hits DT 2D segm. DT 4D segm. CSC 2D hits. CSC segm.. RPC hits RS tracks RS extra RS tr. hits GSF tracks GSF extra GSF tr. hits pixel tracks pixel extra pixel tr. hits Stand-Alone muons Global muons MET m.p. cone, 7 m.p. cone, 5 hybrid super clusters calo-tower cands strip.electrons b/tau tags cone-iso trk count. HLT trigger res. pix.electrons photons Kt jets Particle Candidates Track extensions CTF tr. hits CTF extra corr. hybrid s.c. corr. island s.c. S-Alone extra S-Alone tr. hits Global extra Global tr. hits muons primary vertices e/gamma tracks Event content: RECO/AOD

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 43 Overwhelming response from CSA analysis demonstrations About 25 filters producing ~37 (and 21 jet) datasets ! Variety of outputs and sizes: FEVT, RECOSim, AlCaReco Skimming filters

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 44 Analysis at Bari Goals: to study the effect of tracker misalignment on track reconstruction performances. a)with the perfect tracker geometry; b)in the short term and in the long term misalignment scenario by reading misalignment position and errors via frontier/squid from the offline database ORCAOFF. This step requires to refit tracks with misaligned geometry but it can be done at the T2. The effect of alignment position error APE to be checked. c)by using the tracker module position and errors as obtained by the output of the alignment process that will be run at CERN T0 to verify the efficiency of the alignment procedure on the track reconstruction. Refit of tracks to be done in the T2. Global efficiency of track recostruction, track parameter resolution and fake rate are compared in the a), b) and c) cases. The same analysis was performed in ORCA. Plots and documents at link: Data samples needed: Z and TTbar (the second for computing the fake rate)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 45 The CSA06 chain of the needed data samples a)Z and TTbar samples produced during CSA06 pre-production with CMSSW_0_8_2. b)CSA06 events reconstructed at T0 with CMSW_1_0_3 (and Z with CMSSW_1_0_5 in transfer) c)2 skim cfg files used for skimming Z and TTbar sample. Skim jobs just run at T1 with CMSW_1_0_4 and CMSSW_1_0_5 and output data in reduced format RECOSIM are produced. RECOSIM includes enough information for misalignment analysis. Z filter: to select HepMC muons from Z decay with | with p T > 5 GeV/c 2 and 50 < m (Z Filter efficiency between 50 and 60 %. bar filter: to select events with two muons with | | 15 GeV/c2 d)RECOSIM produced with CMSSW_1_0_4 transferred at T2-Bari and misalignment analysis run over RECOSIM with CMSSW_1_0_6. e)¼ of the full statistics already analyzed at T2-Bari ….waiting for all the statistics of the samples.

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 46 Track selection Selection: track seeding, building, ambiguity resolution, smoothing with KF. ctfWithMaterialTracks refit after applying alignment uncertainties track associator by to match simtracks with rectracks Efficiency: number of reco tracks matching simul. tracks / number of simul tracks - Simul. track: p T 0.9 GeV/c, Reco. track: p T 0.7 GeV/c, 0< <2.6, d0 120 cm, z0 170 cm, n hit8 Fake Rate: number of reco tracks not associated to simul tracks / number of reco tracks - Simul. track: p T 0.7 GeV/c, 0 8 not used because Simtrack does not have the number of simihit method Tracking Particle will have but TP is not compatible with CSA data samples - Reco. track: p T 0.9 GeV/c 0< <2.5, d0 3 cm, z0 30 cm, n hit8 Track parameters resolution: sigma of Gauss fit to distribution of residuals

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 47 Analysis jobs at Bari CRAB_1_4_0 used to submit 1.8 k jobs grid efficiency = 99 %, appl. eff = 94 % Bunch of 150 jobs run in different time slots max 45 jobs run in parallel the configuration of squid tuned to ensure that the alignment data were read by the local cache of squid via the frontier client rather than from CERN (blue histo). frontier/squid works as expected at tier-2 Bari when accessing alignment data

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 48 Eff. / P T resolution with muons from Z The effect of misalignment affects the global track reconstruction efficiency in the first data taking scenario. The effect of tracker misalignment is enough relevant in track parameters resolution (factor 2-3 of degradation)

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 49 Track param. Resol.: d 0 and z 0 A factor between 2 and 3 in impact parameters resolution due to misalignment

Nicola De Filippis Workshop sulla fisica di ATLAS e CMS, Bologna, Nov p. 50 Z mass from di-muons Using CSA06 Z sample The Z mass resolution is increased by a factor larger than 2 in the first data taking scenario (RMS from 1.3 to 2.8)