Predrag Buncic ALICE Status Report LHCC Referee Meeting CERN

Slides:



Advertisements
Similar presentations
Status GridKa & ALICE T2 in Germany Kilian Schwarz GSI Darmstadt.
Advertisements

T1 at LBL/NERSC/OAK RIDGE General principles. RAW data flow T0 disk buffer DAQ & HLT CERN Tape AliEn FC Raw data Condition & Calibration & data DB disk.
DATA PRESERVATION IN ALICE FEDERICO CARMINATI. MOTIVATION ALICE is a 150 M CHF investment by a large scientific community The ALICE data is unique and.
ALICE Operations short summary and directions in 2012 Grid Deployment Board March 21, 2011.
ALICE Operations short summary LHCC Referees meeting June 12, 2012.
ALICE Operations short summary and directions in 2012 WLCG workshop May 19-20, 2012.
Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.
Claudio Grandi INFN Bologna CMS Operations Update Ian Fisk, Claudio Grandi 1.
Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.
Status of 2015 pledges 2016 requests RRB Report Concezio Bozzi INFN Ferrara LHCb NCB, November 3 rd 2014.
ALICE Upgrade for Run3: Computing HL-LHC Trigger, Online and Offline Computing Working Group Topical Workshop Sep 5 th 2014.
Predrag Buncic CERN ALICE Status Report LHCC Referee Meeting September 22, 2015.
ATLAS in LHCC report from ATLAS –ATLAS Distributed Computing has been working at large scale Thanks to great efforts from shifters.
ALICE – networking LHCONE workshop 10/02/ Quick plans: Run 2 data taking Both for Pb+Pb and p+p – Reach 1 nb -1 integrated luminosity for rare.
PWG3 Analysis: status, experience, requests Andrea Dainese on behalf of PWG3 ALICE Offline Week, CERN, Andrea Dainese 1.
Marco Cattaneo LHCb computing status for LHCC referees meeting 14 th June
2012 RESOURCES UTILIZATION REPORT AND COMPUTING RESOURCES REQUIREMENTS September 24, 2012.
Predrag Buncic Future IT challenges for ALICE Technical Workshop November 6, 2015.
LHCbComputing LHCC status report. Operations June 2014 to September m Running jobs by activity o Montecarlo simulation continues as main activity.
LCG-LHCC mini-review ALICE Latchezar Betev Latchezar Betev for the ALICE collaboration.
LHCb report to LHCC and C-RSG Philippe Charpentier CERN on behalf of LHCb.
1 Offline Week, October 28 th 2009 PWG3-Muon: Analysis Status From ESD to AOD:  inclusion of MC branch in the AOD  standard AOD creation for PDC09 files.
Predrag Buncic ALICE Status Report LHCC Referee Meeting CERN
ATLAS Distributed Computing perspectives for Run-2 Simone Campana CERN-IT/SDC on behalf of ADC.
Predrag Buncic CERN ALICE Status Report LHCC Referee Meeting 01/12/2015.
Ian Bird WLCG Networking workshop CERN, 10 th February February 2014
Ian Bird Overview Board; CERN, 8 th March 2013 March 6, 2013
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
ALICE Computing Model A pictorial guide. ALICE Computing Model External T1 CERN T0 During pp run i (7 months): P2: data taking T0: first reconstruction.
ALICE Grid operations +some specific for T2s US-ALICE Grid operations review 7 March 2014 Latchezar Betev 1.
1 Reconstruction tasks R.Shahoyan, 25/06/ Including TRD into track fit (JIRA PWGPP-1))  JIRA PWGPP-2: Code is in the release, need to switch setting.
Main parameters of Russian Tier2 for ATLAS (RuTier-2 model) Russia-CERN JWGC meeting A.Minaenko IHEP (Protvino)
16 September 2014 Ian Bird; SPC1. General ALICE and LHCb detector upgrades during LS2  Plans for changing computing strategies more advanced CMS and.
Meeting with University of Malta| CERN, May 18, 2015 | Predrag Buncic ALICE Computing in Run 2+ P. Buncic 1.
H. Matis, S. Hedges, M. Placidi, A. Ratti, W. Turner [+several students] (LBNL) R. Miyamoto (now at ESSS) H. Matis - LARP CM18 - May 8, Fluka Modeling.
Domenico Elia1 ALICE computing: status and perspectives Domenico Elia, INFN Bari Workshop CCR INFN / LNS Catania, Workshop Commissione Calcolo.
Computing infrastructures for the LHC: current status and challenges of the High Luminosity LHC future Worldwide LHC Computing Grid (WLCG): Distributed.
Predrag Buncic CERN Plans for Run2 and the ALICE upgrade in Run3 ALICE Tier-1/Tier-2 Workshop February 2015.
LHCb LHCb GRID SOLUTION TM Recent and planned changes to the LHCb computing model Marco Cattaneo, Philippe Charpentier, Peter Clarke, Stefan Roiser.
Predrag Buncic CERN ALICE Status Report LHCC Referee Meeting 24/05/2015.
ATLAS – statements of interest (1) A degree of hierarchy between the different computing facilities, with distinct roles at each level –Event filter Online.
Data Formats and Impact on Federated Access
T1/T2 workshop – 7th edition - Strasbourg 3 May 2017 Latchezar Betev
Ian Bird WLCG Workshop San Francisco, 8th October 2016
SuperB and its computing requirements
Analysis trains – Status & experience from operation
evoluzione modello per Run3 LHC
DPG Activities DPG Session, ALICE Monthly Mini Week
Workshop Computing Models status and perspectives
LHC experiments Requirements and Concepts ALICE
Data Challenge with the Grid in ATLAS
Status of the CERN Analysis Facility
Vanderbilt Tier 2 Project
Update on Plan for KISTI-GSDC
Offline data taking and processing
LHCb Software & Computing Status
Philippe Charpentier CERN – LHCb On behalf of the LHCb Computing Group
Dagmar Adamova (NPI AS CR Prague/Rez) and Maarten Litmaath (CERN)
Disk capacities in 2017 and 2018 ALICE Offline week 12/11/2017.
Simulation use cases for T2 in ALICE
ALICE Computing Model in Run3
Status of ALICE pledges Predrag Buncic
ALICE Computing Upgrade Predrag Buncic
ILD Ichinoseki Meeting
The latest developments in preparations of the LHC community for the computing challenges of the High Luminosity LHC Dagmar Adamova (NPI AS CR Prague/Rez)
New strategies of the LHC experiments to meet
ZEUS Computing Board Report
J/Y Simulations for Trigger
Computing at the HL-LHC
The ATLAS Computing Model
Presentation transcript:

Predrag Buncic ALICE Status Report LHCC Referee Meeting CERN 09/05/2017 ALICE Status Report Predrag Buncic CERN

Run 2: Data taking objectives For Pb-Pb collisions: Reach the target of 1 nb-1 integrated luminosity in Pb-Pb for rare triggers. Increase the statistics of the unbiased data sample, including minimum bias and centrality triggered events. For pp collisions: Collect a reference rare triggers sample with an integrated luminosity of 40 pb-1, which is equivalent to the 1 nb-1 sample in Pb-Pb collisions. Enlarge the statistics of the unbiased data sample, including minimum bias collisions at top energy. Collect a reference sample of 109 events at the reference energy of 5.02 TeV For p-Pb collisions: Enlarge the existing data sample, in particular the unbiased events sample at 5.02 TeV.

Run 2: Progress and data processing status p-p HLT compression, High IR Pb-Pb End of year stop No HLT compression HLT + ROOT compression, Low IR p-A Readout electronics upgrade (RCU2) with goal to increase the readout rate by a factor of 2 Originally planned for 2015, the final version installed from 2016 Data taking follows strictly the approved physics programme Better data quality by limiting the Interaction Rate Continuously reducing RAW data volume by improving the HLT data compression RAW data size reduction by factor ~4.3 to ~5.5 in 2016, expecting factor 6 in 2017 Data processing – new high-precision calibration schema developed and certified 2015 Pb-Pb and 2016 p-Pb data fully processed, p-p data processing at 90% 2015 p-p data at 60%, processing will be completed by June 2017

Run 2: Computing model Three reconstruction passes are performed for every data set with increasing quality The first reconstruction pass is started after the end of data taking. The second and third reconstruction pass are performed after one month delay following the end of the previous pass. Analysis on a growing set of AOD is run daily using Analysis Trains MC production is continuously spread over the year. One MC event per real pp event is produced and 0.175 MC events per real AA event. A subset of analysis still runs on ESDs

Changes in Computing Model Parameters RCU2 not ready for 2015 pp/PbPb data taking and became fully operational in 2016 The most data intensive data taking with central PbPb collisions postponed to 2018 Requires CPU intensive simulation campaign Larger than expected distortions in TPC observed at high IR Fully corrected during 2016 at the cost of +5% CPU increase Average observed pp event size larger than expected (3MB instead of 1.1MB) Due to pileup that we cannot reject online HLT compression improved Compression 4.3 -> 5.5 -> 6 Improved data taking efficiency ALICE efficiency 95% LHC efficiency 40% -> 60% Taking data at lower IR Increased combined efficiency allows us to collect expected number of triggers while taking data at lower IR reducing the pileup, event size and processing time All these factors were taken into account in October 2016 when we updated our requirements for 2017-2019.

Expectations for 2017 and 2018 data volume During pp data taking mode will be set to limit the TPC readout rate to 400 Hz The goal is to reach the statistics objective set for Run 2 in all trigger categories as well as at the reference energy of 5.02 TeV The total amount of data recorded will be 17.5 PB During the Pb-Pb run in 2018, assuming the HLT compression of a factor of 6 we anticipate a total readout rate of 10 GB/s The total amount of data recorded will be 12 PB

Thanks to CERN and T1s, our tape request has been fully pledged in 2017

CPU Usage Up to 130k concurrent jobs Up to 3PB of data analyzed daily Up to 130k concurrent jobs Available resources are effectively used in spite of increasing complexity and data volume

NOT APPROVED 19% 57%

NOT APPROVED 33% 86% NOT APPROVED 20% 51%

Data popularity and cleanup 11PB 1.5PB already removed since February Relatively significant volume of data (11PB) that has not been accessed in past 12 months If the window is enlarged to 2 years unused data portion drops down to 2.5 PB We do not systematically back up ESD/AOD to tape, only RAW data Regular cleanup procedure with cooperation of Physics Working Groups Automated cleanup procedure would require To systematically back up ESD/AOD to tape An additional tape budget and larger disk buffer in front of tape system to accommodate ESD/AODs in addition to raw data

What occupies ALICE disk storage? Storage availability 13 SEs, (20%), 2.6PB, 5% 99% 95% 90% 100% 4 days downtime 18 days downtime 37 days downtime 28 SEs (40%) 20 SEs (30%) 7 SEs, (10%) 7.4PB, 15% In one year 1 2 Each raw event is stored only once on tape and temporary staged in staging disk buffer Old inactive MC productions are systematically removed All reconstruction output + associated MC Recent ESDs are stored as single file w/o replicas due to lack of disk space 2 replicas for AODs Already operating at the lowest limit with no room for more reduction of disk usage ESD based analysis is at the pain threshold with up to 5% jobs failing in each round

ALICE share of computing resources +22.31% +4.74% +26.4% +2.11% +36.2% +3.71% CPU Disk Tape kHS06 PB ALICE share of total resources remains balanced vs. the other experiments Historically, growth in all resource categories was faster than +20% year on year The impact of fully implemented ALICE 2018 request vs just +20% increase is relatively small PB

Projection 2017-2020 assuming 20% growth CPU Disk Tape Tier Requested1) Pledged2) missing requested pledged 2017 292.00 0.00% 20.60 22.40 8.74% 34.5 36.9 6.96% 1 256.00 235.48 -8.02% 24.40 21.81 -10.64% 26.60 30.60 15.04% 2 366.00 323.05 -11.74% 31.20 26.39 -15.41%   Total 914.00 850.53 -6.94% 76.20 70.60 -7.35% 61.10 67.50 10.47% 2018 350.00 350.40 0.11% 27.00 26.88 -0.44% 55 306.00 282.58 -7.65% 32.00 26.17 -18.23% 41.00 438.00 387.66 -11.49% 31.67 -22.76% 1094.00 1020.63 -6.71% 100.00 84.72 -15.28% 96.00 2019 534.00 420.48 -21.26% 33.60 32.26 -4.00% 501.00 339.09 -32.32% 39.90 31.40 -21.31% 49.50 635.00 465.19 -26.74% 51.10 38.00 -25.63% 1670.00 1224.76 -26.66% 124.60 101.66 -18.41% 104.50 2020 504.58 -5.51% 38.71 15.20% 406.91 -18.78% 37.68 -5.57% 558.22 -12.09% 45.61 -10.75% 1469.71 -11.99% 121.99 -2.09% The resource request is based on April submission to C-RSG In this table the resources provided by US, Japan and Austria are counted as fully pledged If we are forced to remain under 20% resource growth envelope and cannot find additional resource we won’t be able to process 2018 pp and PbPb data during 2019 The only solution would be to extend the processing time to 2019 + 2020 This would put ALICE at disadvantage with respect to other LHC experiments in areas were they are competitive with us Manpower would remain tied to Run2 processing during time that we need to focus on preparations for Run3 

Conclusions ALICE data taking Strictly following the approved physics programme Intensifies in second half of Run 2 as we profit from improved readout and LHC performance to catch up with our plans Pledged and delivered computing resources Systematic under-pledges and under-delivery at T2s over several years This artificially skews our 2018 requirements/pledges ratio prompting C-RSG to question our 2018 requirements 2018 data taking will be particularly data intensive pp and PbPb runs with high multiplicity central triggers If our request are not fulfilled, we will have to extend the processing time to 2019 + 2020 Radical measures already taken Reducing replicas, now at absolute minimum (no replicas for any file but AODs), aggressive disk cleanup We are in discussion with FAs and have positive initial feedback concerning the pledges but we need support for LHCC/C-RSG or else there will be no funding available for us in October