1. The CMS-HI Computing Plan Vanderbilt University
Charles F. Maguire
Vanderbilt University
for the CMS-HI Group
Version V1, May 29
21:41 Geneva time
About half completed
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

2. Outline of the Review Talks
Dennis Hall: Support of Vanderbilt University for CMS-HI
Bolek Wyslouch: Overview of CMS-HI Physics Plans
Charles Maguire: Detailed View of CMS-HI Computing
Markus Klute: T0 Operations in Support of CMS-HI
Lothar Bauerdick: FNAL T1 Operations for CMS-HI
Raphael Granier de Cassagnac: Contribution of non-US T2s
Edward Wenger: HI Software Status Within CMS Model
Alan Tackett: The Role of ACCRE in CMS-HI Computing
Esfandiar Zafar: ITS Support of ACCRE in CMS-HI
Alan Tackett: An Inspection Tour of the Proposed T2 Site
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

3. Summary of Updated Computing Proposal
CMS-HI Computing Plans Follow Very Closely the CMS Model
Extensive use of world-wide CMS computing resources and working groups
Continuous oversight by upper level CMS computing management
Alignment, Calibration, and On-Line DQM at the T0
A Complete Prompt Reconstruction Pass to be Done at the T0
Archival Storage at the T0 and Standard Transfer to FNAL T1 Site
Secondary Archival Storage at the FNAL T1
Transfer of Files to Disk Storage at a New Vanderbilt T2 Site
Vanderbilt Site Will:
Perform analysis passes on the prompt reco data set
Distribute analyzed data sets to MIT and to four non-US T2 HI sites
Complete multiple reconstruction and analysis re-passes on the data sets
Contain a T3 component which will host US CMS-HI participants (MIT already does)
An Enhanced Role of the MIT HI Site for Simulation and Analyses
Non-US T2 Sites Contribute a Significant Fraction of Analysis Base
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

4. What is New and Different This Year
Sustained Use of the T0 for Prompt Reconstruction
Confirmed by new, rigorous simulations monitoring time and memory use
Each annual HI data set can be promptly reconstructed in time at the T0: just a few days for the 2010 HI min bias data; 18 – 25 days for later HLT data
NOTE: The T0 cannot be used as a T1 or a T2 site by CMS-HI (or by anyone else)
Standard Transfer of HI Files from the T0 to the FNAL T1
To be supervised by the transfer group at the T0; simply one more month of duty
Further details contained in Markus Klute’s presentation
Strongly recommended by CMS computing management at Bologna workshop No adverse impact is seen for the pp program
More explanations found in Lothar Bauerdick’s presentation
CPU Requirements for CMS-HI Calibrated in HS06 Units
Standard measure for the LHC experiments used throughout CMS
CMS-HI software all runs in latest, official CMSSW framework
Large investment of effort in determining processing times, memory use, and file sizes
Careful Inventory of Potential non-US T2 HI Contributions
Full accounting is tabulated in Raphael Granier de Cassagnac’s presentation
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

5. Overview of HI Computing: DAQ and T0
Anticipated Start of HI Running
Change to Pb + Pb acceleration expected on November 1
Two weeks of set-up time
Physics data taking expected to begin in mid-November
Pb + Pb running for 4 weeks, assuming 106 seconds live time
Special DAQ Considerations (see Markus Klute’s talk)
Non-zero suppression of ECal and HCal, leads to ~10 MBytes/event
Zero suppression to be achieved off-line at the T0, leads to 3 MB/event
Work flows and CPU requirements for this task to be completed in summer 2010
Data Processing
AlCa and DQM to be performed on-line as in the pp running
DQM for HI being advanced by Julia Velkovska (at Vanderbilt) and Pelin Kurt (at CERN
“Live” demonstration during pp running scheduled at the post-review ROC tour
Prompt reco of the zero-suppressed files at the T0, < 7 days total
See the next set of slides for how the CPU requirements were determined
Leads to 1 MByte/event, ~400 TB of raw data and prompt reco files transferred to FNAL T1
Processing after FNAL should be at Vanderbilt (but see back-up “Plan B” later)
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

6. Determination of the CMS-HI Computing Requirements
In a Processor Independent Fashion
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

7. Method of Determining CPU Requirements
All CMS-HI Software is in Standard CMSSW Release Schedule
Major accomplishment since the last review (see Edward Wenger’s talk)
Ensures that the validation tests are done and the timing results are correct
Simulation Data Testing (done by MIT and Vanderbilt graduate students)
Ran in CMS simulation configurations modified especially for HI use
Looked at minimum bias events for modeling 2010 HI run
Looked at central events (< 10% centrality) for modeling future HLT runs
Each stage of the simulation process was specifically checked
GEANT4 tracking stage in the CMS detector (by far the most time consuming step)
Reconstruction step, using simulated raw data files from the first step
CPU times, memory consumptions, and file output sizes were all recorded
Tests were done on processors with already known HS06 rating
Estimate of CPU Requirements Was Made in a Data-Driven Fashion
Scaled according to projected kind and annual number of events (see next slide)
Provision was made for a standard number of reco and analysis re-passes annually
Experienced based assumptions were made for analysis and simulation demands
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

8. Annual Raw Data Volume Projections
Table 4 (page 20): Projected luminosity and up-time profile for CMS-HI runs
Year
Ave. Lumin.
(cm-2s-1)
Up Time
(s)
Events Taken
(106)
Raw Data
(TByte)
2010
2 – 5 x 1025
105
40 – 80 (Min Bias)
~250
2011
5 x 1026
5 x 105
50 (HLT)
150
2013
1 x 1027
106
75 (HLT)
300
2014
Notes:
1) First year figures are relatively uncertain
2) HI collision event size characteristics are completely unexplored at this energy
3) LHC down year is in 2012
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

9. HI Data Reconstruction Estimates
Table 5 (page 21): Projected raw data reconstruction computing effort
Year
Trigger
Events
(106)
Compute Load
(1010 HS06-sec)
T0 Reco
(days)
Re-passes
Time at VU
(Days/Re-pass)
2010
Min Bias
105
1.7 4 3 72
2011
HLT
5 x 105
8.5 18
1.3
133
2012
No Beam
Above
None
2.7 65
2013
106
12.8 25 2 79
2014
Notes:
1) Column four compute-load is the integrated power for one reconstruction pass
2) Column six assumes the four year annual growth of HS06 power at VU to be , 8588, 17708, 23028
3) Column six also assumes VU reco fractions are 0.65, 0.55, 0.55, 0.45, and 0.45
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

10. HI Data Analysis Estimates
Table 8 (page 24): Integrated T2 Computing Load Compared to Available Resources
Year
Analysis + Simulation Need
(1011 HS06-sec)
Vanderbilt T2
Total T2 Base
Ratio:
Available/Need
2010
1.47
0.29
1.52
104%
2011
2.54
0.98
2.45
97%
2012
3.73
2.01
100%
2013
4.71
3.20
5.15
109%
2014
114%
Notes:
1) Column two analysis and simulation needs are computed in back-up slides
2) Column three VU T2 values are computed using HS06 growth model in slide 9
3) Column four total T2 base assumes an MIT HS06 growth model of (already in place), 2850, 3800, 4750, and 5700
4) Column four total T2 base also assumes 3000 HS06 from non-US T2 HI sites
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

11. Determination of the Storage Volume Requirements
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

12. Vanderbilt Local Infrastructure: ACCRE and ITS
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

13. MIT Heavy Ion Analysis Center
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

14. HI Computing Operations
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

15. HI View of Personnel Responsibilities
1) It is important that all the CMS experts in each system shown on the left (Detector, Readout, ...) realize that their expertise will be critically needed during the HI run. For some systems, these experts will be almost completely in charge, while in others (RECO, SIM, ...) the HI persons will be taking the lead in giving directions.
2) In April we experienced the loss of a key HI person with knowledge of the T0 operations. It is very unlikely we can replace that person in time to make a difference in the summer preparations. So early, dedicated help for the T0 operations assisting the HI students and post-docs will be vital
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

16. HI Computing Organization
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

17. DOE-NP On-Site Review at Vanderbilt
Proposal Budget
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

18. DOE-NP On-Site Review at Vanderbilt
Project Management
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

19. Summary of Responses to First Review Comments
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

20. Comments on Resource Requirements
To Be Completed
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

21. Comments on Quality of Service Burdens
To Be Completed
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

22. Comments on Computing Operations
To Be Completed
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

23. DOE-NP On-Site Review at Vanderbilt
Summary
To Be Completed
June 2, 2010
DOE-NP On-Site Review at Vanderbilt

24. DOE-NP On-Site Review at Vanderbilt
Backup Slides (TO BE COMPLETED)
June 2, 2010
DOE-NP On-Site Review at Vanderbilt