1. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger1 The CMS High Level Trigger System LHC Days in Split 5-9 October 2004 Vuko Brigljević Institut Ruđer Bošković

2. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger2 HLT: why should even theorists care? A lot of physics will pour out of pp collisions at the LHC! may be even your preferred new physics signal; yes, but… H ?  Heavy Top ?  Z Z’? … will it be in the tiny fraction that we will keep? Plitvice Lakes National Park

3. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger3 Physics selectivity at LHC Event Rates: ~10 9 Hz Event size: ~1 M Byte Event Rates: ~10 9 Hz Event size: ~1 M Byte Event rate Level-1 Output 100 kHz Mass storage10 2 Hz Event Selection: ~1/10 13 Level-1 Output 100 kHz Mass storage10 2 Hz Event Selection: ~1/10 13 On tape Level-1 Reconstructed tracks with pt > 25 GeV Operating conditions: Higgs in 4 muons + ~20 minimum bias All charged tracks with pt > 2 GeV

4. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger4 HLT in CMS: the grand picture

5. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger5 The HLT in the CMS DAQ Builder Unit (BU) connects to switch and distributes fully built events to a collection of Filter Units (FU) The FU’s run the HLT algorithms and ask for data on a need basis DSN FS BDN CSN BCN EVM BU RURCN 100 kHz L1 output 200 Hz / BU (200 MB/s)

6. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger6 HLT requirements and operation n Boundary conditions: u Code runs in a single processor, which analyzes one event at a time u HLT (or Level-3) has access to full event data u Only limitations: l CPU time: guarantee deadtimeless operation at nominal L1 output rate l Output selection rate (~10 2 Hz) n Main requirements: u Satisfy physics program: high efficiency u Selection must be inclusive (to discover the unpredicted as well) u Allow complete freedom of HLT algorithms u Must not require precise knowledge of calibration/run conditions u All algorithms/processors must be monitored closely

7. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger7 CPU for the HLT: Filter FARM n Final stage of the filtering process: almost an offline- quality reconstruction & selection u Need real programmable processors; and lots of them n PC+Linux: the new supercomputer for scientific applications n CMS full DAQ system: ~ 2’000 dual CPU PCs = 4’000 Filter Units = ~ 40 ms / event

8. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger8 Managing complexity: Divide et impera Filter Farm divided in subfarms controlled by a Subfarm Manager headnode n Facilitates installation staging n Isolates problems n Allows DAQ subpartitions n Test of different SW version Communication protocols: Data (BU-FU): low level TCP Control & Monitoring: http, SOAP, XML

9. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger9 HLT Algorithms n Strategy/design guidelines u Use offline software as much as possible (only specific I/O) l Ease of maintenance, but also understanding of the detector l Make use of large developer community l But tight quality requirements n Flexibility & freedom to change Trigger table n Reconstruct ALL and ONLY what is needed to decide quickly: u Unpack only needed raw data (also reduces BU output) u Regional reconstruction u Intelligent steering of algorithm sequence: use L1 input All of this is made possible thanks to the “Reconstruction on demand” Design built in the CMS Reconstruction software

10. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger10 HLT (regional) reconstruction (I) Global process (e.g. DIGI to RHITs) each detector fully then link detectors then make physics objects Regional process (e.g. DIGI to RHITs) each detector on a "need" basis link detectors as one goes along physics objects: same

11. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger11 HLT (regional) reconstruction (II) n For this to work: u Need to know where to start reconstruction (seed) n For this to be useful: u Slices must be narrow u Slices must be few n Seeds from Lvl-1:  e/  triggers: ECAL   triggers:  sys u Jet triggers: E/H-CAL n Seeds ≈ absent: u Other side of lepton u Global tracking u Global objects (Sum E T, Missing E T )

12. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger12 Example: electron selection (I) n “Level-2” electron: u 1-tower margin around 4x4 area found by Lvl-1 trigger u Apply “clustering” u Accept clusters if H/EM < 0.05 u Select highest E T cluster n Brem recovery: u Seed cluster with E T >E T min  Road in  around seed u Collect all clusters in road  “supercluster” and add all energy in road:

13. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger13 Example: electron selection (II) n “Level-2.5” selection: add pixel information  Very fast, high rejection (e.g. factor 14), high efficiency (  =95%) l Pre-bremsstrahlung l If # of potential hits is 3, then demanding  2 hits quite efficient

14. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger14 Example: electron selection (III) n “Level-3” selection u Full tracking, loose track- finding (to maintain high efficiency): u Cut on E/p everywhere, plus Matching in  (barrel) l H/E (endcap) u Optional handle (used for photons): isolation

15. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger15 HLT Steering n HLT Trigger table is equivalent to a logical decision tree n Evaluation sequence optimized to minimize computation time n Allow Veto mode: HL subtriggers computed only if corresponding L1 accept on n Mean rejection time dominates the computation time HLT table can be dynamically loaded / modified during running (XML Document)

16. Vuko Brigljević, 2004 LHC Days in Split Physics Plan and Trigger Table (as of DAQ TDR)

17. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger17 Trigger table determination (I) n Startup configuration: don’t need 100 kHz on day 1 u Machine conditions non-optimal u Funds for completion of DAQ will be present later u Exploit technological developments – buy ALAP n Startup setup: u Physics startup assumptions: 2x10 33 cm -2 s -1, and a DAQ with 4 RU builders, i.e. 50 kHz throughput n Starting point: 50kHz/3  16kHz to allocate u Factor 3 is safety: accounts for all processes that have not been simulated, uncertainties in generator/simulation and beam conditions l This factor varies across experiments  Initial step: equal allocation across (1&2e/ , (1  2  ), (1&2  and jets/cross channels (e& ,  *jet, etc) u Get thresholds, efficiencies; look at physics cost; iterate

18. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger18 Trigger table determination (II) (and of course: operate at point of rapid slope change) relevant range Deciding Lvl-1 cuts: 1e/  vs 2e / , 1  vs 2 , 1  vs 2  u Create iso-rate plot (contours of “equal cost”) u For each contour (in relevant range, e.g. 2kHz, 3kHz, 4kHz) get efficiency of physics channel in 1-obj vs 2-obj requirement

19. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger19 Level-1 trigger table (low lumi) n Total Rate: 50 kHz. Factor 3 safety, allocate 16 kHz TriggerThreshold (  =90-95%) (GeV) Indiv. Rate (kHz) Cumul rate (kHz) 1e/ , 2e/  29, 174.3 1 , 2  14, 33.67.9 1 , 2  86, 593.210.9 1-jet1771.011.4 3-jets, 4-jets86, 702.012.5 Jet * Miss-E T 88 * 462.314.3 e * jet21 * 450.815.1 Min-bias0.916.0

20. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger20 HLT table (low luminosity) n Total Rate: 105 Hz TriggerThreshold (  =90-95%) (GeV) Indiv. Rate (Hz) Cumul rate (Hz) 1e, 2e29, 1734 1 , 2  80, (40*25)943 1 , 2  19, 72972 1 , 2  86, 59476 Jet * Miss-E T 180 * 123581 1-jet, 3-jet, 4-jet657, 247, 113989 e * jet19 * 52190 Inclusive b-jets237595 Calibration/other10105

21. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger21 HLT table n Issues to “fight” u Purity of streams is not the same (e.g. electrons vs muons) u Overlap (kinematically) is necessary; but also: redundancy Question most asked in large analysis meetings, when a problem is under investigation in W->e : do we see this in the muons? u But, above all, comparison of unlike things: l How much more bandwith should go to lower-P T muons than to electrons? l How should one share the bandwidth between jet*missE T and di-electrons? u Only guidance in the end of the day is efficiency to all the known channels l While keeping the selection INCLUSIVE l For this is online. Events rejected are lost forever.

22. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger22 HLT performance n With previous selection cuts ChannelEfficiency (for fiducial objects) H(115 GeV)  77% H(160 GeV)  WW*  2  92% H  ZZ  4  92% A/H(200 GeV)  2  45% SUSY (~0.5 TeV sparticles)~60% With R P -violation~20% W  e 67% (fid: 60%) WW 69% (fid: 50%) Top  X 72%

23. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger23 HLT: CPU usage n All numbers for a 1 GHz, Intel Pentium-III CPU u Total: 4092 s for 15.1 kHz  271 ms/event l Therefore, a 100 kHz system requires 1.2x10 6 SI95 u Expect improvements, additions. Time completely dominated by muons (GEANE extrapolation) – will improve u This is “current best estimate”, with ~50% uncertainty. TriggerCPU (ms)Rate (kHz)Total (s) 1e/ , 2e/  1604.3688 1 , 2  7103.62556 1 , 2  1303.0390 Jets, Jet * Miss-E T 503.4170 e * jet1650.8132 B-jets3000.5150

24. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger24 CPU Usage n Today: need ~300 ms on a 1GHz Pentium-III CPU u For 50 kHz, need 15,000 CPUs u Moore’s Law: 2x2x2 times less time (fewer CPUs) in 2007 l Central estimate: 40 ms in 2007, i.e. 2,000 CPUs l Thus, basic estimate of 1,000 dual-CPU boxes in TDR l (Note: not an excess of CPU, e.g. no raw-data handling) u Start-up system of 50kHz (Level-1) and 105 Hz (HLT) can satisfy basic “discovery menu” l Some Standard Model physics left out; intend to do it, at lower luminosity and pre-scales as luminosity drops through fill –Examples: inclusion of B physics (can be done with high efficiency and low CPU cost; limitation is Level-1 bandwidth); details in TDR. Also low-mass dijet resonances. n Single-farm design works.

25. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger25 FAQ n What happens if we turn on and we only need 42kHz (i.e. safety factor is <3)? u We lower thresholds, add triggers,etc to use full bandwidth available n What happens if we turn on and we need 70 kHz? u The Level-1 trigger is programmable, it can, e.g. mask hot regions, etc etc. Requirement is to stay within 50 kHz. l Must look carefully at beam-gas etc n Can we add triggers? u All tables: just indications of type of combinations and requirements we can have on “day-1”. (Actually at a lumi of 2x10 33 cm -2 s -1 ) l Much will depend on the Tevatron, on when we turn on, on actual beam conditions, on actual event size, on actual DAQ system…

26. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger26 Summary n CMS HLT implemented on a farm of PCs u Farm design scales with CPU needs u Running offline quality selection code u As for DAQ, we have a working design, the specific implementation will follow needs & technology n HLT framework allows flexible and efficient algorithm implementation n DAQ TDR shows alpha version HLT trigger table u Certainly not the final thing, will be moving target anyway u Will follow input from HERA, Tevatron, theory,… My question to the offline community: Why not more than 100 Hz?

27. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger27 A parting thought With respect to offline analysis:

28. Vuko Brigljević, 2004 LHC Days in SplitThe CMS High Level Trigger28 So make sure it ends up in there! H