1. The CMS Level-1 Trigger: System and Operation Christian Hartl (CERN) on behalf of CMS SPG-ÖPG, Lausanne, 15 June 2011

2. In this talk 1. Basics 1. Basics 2. Triggering at CMS 2. Triggering at CMS 3. Level-1 Trigger System 3. Level-1 Trigger System 4. Operational Experience 4. Operational Experience

3. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne3 Before triggering Early experiments in particle physics observed phenomena of interest directly. Early experiments in particle physics observed phenomena of interest directly. counting experiments counting experiments no trigger no trigger Later experiments to spot rare events by eye Later experiments to spot rare events by eye e.g. bubble chamber pre-trigger to take snapshot in right moment (no events rejected online) trained analysts scanned up to 250k photos/month! omega-minus 1967 @ BNL Rutherford's Gold Foil Experiment 1909

4. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne4 Triggering for physics at LHC looking for extremely rare signals looking for extremely rare signals standard model physics = background e.g. 10 13 inelastic p-p collisions compared to one H SM  4µ can't record everything can't record everything must reject background must remain efficient for signal need to decide fast need to decide fast to avoid dead-time How? How? LHC design energy and luminosity:

5. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne5 Simple concepts electrical signals from detectors are combined to define trigger signal electrical signals from detectors are combined to define trigger signal "traditional approach" "traditional approach" trigger signal causes digitization and recording of data trigger signal causes digitization and recording of data no further trigger possible until data on tape (dead time penalty) no further trigger possible until data on tape (dead time penalty) NIM logic modules NIM logic modules O(10) operations per module AND OR NOT threshold delay electronic chips (FPGA, ASIC) electronic chips (FPGA, ASIC) O(10 4 ) logic operations per chip dead-time fraction = trigger rate x readout time back then today

6. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne6 Multi-level triggering – from LEP to LHC intermediate readout buffer(s) intermediate readout buffer(s) fast local readout, reduced dead-time after trigger fast local readout, reduced dead-time after trigger bunch crossing clock as pre-trigger in hadron colliders bunch crossing clock as pre-trigger in hadron colliders time between bunch collisions used to calculate trigger decision time between bunch collisions used to calculate trigger decision several trigger levels several trigger levels fast (low-latency) early level(s), advanced later level(s) fast (low-latency) early level(s), advanced later level(s) Pipelined trigger and readout Pipelined trigger and readout process several events simultaneously in a pipeline (triggering & readout) prevents long dead-time after accept in early trigger stage more channels  less occupancy more channels  less occupancy deal with pile-up at LHC luminosities several proton- proton interactions may pile up in a bunch crossing Aleph @ LEP already in LEP new in LHC needed if: time between bunch crossings << calculation and distribution of trigger decision

7. In this talk 1. Basics 1. Basics 2. Triggering at CMS 2. Triggering at CMS 3. Level-1 Trigger System 3. Level-1 Trigger System 4. Operational Experience 4. Operational Experience

8. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne8 CMS at LHC

9. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne9 CMS uses two trigger levels LHC design LHC design collisions p-p (Pb-Pb) collisions p-p (Pb-Pb) √s=14 TeV (Pb-Pb: 5.52 TeV) √s=14 TeV (Pb-Pb: 5.52 TeV) 40 MHz bunch collision rate 40 MHz bunch collision rate 25 ns spacing (LEP: 22 µs) 25 ns spacing (LEP: 22 µs) 2835/3564 slots filled 2835/3564 slots filled luminosity 10 Hz/nb luminosity 10 Hz/nb 10 11 p/bunch 10 11 p/bunch average pile-up = 18 events/bunch crossing average pile-up = 18 events/bunch crossing Level-1: 40 MHz  100 kHz Level-1: 40 MHz  100 kHz custom electronics based on FPGAs and ASICs massive parallel pipeline, fixed latency (~3.2 µs) decision based on coarse calorimetric and muon data High Level Trigger: 100 kHz  300 Hz High Level Trigger: 100 kHz  300 Hz CPU farm running algorithms processing time depends on event complexity (~40 ms) decision based on full detector data, seeding on Level-1 info 40 MHz 100 kHz 300 Hz to tape L1-HLT throughput: 100 GB/s data acquisition scheme

10. In this talk 1. Basics 1. Basics 2. Triggering at CMS 2. Triggering at CMS 3. Level-1 Trigger System 3. Level-1 Trigger System 4. Operational Experience 4. Operational Experience

11. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne11 Level-1 trigger approach channels used: channels used: calorimeter systems calorimeter systems muon detectors muon detectors beam monitoring systems beam monitoring systems tracker channels NOT used tracker channels NOT used construction of trigger objects: construction of trigger objects: "local"  "regional"  "global" applying cuts applying cuts only at global trigger level Level-1 Global Trigger: Level-1 Global Trigger: receives four best objects of each kind checks 128 programmable conditions: object multiplicity, transverse momentum & energy, position, quality, topology checks 64 technical trigger conditions: signals provided by sub-detectors combines conditions in final OR with mask and prescales Level-1 accept distributed by Trigger Control System according to detector readiness and trigger rules

12. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne12 Level-1 trigger architecture RPC = resistive plate chambers RPC = resistive plate chambers CSC = cathode strip chambers CSC = cathode strip chambers DT = drift tubes DT = drift tubes ECAL = lead-tungstate e/m calorimeter ECAL = lead-tungstate e/m calorimeter HCAL = brass- scintillator hardronic calorimeter HCAL = brass- scintillator hardronic calorimeter TTC = Timing, Trigger & Control system TTC = Timing, Trigger & Control system TTS = Trigger Throtting system TTS = Trigger Throtting system DAQ = data acquisition system DAQ = data acquisition system 0.9<|  |<2.4 4 µ 4+4 µ 4 µ MIP + ISO bits e, J, E T, H T, E T miss, H T miss Level-1 Accept 40 MHz pipeline Calorimeter Trigger ECAL Trigger Primitives HCAL/HF Trigger Primitives Regional Calorimeter Trigger Global Calorimeter Trigger Muon Trigger RPC hitsCSC hitsDT hits Segment finder Track finder Pattern Comparator Segment finder Track finder Global Muon Trigger Global Trigger TTC systemTTS system Detector Frontend Status Link system 32 partitions 0<|  |<5 |  |<3 |  |<1.2|  |<3

13. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne13 Custom-built Level-1 trigger hardware detector status (trigger throttling) Level-1 Accept, control signals calorimeter objects: e, J, E T, H T, E T miss, H T miss muon objects Regional Calorimeter Trigger Resistive Plate Chambers Pattern Comparator Cathode Strip Chambers Track Finder Drift Tube Track Finder Global Calorimeter Trigger Global Trigger Global Muon Trigger Timing Trigger & Control System Fast Merging Modules WisconsinWarsaw FloridaBologna Madrid Vienna Imperial College Vienna

14. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne14 Level-1 trigger objects match & merge match & merge barrel: DT-RPC endcap: CSC-RPC cancel duplicates cancel duplicates overlap region: DT-CSC sort by momentum and quality sort by momentum and quality jet algorithm (GCT) electron/photon algorithm (RCT) muon track algorithm (DT Track Finder) Global Muon Trigger algorithm:

15. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne15 Global Trigger, Trigger Control System Every bunch crossing, for which a potential Level-1 accept is inhibited, is counted as dead. Every bunch crossing, for which a potential Level-1 accept is inhibited, is counted as dead. needed to calculate recorded integrated luminosity needed to calculate recorded integrated luminosity CMS regional trigger GCT GMT 40 MHz pipeline calo & muon objects GLOBAL TRIGGER LOGIC 1. Logic OR 2. Prescale 3. Rate Counters 128x algos technical trigger signals (beam monitoring systems etc.) 64x technical GLOBAL TRIGGER FINAL DECISION LOGIC physics calib random randcalib TRIGGER CONTROL SYSTEM L1A candidate Level-1 Accept (via TTC system to detector) backpressure, trigger rules deadtime counters

16. Level-1 trigger software system Trigger Supervisor (TS) framework Trigger Supervisor (TS) framework based on XDAQ middleware based on XDAQ middleware distributed control & monitoring system distributed control & monitoring system hardware access hardware access basic unit = TS cell basic unit = TS cell main executive, >=1 per subsystem main executive, >=1 per subsystem deployed as linux service on machine connected to hardware deployed as linux service on machine connected to hardware Services provided by each TS cell Services provided by each TS cell hardware configuration hardware configuration interconnection testing interconnection testing monitoring monitoring alarming alarming logging logging database access database access inter-TS-cell communication inter-TS-cell communication AJAX-based web interfaces AJAX-based web interfaces Level-1 page Level-1 page monitor and start/stop processes central monitoring of important subsystem items speed-dial for important applications current configuration status and identifieres (keys) … Trigger Function Manager Trigger Function Manager interface adapter: Run Control – Level-1 online software example: configuration service (component diagram) courtesy of Marc Magrans

17. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne17 Level-1 trigger web applications Level-1 Page Central TS Cell GMT TS Cell GT TS Cell Trigger Supervisor applications (examples) Level-1 trigger Data Quality Monitoring (CMS Data Analysis Software – CMSSW) RCT TS Cell

18. In this talk 1. Basics 1. Basics 2. Triggering at CMS 2. Triggering at CMS 3. Level-1 Trigger System 3. Level-1 Trigger System 4. Operational Experience 4. Operational Experience

19. Good start in 2010 Optimized synchronization of Level-1 trigger systems  efficiency increase Optimized synchronization of Level-1 trigger systems  efficiency increase Trigger performance good in first 2010 run Trigger performance good in first 2010 run Level-1 and HLT trigger menus continuously adapted to LHC conditions Level-1 and HLT trigger menus continuously adapted to LHC conditions Below ~ 0.4 Hz/µb minimum-bias trigger simple threshold cuts no (little) pre-scaling Up to 200 Hz/µb (end of 2010) triggers based on physics objects controlling rates with prescales effects from pile-up, detector etc. example: DT trigger performance (from J/Psi) nominal trigger cut: 5 GeV example: RPC trigger w.r.t. mininum- bias trigger 43 pb -1 collected in 2010 (p-p) 348 bunches colliding 150 ns bunch spacing

20. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne20 2011 – Level-1 trigger improvements better muon algorithms better muon algorithms DT ghost busting DT ghost busting RPC pattern recognition RPC pattern recognition better energy resolution in calorimeter trigger better energy resolution in calorimeter trigger electron energy corrections electron energy corrections jet energy corrections jet energy corrections improved online software stability improved online software stability avoid operational downtime (many bottlenecks) avoid operational downtime (many bottlenecks) veto triggers that pre-fire (25 ns early, 5% of time) veto triggers that pre-fire (25 ns early, 5% of time) pre-firing kills collision events (trigger rules disallow two consecutivee triggers) pre-firing kills collision events (trigger rules disallow two consecutivee triggers) avoid this: veto on signal from BPTX (electrostatic beam-pickup), advanced by 25 ns avoid this: veto on signal from BPTX (electrostatic beam-pickup), advanced by 25 ns for example:

21. Christian HartlThe CMS Level-1 Trigger @ SPG-ÖPG 2011, Lausanne21 2011 – experience and outlook Very good performance of Level-1 & HLT Very good performance of Level-1 & HLT smooth & efficient operations smooth & efficient operations > 850 pb -1 (p-p @ 7 TeV) recorded > 850 pb -1 (p-p @ 7 TeV) recorded continous improvements continous improvements excellent work in hardware, software, analysis, trigger menu development excellent work in hardware, software, analysis, trigger menu development Steep increase in luminosity Steep increase in luminosity peak lumi > 1.25 Hz/nb (1042 bunches colliding) peak lumi > 1.25 Hz/nb (1042 bunches colliding) current menu is for 1.4 Hz/nb current menu is for 1.4 Hz/nb possibly reach 5 Hz/nb in 2011 possibly reach 5 Hz/nb in 2011 best wishes to the LHC! best wishes to the LHC! Level-1 and HLT trigger menus are steadily evolving. physics prioritiesluminosity physics resultscontrol rates

22. Thank you