1. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 1 The Level-1 Trigger of the CMS Experiment at LHC Design, challenges and performance Manfred Jeitler MGU, 11 July 2011 Московский государственный университет Institute of High Energy Physics of the Austrian Academy of Sciences`

2. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 2 The Large Hadron Collider

3. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 3 LHC p-p Collisions Operations 2010-2011 n 2010: √s = 7 TeV –Peak Instantaneous Luminosity: > 2×10 32 cm -2 s -1 n 2011: √s = 7 TeV –Peak Instantaneous Luminosity: > 1.25 ×10 33 cm -2 s -1 –And increasing! –Current Maximum of ~1400 bunches/beam in LHC –Possible to gain another factor of 4 2808

4. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 4 CMS Detector

5. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 5 n first particle physics experiments needed no trigger n were looking for most frequent events n people observed all events and then saw which of them occurred at which frequency what is a trigger?

6. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 6 n later physicists started to look for rare events –“frequent” events were known already n searching “good” events among thousands of “background” events was partly done by auxiliary staff –“scanning girls” for bubble chamber photographs what is a trigger?

7. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 7 Получше — в горшочек, похуже — в зобочек!

8. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 8 Higgs -> 4  +30 MinBias

9. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 9 find the needle!

10. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 10 n due to the extremely small cross sections of processes now under investigation it is impossible to check all events “by hand” –~ 10 13 background events to one signal event n it would not even be possible to record all data in computer memories n we need a fast, automated decision (“trigger”) if an event is to be recorded or not X-section  

11. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 11 n detectors yielding electrical output signals allow to select events to be recorded by electronic devices –thresholds (discriminators) –logical combinations (AND, OR, NOT) –delays –available in commercial “modules” –connections by cables (“LEMO” cables)

12. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 12 n because of the enormous amounts of data at major modern experiments electronic processing by such individual modules is impractical –too big –too expensive –too error-prone –too long signal propagation times n  use custom-made highly integrated electronic components (“chips”) 400 x  1 x ~ 10 logical operations / module  ~ 40000 logical operations in one chip

13. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 13 example: trigger logic of the L1-trigger of the CMS experiment

14. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 14 When do we trigger ? n „bunch” structure of the LHC collider –„bunches” of particles –40 MHz »a bunch arrives every 25 ns »bunches are spaced at 7.5 meters from each other »bunch spacing of 125 ns for heavy-ion operation n at nominal luminosity of the LHC collider (10 34 cm -2 s -1 ) one expects about 20 proton-proton interactions for each collision of two bunches for ATLAS and CMS –only a small fraction of these “bunch crossings” contains at least one collision event which is potentially interesting for searching for “new physics” –in this case all information for this bunch crossing is recorded for subsequent data analysis and background suppression

15. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 15 Event size (bytes) trigger: first level high level ATLAS, CMS40 MHz  100 kHz  200 Hz LHCb40 MHz  1 MHz  2 kHz ALICE10 kHz  1 kHz  100 Hz

16. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 16 How do we trigger ? n use as much information about the event as possible –allows for the best separation of signal and background –ideal case: “complete analysis” using all the data supplied by the detector n problem: at a rate of 40 MHz it is impossible to read out all detector data –(at sensible cost) n have to take preliminary decision based on part of the event data only n be quick –in case of positive trigger decision all detector data must still be available –the data are stored temporarily in a “pipeline” in the detector electronics »“short term memory” of the detector »“ring buffer” »in hardware, can only afford a few μs n how to reconcile these contradictory requirements ? write read

17. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 17  multi-level trigger n first stage takes preliminary decision based on part of the data –rate is already strongly reduced at this stage –~1 GHz of events (= 40 MHz bunch crossings)  ~100 kHz –only for these bunch crossings are all the detector data read out of the pipelines –still it would not be possible (with reasonable effort and cost) to write all these data to tape for subsequent analysis and permanent storage n the second stage can use all detector data and perform a “complete analysis” of events –further reduction of rate: ~100 kHz  ~100 Hz –only the events thus selected (twice filtered) are permanently recorded 40 MHz 100 kHz 300 Hz to tape

18. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 18 How does the trigger actually select events ? n the first trigger stage has to process a limited amount of data within a very short time –relatively simple algorithms –special electronic components »ASICs (Application Specific Integrated Circuits) »FPGAs (Field Programmable Gate Arrays) –something in between “hardware” and “software”: “firmware” »written in programming language (“VHDL”) and compiled »fast (uses always same number of clock cycles) »can be modified at any time when using FPGAs n the second stage (“High-Level Trigger”) has to use complex algorithms –not time-critical any more (all detector data have already been retrieved) –uses a “computer farm” (large number of PCs) –programmed in high-level language (C++)

19. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 19 How does the trigger actually select events ? n the first trigger stage has to process a limited amount of data within a very short time –relatively simple algorithms –special electronic components »ASICs (Application Specific Integrated Circuits) »FPGAs (Field Programmable Gate Arrays) –something in between “hardware” and “software”: “firmware” »written in programming language (“VHDL”) and compiled »fast (uses always same number of clock cycles) »can be modified at any time when using FPGAs n the second stage (“High-Level Trigger”) has to use complex algorithms –not time-critical any more (all detector data have already been retrieved) –uses a “computer farm” (large number of PCs) –programmed in high-level language (C++)

20. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 20 signals used by the first-level trigger n muons –tracks –several types of detectors (different requirements for barrel and endcaps): –in ATLAS: »RPC (Resistive Plate Chambers):barrel »TGC (“Thin Gap Chambers”):endcaps »not in trigger: MDT (“Monitored Drift Tubes”) –in CMS: »DT (Drift Tubes):barrel »CSC (Cathode Strip Chambers): endcaps »RPC (Resistive Plate Chambers):barrel + endcaps n calorimeters –clusters –electrons, jets, transverse energy, missing transverse energy –electromagnetic calorimeter –hadron calorimeter n only in high-level trigger: tracker detectors –silicon strip and pixel detectors, in ATLAS also straw tubes –cannot be read out quickly enough

21. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 21 how to find muon tracks in the CMS solenoidal field? (Drift Tubes)

22. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 22 RPC Comparator Pattern matching barrel: 3 layers/4 or 4/6 endcaps: 3/3 Pattern matching barrel: 3 layers/4 or 4/6 endcaps: 3/3 how to find muon tracks in the CMS solenoidal field? (Resistive Plate Chambers)

23. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 23 Muon Trigger 3 muon detectors to |  |<2.4 Drift Tubes Track Segment ID and Track Finder Cathode Strip Chambers Track Segment ID and Track Finder Resistive Plate Chambers Pattern Matching 4 candidates per subsystem to Global Muon Trigger Global Muon Trigger sorts, removes duplicates, 4 top candidates to Global Trigger Track building at 40 MHz

24. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 24 Global Muon Trigger match & merge barrel: DT-RPC endcap: CSC-RPC cancel duplicates overlap region: DT-CSC sort by momentum and quality

25. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 25 calorimeter system Hadronic calorimeter (HCAL) Electromagnetic (ECAL) HF Hadron Forward Calorimeter

26. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 26 calorimeter trigger

27. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 27 Photon and electron trigger objects L1 e/gam L1 e/gam HLT e/gam HLT e/gam n L2 Step Spatial matching of ECAL clusters with e/γ candidates at L1 Superclusters are formed ET cut applied Calorimetric (ECAL+HCAL) isolation L3 Photons Tight track isolation L3 Electrons Electron track reconstruction Spatial matching of ECAL cluster and pixel track Loose track isolation in a “hollow” cone

28. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 28 e/  and Jet Algorithms 4x4 Tower sums from RCT to GCT Jet or  E T 12x12 trig. tower  E T sliding in 4x4 steps w/central 4x4 E T > others  : isolated narrow energy deposits Energy spread outside  veto pattern sets veto Jet   if all 9 4x4 region  vetoes off GCT uses Tower sums used for E T,ME T jets for H T, MH T Electron (Hit Tower + Max) 2-tower  E T + Hit tower H/E Hit tower 2x5-crystal strips >90% of E T in 5x5 (Fine Grain) Isolated Electron (3x3 Tower) Quiet neighbors: all towers pass Fine Grain & H/E One “L” of 5 EM E T < Thr.

29. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 29 Global Trigger Receives Trigger Objects: 4 forward and 4 central jets, 4  -jets, 4 isolated and 4 non-isolated e/ , total E T, missing E T, H T and position information from GCT 4  with position, sign, and quality information from GMT 128 different conditions (thresholds, topological cuts) can be combined to make 128 physics triggers Forwards Level-1 Accept Trigger, Timing and Control (TTC) system for read-out of the detector front-end electronics

30. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 30 Global Trigger and Trigger Control System n Every bunch crossing for which a potential Level-1 accept is inhibited is counted as “dead”. –needed to calculate recorded integrated luminosity CMS regional triggers GCT GMT 40 MHz pipeline calo & muon objects GLOBAL TRIGGER LOGIC 1. Logic OR 2. Prescale 3. Rate Counters 128x algos technical trigger signals (beam monitoring systems etc.) 64x technical GLOBAL TRIGGER FINAL DECISION LOGIC physics calib random randcalib TRIGGER CONTROL SYSTEM L1A candidate Level-1 Accept (via TTC system to detector) backpressure, trigger rules deadtime counters

31. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 31 L1 Trigger Custom Hardware –Hundreds of boards –Thousands of: »ASICs »FPGAs »Copper Cables »Optical Fibers »(Wo)man hours Global Muon Trigger & Global Trigger (Vienna) RCT (Wisconsin) CSCTF (Florida) DTTF (Vienna) GCT (Imperial) RPC PaC (Warsaw)

32. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 32

33. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 33 The CMS Level-1 Trigger n Only calorimeter and muon systems participate in CMS L1 0.9<|  |<2.4 4  4+4  4  MIP+ ISO bits e, J, E T, H T, E T miss L1A 40 MHz pipeline Calorimeter Trigger ECAL Trigger Primitives ECAL Trigger Primitives HCAL/HF Trigger Primitives HCAL/HF Trigger Primitives Regional Calorimeter Trigger Regional Calorimeter Trigger Global Calorimeter Trigger Global Calorimeter Trigger Muon Trigger RPC hits CSC hits DT hits Segment finder Track finder Track finder Pattern Comparator Pattern Comparator Segment finder Track finder Track finder Global Muon Trigger Global Trigger TTC system TTS system Detector Frontend Status Link system 32 partitions CMS experiment 0<|  |<5 |  |<3 |  |<1.2|  |<3

34. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 34 Global Trigger interface

35. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 35 ATLAS+CMS: what’s the difference ? n similar task n similar conditions n similar technology

36. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 36 ATLAS+CMS: what’s the difference ? n similar task n similar conditions n similar technology

37. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 37 ATLAS+CMS: what is common ? n same physics objectives n same input rate –40 MHz bunch crossing frequency n similar rate after Level-1 trigger –50.. 100 kHz n similar final event rate –100.. 200 Hz to tape n similar allowed latency –pipeline length –within this time, Level-1 trigger decision must be taken and detectors must be read out –~ 3 μs »2.5 μs for ATLAS, 3.2 μs for CMS

38. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 38 ATLAS+CMS: what is different ? n different magnetic field –toroidal field in ATLAS (plus central solenoid) »get track momentum from η (pseudorapidity) –solenoidal field only in CMS »get track momentum from φ (azimuth) n number of trigger stages –two stages (“Level-1” and “High-Level Trigger”) in CMS –ATLAS has intermediate stage between the two: “Level-2” »Level-2 receives “Regions of Interest (RoI)” information from Level-1 »takes a closer look at these regions »reduces rate to 3.5 kHz »allows to reduce data traffic

39. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 39 n data are stored in “pipelines” for a few microseconds –e.g. in CMS: 3.2  s = 128 clock cycles at 40 MHz –question of cost –decision must never take longer! (must be synchronous!) –  no iterative algorithms n decision based on “look-up tables” –all possible cases are provided for –OR, AND, NOT can also be represented in this way ATLAS+CMS: principle of the first-level trigger

40. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 40 principle of event selection: ATLAS vs. CMS CMS: n no “cuts” at individual low-level trigger systems –only look for muons, electrons, jets, choose the “best” of these candidate objects and forward to Level-1 Global trigger n so, all possible kinds of events may be combined n selection is only made by “Level- 1 Global Trigger” –trigger information from all muon detectors and calorimeter systems available: completely flexible n High-Level trigger analyzes complete detector data for the whole detector ATLAS: n Level-1 trigger delivers numbers of candidate objects –muon tracks, electron candidates, jets over appropriate threshold –no topological information used (no angular correlation between different objects) n Level-2 trigger carries out detailed analysis for “Regions of Interest” –using complete detector information for these regions n High-Level trigger analyzes complete detector data for the whole detector

41. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 41 First Collisions n Tuesday March 30, 2010 n 12:58 n L~10 27 cm -2 s -1 n ~ 60 Hz Collision Rate n Time to optimize the trigger synchronization!

42. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 42 Minimum Bias trigger detectors HF – Hadron Forward Calorimeter BSC – Beam Scintillator Counters (hit and coincidence rate, MIP detection eff >96%) BPTX – Beam Pick-up Timing for the eXperiments monitors (precise info about bunch structure and timing of the incoming beam, res. 0.2 ns)

43. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 43 2010 Commissioning: Synchronizing the Trigger n Collision products take longer to get to outer part of detector –e.g. to reach CSC takes longer than ECAL n Take into account many different cable lengths –Even within a sub-detector n First runs with only a Minimum-Bias Trigger –Take data with a number of delays –Find the best alignment between subsystem (CSC shown) and Minimum-Bias Trigger –Repeated for each Trigger subsystem –Cross-checks available in DQM

44. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 44 LHC is design to deliver one bunch crossing (BX) every 25 ns Trigger has to provide a correct BX assignment Optimization of the trigger synchronization allows for an overall efficiency increase Example : L1 RPC trigger synchronization is in a very good shape no pre-triggers TRIGGERS RPC Hits BX=0 : 99.999% 2010 Commissioning: Synchronizing the Trigger

45. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 45 2010 Operational Challenge: Rates n Final month of 2010 p-p running, max inst. lumi increased by > O(4)! –No longer relying on Minimum Bias but a full menu of physics object triggers –L1 works with the HLT to maintain a sustainable HLT stored data rate »Up to 65 kHz in, ~400 Hz out –New L1 & HLT configurations and menus: »Higher thresh. L1 Triggers for HLT seeds »Prescaling at L1 and HLT to reduce rates »Several prescale “sets” to allow as much data to tape as possible Lumi decreases -> change set of prescales, keep rate about the same –Possible to predict output rates with MC –Must take into account other factors »Cosmic backgrounds, pile-up effects, detector effects, etc.

46. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 46 n Threshold efficiency – probability that for reconstructed muon with momentum P T, track finder assigns momentum greater or equal to the threshold value Eta region 1.2 <|η|<2.1 CSC trigger performance DT trigger performance DTTF PT > 10 GeV/c from J/Psi “Turn-on” expected to reach > 90% efficiency at the nominal cut Eta region |η|< 1.1 2010 Trigger Efficiencies

47. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 47 2011: Operations: Controlling the Rates LHC has been steadily increasing instantaneous luminosity delivered to the experiments in 2011 –Well beyond what was delivered in 2010 –How do we best control rates while keeping as much physics as possible? Just a few examples of what the L1 Trigger can do: –e/  triggers »Energy corrections to improve resolution »ECAL Spike Killing »Isolation and H/E for e/  candidates –Jet triggers »Improve the resolution with jet energy corrections –Muon triggers »Improve ghost busting and pattern recognition –Continue to develop the L1-HLT menus with prescale sets »Must anticipate physics groups’ needs as conditions change

48. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 48 2011: Operations: Avoiding pre-triggers n Problem: –Triggers are pre-firing at about ~5% of the time –Forward hadron calorimeter n Can we avoid this source of inefficiency? –CMS BPTX_AND Trigger »AND of two precision electrostatic beam pick-ups »Provides colliding bunch structure in CMS for every fill –Copy, reduce delay by one bunch-crossing unit (25 ns) and shorten to ~20ns –Use it as veto on pre-triggers –But it vetoes the slow particle (HSCP) trigger (BX+1) when the bunch spacing is 50 ns! n How to fix this? (Next Slide) before after

49. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 49 2011 Operations: Trigger on HSCPs n Heavy Stable Charged Particles (HSCPs) take as much as 2 BX the detector –Look for late signal in muon triggers –But preBPTX veto during 50 ns bunch spacing vetoes these! n RPC Trigger can extend detector signal to 2 BX –Reduce RPC delay into GMT by 1 BX –preBPTX will veto early in-time muons –In-time muons still captured with collision BX –HSCPs aligned with collision BX –Create special HLT path to keep slow particles (the BXid will be +1 wrt to Tracker, for example). n HSCPs Triggering! layer 6 layer 5 layer 4 Chamber hitsextended hits BX Muon candidate In-time muon layer 3 layer 2 layer 1 BPTX layer 6 layer 5 layer 4 Chamber hitsExtended hits BX Muon candidate HSCP layer 3 layer 2 layer 1 BPTX

50. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 50 Halfway through 2011 Middle of 2011 p-p run: n 323 pb -1 data taken n Maximum sustained L1 rate of ~65 kHZ –912b – 874 colliding at CMS, 50ns spacing – XX pb -1 recorded in a single fill

51. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 51 CMS Trigger & DAQ Systems  LHC beam crossing rate is 40 MHz & at full Luminosity of 10 34 cm -2 s -1 yields 10 9 collisions/s  Reduce to 100 kHz output to High Level Trigger and keep high-P T physics  Pipelined at 40 MHz for dead time free operation  Latency of only 3.2  sec for collection, decision, propagation

52. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 52... and where do we go from here? n upgrading the LHC to Super-LHC makes sense only if trigger systems are upgraded at the same time n ATLAS and CMS will have to use their trackers in the first-level trigger n but this is easier said than done n probably, computers will get faster and more (all?) trigger processing will be done in software

53. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 53 Conclusions n CMS L1 Trigger is performing well –2010 and 2011 not without its challenges –Complex system, in Hardware and Software »Excellent pool of experts »Problems addressed in a timely manner n LHC still increasing the luminosity –Will still stress system, both detectors and triggers »Must remain diligent, even if luminosity stabilizes –Expect L1 Menus to continue evolving »Physics results may dictate changes instead of lumi –Challenge to balance physics needs with rate limitations, »L1T must continue to work with HLT n Looking forward to lots of interesting physics results!

54. Manfred Jeitler, HEPHY Vienna CMS Level-1 Trigger MGU, 11 July 2011 54 Спасибо за приглашение! manfred.jeitler@cern.ch http://wwwhephy.oeaw.ac.at/u3w/j/jeitler/www