1. Compact Muon Solenoid Christian Hartl CERN Austrian Doctoral Student June 2009 Institut für Hochenergiephysik, ÖAW CMS Global Trigger Control and Monitoring Alexander Winkler Anton Taurok Barbara Neuherz Bernhard Arnold Christian Hartl Claudia-Elisabeth Wulz Herbert Bergauer Herbert Rohringer Ivan Mikulec Josef Strauss Kurt Kastner Manfred Jeitler Markus Eichberger Michael Padrta Philipp Glaser Philipp Wagner Thomas Schreiner Thomas Themel Vasile Mihai Ghete

2. 2 Outline 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

3. 3 Outline : Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

4. 4 CMS @ LHC: High Luminosity : Need Event Filtering CMS designed for high luminosity 10 34 cm -1 s -1 –proton bunches collide every 25 ns –bunches contain ~10 15 protons –collision energy √s = 2 x 7 TeV need to discern –huge background –small fraction of physics of interest recapitulation

5. 5 CMS @ LHC: Two Step Filter: L1 and HLT Level 1 Trigger –hardware (FPGAs / ASICs) –pipelined, 40 MHz –reduce event rate from ~1GHz to 100 kHz Higher Level Trigger –software (computer farm) –reduce event rate from 100 kHz to 100 Hz recapitulation

6. 6 CMS Global Trigger: Final Level 1 Filter Logic menu of trigger algorithms –parallel calculation of >100 algorithms complex combinations of simple conditions –based on CMS trigger objects muons (momentum + coord.) electrons and jets (energy + coord.) energy sums and missing energy only selected algorithms contribute –trigger candidate = logic OR GTL Global Trigger Logic FDL Final Decision Logic recapitulation

7. 7 CMS Global Trigger: Trigger Decision Distribution Does every trigger candidate pass? No… Trigger Control System decides: –every beam crossing (40 MHz) candidate triggers too close? any CMS system not ready? After a Level 1 Accept? –all systems send data to DAQ (data acquisition) Global Trigger also sends: –muon & calorimeter data; decision info DEAD- TIME TCS Trigger Control System GTFE Global Trigger Front End recapitulation

8. 8 Outline: Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

9. 9 Global Trigger Online Software: Main Tasks: Configuration & Monitoring Why "online"? –it's for data taking hardware configuration –put hardware chips into well defined state e.g. trigger menu parameters –configuration data from CMS database hardware monitoring –system parameters (e.g. detector input status) –physics parameters (trigger rates & deadtime) architecture ?OK

10. 10 Global Trigger Online Software: GT Cell (Trigger Supervisor Application) GT Cell is a web application –C++ web service (XDAQ = CMS standard) –node in L1 "Trigger Supervisor" application tree GT Cell is running as a Linux service 24/7 … on a machine in "service cavern" (CMS network) –PCI link to hardware (CAEN VME interface) –configuration & monitoring service –netw. communication with other L1 applications –netw. access to CMS database architecture

11. 11 Global Trigger Online Software: GT Cell in Distributed Control System CMS Run Control Regional Calorimeter Trigger Global Calorimeter Trigger Resistive Plate Chamber Trigger Drift Tube Track Finder Cathode Strip Chamber Track Finder Tracker, HCAL, ECAL, DT, RPC, CSC... Global Muon Trigger Clock & Signal Distribution (TTC) Global Trigger Detector Applications Central Trigger Cell L1 Trigger Function Manager Level 1 Trigger Applications DAQ, DQM, HLT… Data Acquisition, Filtering, Monitoring Applications Global Trigger Global Muon Trigger … Detector Trigger Hardware architecture

12. 12 Global Trigger Online Software: GT/GMT Software Architecture Trigger Supervisor Framework commands, operations, control panels database access, cell communication XDAQ Framework lightweight C++ web applications generic access to hardware and database inter-application network messaging GT Libraries access to GT module functions Scientific Linux CERN operating system used by CMS GT/GMT Test Cell Trigger Menu Cell GT Cell GT/GMT Test Applications CMS Private Computer Network CMS Database GMT Cell Ph. Wagner Th. Themel B. Arnold T. Nöbauer I. Mikulec M. Magrans Ph. Wagner G. Kasieczka Ch. Hartl Th. Themel B. Arnold H. Sakulin B. Neuherz J. Strauss B. Arnold M. Jeitler Ph. Glaser A. Winkler Ch. Hartl B. Arnold architecture

13. 13 Outline: Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

14. 14 Global Trigger Database: Bookkeeping for Hardware Setups complete hardware setups stored in database this allows for: –deterministic hardware configuration: full Level 1 trigger configuration defined in advance –bookkeeping: which setup used in which CMS run? –synchronisation of online and offline software: emulation software setup must match hardware setup –debugging: reproduce hardware conditions to debug problems database

15. 15 Global Trigger Database: Hardware Setup: What must be stored? Trigger Logic parameters –selection of trigger algorithms from menu which algorithms allowed to cause trigger? –physics parameters of trigger conditions e.g. energy or momentum thresholds –down-scaling of trigger algorithms consider, say, every 100 th trigger candidate only Hardware Timing –alignment of trigger inputs –readout logic delays database This is a glimpse. The complete hardware setup must be defined in the database.

16. 16 Global Trigger Database: Hardware Setup Data Organization structured information for all GT modules decoded items (human-interpretable names) TSC TSP L1T Menu Key GT key database

17. 17 Global Trigger Database: Frequently Changing: "Run Settings" fine-tuned setup: modifiable ad hoc on shift –"GT Run Settings" Database database

18. 18 Global Trigger Database: "GT Run Settings" Database Update GT Run Settings GUI: –update "current Run Settings" in the database i.e. those which are to be used in the next run database trigger sources mix selection of algorithms from Level 1 Menu down-scaling of algorithm pre- decisions 1 2a 2b 2c 3

19. 19 Global Trigger Database: "GT Run Settings": Selection of Algos database

20. 20 Global Trigger Database: Level 1 Trigger Menu Versions trigger menu implemented in firmware –Global Trigger Logic condition chips: main characteristics: firmware version threshold definition: register content (set by software) Level 1 trigger menu versions bookkeeping –crucial to analysis of CMS data strict versioning enforced in database difference between two versions? –only implementation differs? e.g. trigger objects rescaling –versions physically different? new firmware? threshold change? database

21. 21 Global Trigger Database: Level 1 Trigger Menu Data Organization database L1T Menu DB populated by Trigger Menu Editor

22. 22 Outline: Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

23. 23 Global Trigger Configuration: Configure and Start the Run… data taking with cosmic muons (Global Runs)… CMS systems coordinated by Run Control: configure 1.GT: hardware setup 2.GT: periodic orbit signal (BC0) to CMS systems enable 1.GT: start signal to CMS systems 2.GT: trigger sources ON (physics, random, calibration) stop 1.GT: trigger sources OFF 2.GT: stop signal to CMS systems configured enabled halted suspended configure enable suspend stop stop' enable' configuration

24. 24 Global Trigger Configuration: What Data to Send after Level 1 Accept? Level 1 Accept to the detector: –GT sends data to Data Acquisition: input data from Global Muon and Calorimeter Trigger decision word: which algorithms caused the L1A? –crucial for Level 1 "seeding" of Higher Level Trigger Global Trigger readout configuration –what data is sent, what not? –send how big a window around the time of L1A? 3·25 ns or 5·25 ns configuration

25. 25 Global Trigger Configuration: No Changes While Running physics data require stable trigger setup –no trigger configuration changes while running –exception 1: pre-scaling of trigger algorithms: luminosity drops while running  trigger rates decrease  decrease pre-scaling (in steps) to compensate –using pre-defined pre-scaling sets (set index 0, 1, 2, …) –no information lost: event data contain set index –exception 2: random trigger rate (if "randoms on") –no information lost: event data show dedicated ID configuration

26. 26 Global Trigger Configuration: Operation Transitions in More Detail additional transitions: (launched by Run Control) cold-reset (halted-halted) –reload firmware into chips from PROMs (necessary when LHC clock changes) repartition (configured-configured) –define which detector partitions (there are 32) should be serviced by the Trigger Control system change prescaling (enabled-enabled) –to compensate for decreasing luminosity configured enabled halted suspended configure enable suspend stop stop' enable' cold-reset repartition change prescaling configuration

27. 27 Global Trigger Configuration: Service Eight Detector Partition Groups configured enabled halted suspended configure enable suspend stop stop' enable' cold-reset repartition change prescaling 0 configured enabled halted suspended configure enable suspend stop stop' enable' cold-reset repartition change prescaling 7 … Up to eight runs can be serviced by GT in parallel/independently. e.g.: calorimeter systems (ECAL, HCAL, RCT, GCT) e.g.: muon systems (DT, DTTF, CSC, CSCTF) configuration

28. 28 Global Trigger Configuration: From Run Control to Global Trigger Crate Communication Flow: Run Control defines: which detector parts are read out? –Global Trigger reacts in repartition transition assign detector partition to due GT partition (0…7) enable detector partition to receive triggers and signals listen and react to detector partition's status information CMS Run Control Level 1 Trigger Function Manager Central Cell Global Trigger Cell Global Trigger Hardware Level1 Trigger Online Software Run Control Software Framework (Java) Trigger Supervisor / XDAQ Framework (C++) configuration

29. 29 Global Trigger Configuration: HardwareFunction C++ Class translation database – hardware: class HardwareFunction defines mapping: –in central place (documentation!) –"automatic" configuration database field names are HardwareFunction names configure by call to HardwareFunction(itemName)::set(value) configuration database field: TCS.p0.PHYSICS_TRIGGER_EN (true / false) Trigger Control System, partition 0: physics triggers enabled? hardware register bits: TCS.PTC0_CMD_REG, bit 12 (1 or 0) register bit for disabling of physics triggers for partition 0. C++ class HardwareFunction

30. 30 Global Trigger Configuration: Database & Hardware Configuration Editor To browse through a given Global Trigger setup in the CMS database –what's in the GT key? To compare with the current hardware configuration –change hardware settings ad hoc for testing –online documentation of database items (and correspondence to hardware) To create a new Global Trigger setup –make new GT key (modify existing one) configuration

31. 31 Global Trigger Configuration: Database & Hardware Configuration Editor configuration Step 1: Select GT setup from database –or key from any other Level 1 Trigger subsystem works for all Level 1 Trigger subsystems click to enter navigator list of available GT setup keys 1 2 3

32. 32 Global Trigger Configuration: Database & Hardware Configuration Editor configuration Step 2: Navigate through GT setup in DB –modify configuration (in editor, not yet in database) online documention of database items: "what hardware register bits does it correspond to?" tree navigation click to store modifications in database (enter preview) sub-node link navigation modify item orange: editor item modified red: hardware value differs make editable, enter name 1 2 3 4

33. 33 Global Trigger Configuration: Database & Hardware Configuration Editor configuration Step 3: Preview & insert new setup ("key") in database double-check: is that what you want to add in the database? add setup in database 1 2

34. 34 Outline: Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

35. 35 Global Trigger Monitoring: Hardware, Trigger, Configuration, Inputs what's the trigger setup? –current trigger sources and selected algorithms what's the configuration status? –configured? which key? running? how is the trigger performance? –trigger rates? which algorithms contribute most? –deadtime? what are the reasons? what's the detector input and GT's status? –status of GT boards and readout? detector inputs? monitoring

36. 36 Global Trigger Monitoring: Monitoring Data Flow monitoring CMS Database Monitoring Collector "SCAL" Application GT Cell PageWBM Page Web Based Monitoring (web server) CMS private network CMS Level 1 Trigger Global Trigger Trigger Function Manager GT Cell service L1 Page Linux File System L1 Page server

37. 37 Global Trigger Monitoring: Output Trigger Rate History GT output trigger rate (from event number) –rate = Δ (number of L1As) / Δ t –live update every 5 s (L1 Page) Cosmics Run @ Zero Tesla (CRUZET) high rate test: 90 kHz (mainly artificial "random" triggers) monitoring

38. 38 Global Trigger Monitoring: Trigger Rates and Deadtime instantaneous trigger (L1A) rate generated triggers (L1As) run number time since start of run (in luminosity segments, ~93 s) rates of individual trigger candidates rate history of generated trigger candidates rate history of lost trigger candidates rate history of physics L1As rate history of random L1As rate history of test L1As rate history of calibration L1As history of generated deadtime (in %) - due to trigger rules - due to calibration cycles - due to detector/DAQ backpressure luminosity segment (x 93 s):164163162 GT Cell Trigger Monitor Panel … monitoring

39. 39 Global Trigger Monitoring: Detector Input Status and GT Status GT Cell Detector & GT Status Monitor Panel monitoring ECAL partitions: BUSY 1 HCAL partition: READY 2 HCAL partitions: OUT OF SYNCH RPC partition: not in readout (not yellow) status is ignored 2 TRK partitions: ERROR GT boards and readout: READY 2 CSC partitions: WARNING Output Status of TCS partition 0 = IDLE (partition 0 not running) Combined Input Status to TCS = ERROR (here: because of ERROR in CSC system) Monitoring Panel to spot quickly problems in GT or detector while taking data.

40. 40 Global Trigger Monitoring: Trigger Sources, Key, Enabled Algos monitoring GT Cell Configuration Panel GT (TCS partition 0) is configured physics triggers enabled (no random or calibration triggers) "algorithms" L1_SingleJet6 and L1_SingleJet10 are enabled (will forward trigger candidates to the Trigger Control System)

41. 41 Outline: Next: 1.Global Trigger (GT) in the Context of CMS 2.GT Software Environment & Architecture 3.GT Database 4.GT Configuration 5.GT Monitoring 6.GT Online Operation

42. 42 Global Trigger Online Operation: Trigger Shifter Tasks before each CMS run: –GT basic configuration defined by GT key as set in Level 1 Trigger Key GUI –GT ad hoc settings defined by GT Run Settings as set in GT Run Settings GUI while CMS is taking data: –monitoring of rates, deadtime, detector status Resync or HardReset in case of detector problem –change of random trigger rate (high rate tests) –change of algorithm decision prescaling operation

43. 43 CMS Global Trigger Control and Monitoring Thank you! more: Global Trigger official web page: http://wwwhephy.oeaw.ac.at/p3w/electronic1/GlobalTrigger/GlobalTriggerCrate.htm collection of wiki articles: https://twiki.cern.ch/twiki/bin/view/CMS/GlobalTrigger