1. Overview of the LHC cryogenics control system and available data for RAMS FCC studies Benjamin Bradu, CERN, EN-ICE 1 September 2015

2. CRYO RAMS STUDIES Contents LHC Cryogenic control system overview LHC cryogenic control hierarchy (UNICOS) Data filtering and archiving Available data in Logging DB Cryo start, Cryo Maintain and Cryo Beep CRYO RAMS STUDIES

3. LHC cryogenic overview Cryogenic plant Cryogenic point 3.3 km LHC tunnel (27 km) (Transfer line) Long Straight Section Courtesy of E. Blanco CRYO RAMS STUDIES

4. LHC refrigeration plant B (3.3 km) Storage Interconnection Box Sector (Magnets) Sector Refrigeration Plant Refrigerator @ 4.5 K Refrigerator @ 1.8 K QSRB QSCB QUI QURCB QSDN QSCCB QURCBCC QLSS QARC CRYO RAMS STUDIES QSDH QSAB QSV

5. CRYO RAMS STUDIES PLC and data servers for LHC cryogenics  80 PLC for 52 000 I/O and 4700 PID control loops for LHC CRYO RAMS STUDIES

6. RM 78 Sector 78 (3.3 Km) Cryogenic refrigerator architecture 6 LHCA QURA LHCCA QURCA QSCCA LHCCB QSCCB LHCB QSRB QSCB QUI QSDN QSAA Comp 4.5K Comp 1.8K Main Dryer Comp 1.8K Comp 4.5K QURCB Cold Box 4.5K LN2 Buffer CB 1.8K Connection Box UCB 4.5K QSRA QSKA QSCA QSAB Main Dryer Local & Central Control Rooms SCADA Data Servers RM 81 Alcoves Sector 81 (3.3 Km) Tunnel Cavern Surface Shaft QSDN RM PA Profibus DP WorldFIP Return Module S78 & S81 Courtesy of E. Blanco

7. CRYO RAMS STUDIES Cryogenic tunnel architecture 7 LSS ARC RadTol electronics LHC Tunnel (3.3 Km) Protected areas PA Radiation areas PA shaft (~100 m) Tunnel Alcoves TT CV Local & Central Control Rooms SCADA Data Servers UNICOS PLCs Ethernet (TN) DP Profibus DP FESA FECs WorldFIP TT, PT, LT, EH, DI Ehsp, LTen Courtesy of E. Blanco CRYO RAMS STUDIES

8. UNICOS process decomposition  UNICOS is a CERN control standard framework  UNICOS is based on a hierarchical decomposition of the process  The orders and the alarms follow this hierarchy  The hierarchy helps to understand dependencies between systems and their interactions 8

9. CRYO RAMS STUDIES LHC Cryogenics hierarchies 9 LHC18CT (new 4.5 K refrigerator) QSCB+QSRB+QSAB CCSCT (1.8 K refrigerator) QSCCB+QURCB+QURCBCC SECTORXX (Tunnel distribution) QARC + QLSS IBCT (Interconnection box) QUI LHCACT (old 4.5 K refrigerator) QSCA+QSRA+QURA+QSAA+QSKA CCSCT (1.8 K refrigerator) QSCCA+QURCA+QURCACC SECTORXX (Tunnel distribution) QARC + QLSS : system B needs system A to work properly AB and/or QSDN (Nitrogen Storage) QSDH (Liquid Helium Storage) QSV (Gaseous Helium Storage)

10. CRYO RAMS STUDIES LHC 4.5 K refrigerator hierarchy 10

11. CRYO RAMS STUDIES 11 LHC 1.8 K refrigerator hierarchy

12. CRYO RAMS STUDIES QUI hierarchy 12

13. CRYO RAMS STUDIES Sector hierarchy 13

14. CRYO RAMS STUDIES SCADA-CRYO + dead-band + time filtering FEC + Dead-band PLC +Dead-band Sensor Data archiving and filtering 14 SCADAR DB (oracle DB in CCR) 6 months of data LOGGING DB (Oracle DB in 513) Forever Time stamp ~500 ms ~10 s ~15 min TT821 =1mK =2s Time stamp ~500 ms SCADA-CIET + dead-band + time filtering ~10 s TT/PT~1s LT~10s EH~0.5s =1mK =2s CRYO RAMS STUDIES

15. Events  When a discreet event is detected in PLC Old and new values are time-stamped in PLC Sent to SCADA Stored in Logging DB  UNICOS events embed: Mode management (Auto/Manual/Forced) On/Off status of digital signals Alarms and interlocks status and acknowledgement Sensor error Controller status (regulation, positioning, etc.) 15

16. CRYO RAMS STUDIES CRYO naming convention (1/3)  Equipment_Location_Name.Attribute  In refrigerators: Equipment = QXYZ Q = cryogenics X = surface (S) or Underground (U) Y = Compressors (C) or refrigerator (R) or Storage (S) or interconnection (I) Z = old cryoplant from LEP (A) or new ones (B) Location = LHC point number (18,2,4,6,8)  In tunnel : Equipment = Cryogenic Distribution Line (QRL) or magnet identifier (ex: LBARB) Location = XXYZ XX = Cell number (01->33) Y = sector position (right or left regarding the closest point) Z = closest LHC point (1->8) 16 CRYO RAMS STUDIES

17. CRYO naming convention (2/3)  Name = object type (+ number) Sensors = temperature (TT), pressure (PT), mass-flow (FT), level (LT), valve position (GT)… Actuators = on/off valve (PV), controlled valve (CV), electrical heater (EH), relief valve (QV)… Numbers depends on function and location along a circuit (e.g. 821 is for magnet cold mass)  Attribute = property of the object (UNICOS) PosSt = Position Status: value of sensor, position of actuator… OnSt = On Status: the valve is open, the compressor is running… OffSt = Off Status: the valve is closed, the compressor is stopped… Ist = Interlock Status: alarm/interlock is activated EvStsReg = Event Status Register: double word where each word is the old and the new value of the object status where each bit represents a property of the object (open, close, in error, in manual mode, in interlock, etc…). The bit number 6 always indicates that a sensor linked to this object is in error The bit number 4 always indicates that the position of this object has been forced by an operator 17 CRYO RAMS STUDIES

18. CRYO naming convention (3/3)  Example1 : LBALA_21R8_TT821.PosSt  LBALA: related to a dipole LBALA  21R8: located in the cell number 21 on the right side of point 8  TT821: temperature sensor related to a magnet cold mass  PosSt: this is the value of the sensor  Example2 : QSRB_8_400VOn.PosSt  QSRB: related to new 4.5 K cryogenic refrigerator on surface  8: located on point 8  400VOn: signal indicating the 400V presence  PosSt: this is the value of the sensor 18 CRYO RAMS STUDIES

19. TIMBER tree organisation  For LHC : CRYO  P18/2/4/6/8 CRYO  CRYO  Qcalc  QGlobal for the general LHC cryo information  Per point: Equipment  Nature  Domain  Nature (UNICOS): AI/DI = Analog/Digital Inputs (sensors) ALARM = alarms on units or on actuators ANALOG/ANADIG/ANADO/ONOFF = actuators AO/DO = Analog/Digital Outputs (orders to actuators) PID = regulation loops PCO = Process Control Objects (process units) APAR = Analog Parameters (thresholds for alarms)  Domain = Location in tunnel or equipment in refrigerator 19 CRYO RAMS STUDIES

20. Cryo start and Cryo maintain  Cryo Start  1 = Beam is allowed to be injected in the machine  0 = Beam cannot be injected in the machine  Cryo Maintain  1 = Beam can be kept in the machine  0 = Beam must be extracted  Computed in a script running every 10 sec in SCADA- CRYO data server 20 CRYO RAMS STUDIES

21. Cryo Start  Sub-signals:  ARC. E.g. CRYOS_AR12_CryoOK (1 x sector)  Inner Triplet. E.g. CRYOS_ITR1_CryoOK (0 or 1 or 2 x sector)  Matching Sections. E.g. CRYOS_MSR1_CryoOK (2 x sector)  To be checked: magnet and DFB temperatures and levels Beam screen temperatures and valves Vacuum Pressures of distribution lines Ethernet communications + operator authorisation  For RF (S34 & S45) : communications, vacuum, levels and pressures for RF cavities 21 CRYO RAMS STUDIES

22. Cryo Start for ARC12 22 CRYO RAMS STUDIES

23. Cryo Maintain  Sub-signals:  ARC. E.g. S12_AR12_CryoMaintain (1 x sector)  Inner Triplet. E.g. S12_ITR1_CryoMaintain (1 or 2 x sector)  Matching Sections. E.g. S12_MSR1_CryoMaintain (2 x sector)  To be checked: magnet and DFB temperatures and levels Beam screen temperatures and valves Vacuum Pressures of distribution lines  For RF (S34 & S45) : levels and pressures for RF cavities  Activated only if one condition is active for more than 30sec. 23 CRYO RAMS STUDIES

24. Cryo Maintain for ARC12 24 CRYO RAMS STUDIES

25. Example on a cryo maintain loss  08 September 2015: cryo maintain lost due to a high temperature (46 K) on the Q6 beam screen 06R2_TT947 at 3h24 UTC 25 QRLEA_06R2TT947.POSST S23_AML3_CRYOMAINTAIN.POSST CRYO RAMS STUDIES

26. Cryo Start and Maintain issues  Only give the process value potentially dangerous for the beam  Don’t provide the real cause of the event  An expert analyse is necessary to understand the cause of the event  Possible causes: Electrical failure (power cut or 24 V loss) Water failure (SF at surface or UW underground) Small cryogenic equipment failure (valve, heater, pump) Big cryogenic equipment failure (warm/cold compressor, turbine) Instrumentation or control failure (sensor fault, PLC fault) Control logic bug Bad regulation tuning Wrong human intervention.. 26

27. CRYO RAMS STUDIES BEEP (or BIP)  LHC cryogenics system has 25 000 alarms: difficult to handle and follow  Cryo operators are notified by a “BEEP” signal when some major errors occur Cryo start/maintain are not necessarily immediately impacted Allow operators to identify quickly important problems  BEEP are computed directly inside PLC A red box appears in the SCADA interfaces An automatic call is sent to CCC  BEEP is a group of interlocks based on the UNICOS decomposition.  QSCB/QSCCB: ~20 BEEP (compressors, oil circuits, coalescers)  QSRB/QURCB: ~30 BEEP (turbines, connection and safety valves, vacuum, adsorbers, helium level)  ARC/LSS: ~50 BEEP (temperatures, levels, pressures, sensor error, heater fault)  + all communications : ~130 BEEP  + all summary to build a tree  Around 1500 BEEP signal in total over LHC  In Logging DB: look for %BIP%IST or %BEEP%IST 27

28. CRYO RAMS STUDIES BEEP overview 28

29. CRYO RAMS STUDIES BEEP details for a sector 29

30. CRYO RAMS STUDIES Conclusion  LHC cryogenics is a large-scale complex system  Control system is using the CERN control standard UNICOS  All data and events are timestamped at the source and stored in the Logging DB  Several signals help experts to understand failures but it is not so easy due to the amount of data and to the complexity of the system 30