1. LHC Machine Interlocks & Beam Operation LHC Machine Interlocks & Beam Operation ARW2011Bruno PUCCIO (CERN) 13 th April 2011 1v0 Thanks to Benjamin Todd & Markus Zerlauth

2. Bruno PUCCIOARW2011 – 13 th April 2011 2 2 Outline  LHC Machine Interlocks overview  Powering Interlocks systems  Beam Interlock system  Characteristics & Layout  Performance  Monitoring & Operational checks  Summary

3. Bruno PUCCIOARW2011 – 13 th April 2011 3 3 10-20x energy per magnet of TEVATRON magnet quenched = hours downtime many magnets quenched = days downtime (few spares) 100x energy of TEVATRON Emergency Discharge Magnet Energy (10 GJ) Powering Protection: Beam DumpBeam Energy (360 MJ) Beam Protection: magnet damaged = $1 million, months downtime many magnets damaged = many millions, many months downtime 0.000005% of beam lost into a magnet = quench 0.005% beam lost into magnet = damage Failure in protection – complete loss of LHC is possible Protection Functions LHC is to a large extent a super-conducting machine: 1232 main dipoles, ~400 main quadrupoles and more than 8000 correctors

4. Bruno PUCCIOARW2011 – 13 th April 2011 4 What are the “Machine Interlocks”? 4 Beam Interlock System (VME based) for protecting Normal Conducting Magnets for protecting the Equipments for Beam Operation BIS Fast Magnet Current change Monitor FMCM Powering Interlock Controllers (PLC based) + PIC Warm magnet Interlock Controllers (PLC based) WIC + Safe Machine Parameters System (VME based) SMP or Super Conducting Magnets

5. Bruno PUCCIOARW2011 – 13 th April 2011 5 LHC Machine Interlocks Hierarchy 5 ( Machine Interlocks systems in red ) EXPERIMENTS

6. LHC Magnet & Powering Interlock Systems

7. Bruno PUCCIOARW2011 – 13 th April 2011 7 Key facts for Powering Interlock Systems 7 Both powering interlock systems use of industrial electronics (SIEMENS PLCs with remote I/O modules) Distributed systems corresponding to LHC machine sectorization All critical signals are transmitted using HW links (Fail safe signal transmission, built in redundancy) All circuit related systems OK => Power Permit, else dump beams and activate Energy extraction (if any) Reaction time: 36 controllers for Superconducting magnets (PIC system) 8 controllers for Normal Conducting magnets (WIC) ~ 1mS 100 ms ~ 10’000 superconducting magnets: powered in 1600 electrical circuits 140 normal conducting magnets powered in 44 electrical circuits(in the LHC) And more than 1000 magnets in the injectors chain Superconducting circuit protection:Normal Conducting circuit protection:

8. Bruno PUCCIOARW2011 – 13 th April 2011 SCADA application: monitoring views… 8 Magnets status Power Conv. status SPS Transfer Lines SCADA: Supervisory Control and Data Acquisition Courtesy of F.Bernard (CERN) Permit A Permit B

9. Bruno PUCCIOARW2011 – 13 th April 2011 SCADA application: History Buffer 9

10. Beam Interlock System 1v0 10

11. Bruno PUCCIOARW2011 – 13 th April 2011 11 Beam Interlock System Function BIS Dumping system or Extraction Kicker or Beam Stopper or Beam source…. Target system Beam ‘Permit’ Signals Σ(User Permit = “TRUE” ) => Beam Operation is allowed IF one User Permit = “FALSE” => Beam Operation is stopped

12. Bruno PUCCIOARW2011 – 13 th April 2011 Beam Interlock System: quick overview 12 User Interfaces User Permit #1 #14 #2 (installed in User’s rack) Beam Interlock Controller (VME chassis) copper cables User System #1 User System #2 User System #14 front rear Optical outputs copper cables or fiber optics links  Remote User Interfaces safely transmit Permit signals from connected systems to Controller  Controller acts as a concentrator  collecting User Systems Permits  generating local Beam Permit  Controllers could be daisy chained (Tree architecture) or could share Beam Permit Loops (Ring architecture) JAVA Application Configuration DB Technical Network Front End Software Application local Beam Permit Cupper links

13. Bruno PUCCIOARW2011 – 13 th April 2011 13 LHC Beam Permit Loops Square wave generated at IR6: Signal can be cut and monitored by any Controller When any of the four signals are absent at IR6, BEAM DUMP! 4 fibre-optic channels: 1 clockwise & 1 anticlockwise for each beam but they can be linked (or unlinked) 17 Beam Interlock Controllers per beam (2 per Insertion Region (IR) + 1 near Control Room) Beam-1 / Beam-2 loops are independent

14. Bruno PUCCIOARW2011 – 13 th April 2011 Beam Interlock Systems currently in Operation 50 Controllers In total: ~ 370 connected systems SPS to LHC Transfer lines 14 c.  SPS ring  (since 2006) 6 c. 4 c. LHC Injection regions LHC ring (since 2007) 34 controllers 14 Resistors65’160 Capacitors32’612 Connectors9’543 Inductors72 Relays1’204 Optocouplers4’816 Integrated Circuits12’508 PLDs884 Diodes32’007 Transistors12’204 Regulators224 Fuses1’204 ELED Transm.72 PIN Receivers72 All components172’582

15. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Performance (1/3) Safe: (Safety Integrity Level 3 was used as a guideline). Must react with a probability of unsafe failure of less than 10 -7 per hour and, Beam abort less than 1% of missions due to internal failure (2 to 4 failures per year). Reliable: (whole design studied using Military and Failure Modes Handbooks) Results from the LHC analysis are: P (false beam dump) per hour = 9.1 x 10 -4 P (missed beam dump) per hour = 3.3 x 10 -9 15 Fail Safe concept: Must go to fail safe state whatever the failure Available: Uninterruptable Powering (UPS)) Redundant Power Supply for Controller (i.e. VME crate) Redundant Power Supply for Remote User Interface

16. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Performance (2/3) Critical process in Hardware: ♦ functionality into 2 redundant matrices ♦ VHDL code written by different engineers following same specification. Critical / Non-Critical separation: ♦ Critical functionality always separated from non-critical. ♦ Monitoring elements fully independent of the two redundant safety channels. 16 Manager board FPGA chip (Monitoring part) CPLD chip (Matrix A) CPLD chip (Matrix B) Used CPLD: 288 macro-cells & 6’400 equivalent gates Used FPGA: 30’000 macro-cells & 1 million gates + all the built in RAM,etc. FPGA: Field Programmable Gate Array CPLD: Complex Programmable Logic Device

17. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Performance (3/3) 100% Online Test Coverage: Can be easily tested from end-to-end in a safe manner => recovered “good as new” 17 Fast: ~20μS reaction time from User Permit change detection to the corresponding Local Beam Permit change Modular

18. Bruno PUCCIOARW2011 – 13 th April 2011 Control Room GUIs 18

19. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Feature YES FALSE Within a fixed partition, half of User Permit signals could be remotely masked “Flexible”: thanks to Input Masking Masking automatically removed when Setup Beam Flag = FALSE Masking depends on an external condition: the Setup Beam Flag -generated by a separate & dedicated system (Safe Machine Parameters) -distributed by Timing 19

20. Bruno PUCCIOARW2011 – 13 th April 2011 History Buffer time

21. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Application: Timing Diagram Courtesy of J.Wenninger (CERN)

22. Bruno PUCCIOARW2011 – 13 th April 2011 Operational Checks Post-Operation checks (included in Post Mortem analysis ) Pre-Operation checks (launched by Beam Sequencer) configuration verification and integrity check fault diagnosis and monitoring During Operation (DiaMon application) response analysis In order to ensure that safety is not compromised, the verification is carried out in three stages

23. Bruno PUCCIOARW2011 – 13 th April 2011 23 Operational experience  Originally designed for LHC and firstly installed in its pre-injector for validation. Fully operational since 2006 for the SPS ring and its transfer lines.  Since restart in Nov.09, LHC-ring BIS extensively exercised with more than 1000 emergency dumps.  Promising overall availability (only few failures with redundant VME Power Supplies and with VME Processor boards) Very high availability concerning in-house part (99.996%) with only one stop due to a failure.  Concerning the remote User Interfaces: as foreseen, some PSU failed; thanks to redundancy, it has not lead to a beam operation disruption. Beam Interlock System Very good experience for both Powering Interlock Systems. Already > 4 years of operation (starting with initial LHC Hardware Commissioning) Highly dependable (only two failures in more than 4 years) Powering Interlock Systems

24. Bruno PUCCIOARW2011 – 13 th April 2011 24 LHC 2010 run: downtime distribution 24 % Powering Interlock System Beam Interlock System Warm Magnet Interlock System : 0 (percentage of total downtime) %

25. Bruno PUCCIOARW2011 – 13 th April 2011 25 Summary (1/2) 25  Core of the LHC machine protection  Fail Safe concept  Fast and modular  Fully redundant and Critical process separated from Monitoring  Redundant Power Supplies + UPS  On-line Testable => recovered “As Good As New” end-to-end  Automated tools to perform regular and quick validation: - internal to Beam Interlock System - external in involving connected systems

26. Bruno PUCCIOARW2011 – 13 th April 2011 26 Summary (2/2) 26  Embedded features for monitoring and testing internal interlock process Together with powerful GUI application: - it provides clear and useful information to Operation crew - it minimize machine downtime  3-stage verification: - Validation prior to beam operation (Pre-Operational checks) - On-line diagnostics during beam operation - Post Operation checks  Reliable systems: in operation since few years with a reduced number of aborted beam operations due to internal failure.

27. Bruno PUCCIOARW2011 – 13 th April 2011 bruno.puccio@cern.ch 27 CERN Thank you for your attention

28. Bruno PUCCIOARW2011 – 13 th April 2011 Spare 28

29. Bruno PUCCIOARW2011 – 13 th April 2011 Protection of NC magnets 29  based on Safety PLC  collect input signals from: - thermo-switches, - flow meters, - red buttons, …  give Power Permit for the corresponding converter Magnet 1 Power Converter Magnet 2 PC Status Thermoswitches Water Flow Red button… Several thermo- switches @ 60°C Power Permit PVSS Operator Console Ethernet PLC + I/Os Beam Permit BIS interface  WIC solution = PLC crate + remote I/O crates Profibus-Safe link remote I/Os Configuration DB NC = Normal Conducting

30. Bruno PUCCIOARW2011 – 13 th April 2011 WIC: remote test feature 30 - Facilitate as-good-as-new testing - perform thanks to relays implanted into the magnet interlock boxes. - simulate the opening of the thermo-switches or the flow sensors. Guarantee the system integrity; in particular after an intervention on the magnet sensors or after a modification of the configuration file. Thermo-Switch Test Button Test Relay magnet interlock box PLC OUTPUT PLC INPUT NE4 or NE8 cable WIC WIC: remote test feature

31. Bruno PUCCIOARW2011 – 13 th April 2011 31 Protection of SC magnets / circuits 31 Magnet 1 Power Converter Magnet 2 HTS Current Leads sc busbar DFB Internal failures / Ground Faults Cooling Failures AUG, UPS, Mains Failures Normal conducting cables Quench Signal Superconducting Diode Energy Extraction Quench- Heater QPS + nQPS Power Permit Powering Interlock Controller CRYO_OK Beam Permit BIS interface SC = Super Conducting Courtesy of M. Zerlauth (CERN)

32. Bruno PUCCIOARW2011 – 13 th April 2011 32 Fast Magnet Current Change Monitors Magnet 1 Power Converter Magnet 2 Beam Dump to BIS Fast Magnet Current Change Monitors are (strictly speaking) not interlocking powering equipment Installed on nc magnets with << natural τ (injection/extraction septas, D1 magnets in IR1/IR5, …) and large impact on beam in case of powering failures DESY invention which has been ported with great success to LHC and SPS-LHC transfer lines U_circuit Courtesy of M. Zerlauth (CERN)

33. Bruno PUCCIOARW2011 – 13 th April 2011 BIS Hardware CIBD CIBMCIBT CIBMD & CIBTD CIBO CIBG CIBI CIBS CIBF u & CIBF c CIBX More than 2000 boards produced (~85% in operation) CIBU CIBE CIBP CIBTD & CIBMD