1. Machine Protection @ LHC Jörg Wenninger CERN Accelerators and Beams Department Operations group CERN-ITER meeting, Dec 2008

2. Introduction 215/12/2008ITER-CERN WS - J. Wenninger

3. LHC history 3 1982 : First studies for the LHC project 1983 : Z0/W discovered at SPS proton antiproton collider (SppbarS) 1989 : Start of LEP operation (Z boson-factory) 1994 : Approval of the LHC by the CERN Council 1996 : Final decision to start the LHC construction 1996 : LEP operation > 80 GeV (W boson -factory) 2000 : Last year of LEP operation above 100 GeV 2001 : Birth of the LHC Machine Protection WG 2002 : LEP equipment removed 2003 : Start of the LHC installation 2005 : Start of LHC hardware commissioning 2008 : LHC commissioning with beam 15/12/2008ITER-CERN WS - J. Wenninger

4. 4 7 years of construction to replace : LEP: 1989-2000 e+e- collider 4 experiments max. energy 104 GeV circumference 26.7 km in the same tunnel by LHC : 2008-2020+ proton-proton & ion-ion collider in the LEP tunnel 4+ experiments energy 7 TeV ATLAS CMS LHCB ALICE 15/12/2008ITER-CERN WS - J. Wenninger

5. 5 Tunnel circumference 26.7 km, tunnel diameter 3.8 m Depth : ~ 70-140 m – tunnel is inclined by ~ 1.4%

6. 6 Top energy/GeV Circumference/m Linac 0.12 30 PSB 1.4 157 CPS 26 628 = 4 PSB SPS 450 6’911 = 11 x PS LHC 7000 26’657 = 27/7 x SPS LEIR CPS SPS Booster LINACS LHC 3 4 5 6 7 8 1 2 Ions protons Beam 1 Beam 2 TI8 TI2 Note the energy gain/machine of 10 to 20 – and not more ! The gain is typical for the useful range of magnets !!! 15/12/2008ITER-CERN WS - J. Wenninger

7. 7 IR6: Beam dumping system IR4: RF + Beam instrumentation IR5:CMS IR1: ATLAS IR8: LHC-B IR2:ALICE Injection ring 2 Injection ring 1 IR3: Momentum collimation (normal conducting magnets) IR7: Betatron collimation (normal conducting magnets) Beam dump blocks LHC Layout  8 arcs.  8 long straight sections (insertions), ~ 700 m long.  beam 1 : clockwise  beam 2 : counter-clockwise  The beams exchange their positions (inside/outside) in 4 points to ensure that both rings have the same circumference ! The main dipole magnets define the geometry of the circle ! 15/12/2008

8. The Challenge : stored energy 8 Increase with respect to existing accelerators : A factor 2 in magnetic field A factor 7 in beam energy A factor 200 in stored beam energy 15/12/2008

9. Dipole 9 7 TeV 8.33 T 11850 A 7M J

10. Powering/circuit layout 10 Powering Sector Sector 1 5 DC Power feed 3 DC Power 2 4 6 8 7 LHC 27 km Circumference  To limit the stored energy within one electrical circuit, the LHC is powered by sectors.  The main dipole circuits are split into 8 sectors to bring down the stored energy to ~1 GJ/sector.  Each main sector (~2.9 km) includes 154 dipole magnets (powered by a single power converter) and ~50 quadrupoles.  This also facilitates the commissioning that can be done sector by sector !

11. Quench protection - arcs 11 1.The quench is detected based on voltage measurements over the coils (U_mag_A, U_mag_B). 2.The energy is distributed over the entire magnet by force-quenching with quench heaters. 3.The power converter is switched off. 4.The current within the quenched magnet decays in < 200 ms, circuit current now flows through the ‚bypass‘ diode that can stand the current for 100-200 s. 5.The circuit current/energy is discharged into the dump resistors. 6.The beam is dumped. >> 2-6 happen ‚in parallel‘

12. 12 Top energy/GeV Stored E/MJ Linac 0.12 PSB 1.4 ~0.005 CPS 26 ~0.2 SPS 450 3 LHC 7000 360 LEIR CPS SPS Booster LINACS LHC 3 4 5 6 7 8 1 2 Ions protons Beam 1 Beam 2 TI8 TI2 In RED : accelerators where machine protection due to beam is critical. 15/12/2008ITER-CERN WS - J. Wenninger

13. 15/12/2008ITER-CERN WS - J. Wenninger13 The LBDS LHC Beam Dumping System LBDS inventory Extraction15 Kicker Magnets + 15 generators 10 Septum Magnets + 1 power converter Dilution10 Kicker Magnets + 10 generators AbsorptionOne dump block ElectronicsBeam energy measurement (BEM) Beam energy tracking (BET) Triggering and re-triggering Post mortem diagnostics (check of every beam dump) Beam line975 m from extraction point to TDE 1) MKD The 15 kicker magnets deflect the beam horizontally 4) MKB The 10 kicker magnets dilute the beam energy 3) MSD The 15 septum magnets deflect the beam vertically 5) TDE The beam is absorbed in a graphite block 2) Q4 The quadrupole enhances the horizontal deflection The beam sweep at the front face of the TDE absorber at 450 GeV

14. The dump block 14 Approx. 8 m concrete shielding beam absorber (graphite)  This is the ONLY element in the LHC that can withstand the impact of the full beam !  The block is made of graphite (low Z material) to spread out the hadronic showers over a large volume.  It is actually necessary to paint the beam over the surface to keep the peak energy densities at a tolerable level !

15. MPS mission 15/12/2008ITER-CERN WS - J. Wenninger15 The central mission of beam related machine protection at the LHC is to ensure that the beam is ALWAYS safely extracted to the dump block since there is no other element that can withstand the impact of the full LHC beam.

16. 16 energy ramp preparation and access beam dump injection phase coast LHC cycle L.Bottura 450 GeV 7 TeV start of the ramp 15/12/2008ITER-CERN WS - J. Wenninger

17. Machine protection organization 17 The machine protection issues that we are discussing here concern only protection of the accelerator from beam related damage. Protection of the personnel and equipment protection against non-beam hazards are dealt elsewhere. 15/12/2008ITER-CERN WS - J. Wenninger

18. ‘MPWG’ : Machine Protection Working Group 18  Machine Protection @ CERN concerns many different hardware systems  Different CERN departments and groups responsible for the equipment  Up to 2000, no coordinated beam MP work, effort mostly concentrated on equipment ‘self-’protection.  Quench protection for SC magnets …  In 2001 the MPWG was launched by R. Schmidt (J. Wenninger sc. secretary ) to ensure a coordinate MP effort.  MPWG coordinates MP work, takes decisions (consensus) and, if needed, resolves ‘conflicts’.  Individual equipment groups remain responsible of their equipment etc. 15/12/2008ITER-CERN WS - J. Wenninger

19. MPWG activities and evolution 19  Reviews and external audits, initiated or encouraged by MPWG, are used to obtain external advice  General review LHC Machine Protection System  Audit of Beam Interlock System  Audit of Beam Dumping System  Audit of Beam Loss Monitoring System  Sub-working groups were launched as appropriate.  Reliability studies sub-WG.  Commissioning sub-WG.  Following the LHC startup with beam in 2008, the MPWG has been transformed and exists now as Machine Protection Panel (‘MPP’) with a reduced number of members.  Follow up of MP issues at ‘running’ LHC.  Defines limits for safe operation. 15/12/2008ITER-CERN WS - J. Wenninger

20. 20 MPS requirements  Safety Assessment (‘reliability’) –IEC 61508 standard defining the different Safety Integrity Levels (SIL) ranking from SIL1 to SIL4 –Based on Risk Classes = Consequence x Frequency –Machine Protection System for the LHC should be SIL3, taking definition of Protection Systems, with a probability of failure between 10 -8 and 10 -7 per hour (because of short mission times) Catastrophy = beam should have been dumped and this did not take place; can possibly cause large damage  Availability –Definition: Beam is dumped when it was not required Operation can not take place because the protection system does not give the green light (is not ready) –Requirement: Downtime comparable to other accelerator equipment; maximum tens of operations per year 15/12/2008ITER-CERN WS - J. Wenninger

21. 21 Dual approach  Prevent faults at the source.  Equipment ‘design’ – reliability.  Fast internal failure detection.  Detect the effect resulting from any fault, including beam instabilities, and react fast enough to prevent damage.  Simulation of failures.  Knowledge of damage levels. 15/12/2008ITER-CERN WS - J. Wenninger

22. Failure studies 2215/12/2008ITER-CERN WS - J. Wenninger

23. Failure categories 23 Beam loss over multiple turns due to many types of failures. Fastest failures >= ~ 10-ish turns Passive protection - Failure prevention (high reliability systems). -Intercept beam with collimators and absorber blocks. Active protection systems have no time to react ! Active Protection - Failure detection (by beam and/or equipment monitoring) with fast reaction time (< 1 ms). - Fire beam dumping system Beam loss over a single turn during injection, beam dump or any other fast ‘kick’. In the event a failure or unacceptable beam lifetime, the beam must be dumped immediately and safely into the beam dump block. Two main classes for failures (with more subtle sub-classes): 15/12/2008ITER-CERN WS - J. Wenninger

24. Failure categories 2415/12/2008ITER-CERN WS - J. Wenninger

25. Collimation system 25 A multi-stage halo cleaning (collimation) system has been designed to protect the LHC magnets from beam induced quenches.  Halo particles are first scattered by the primary collimator (closest to the beam). The scattered particles (forming the secondary halo) are absorbed by the secondary collimators, or scattered to form the tertiary halo.  More than 100 collimators jaws are needed for the nominal LHC beam.  Primary and secondary collimators are made of Carbon to survive severe beam impacts !  the collimators have a key role for protection as they define the aperture : in (almost) all failure cases the beam will touch collimators first !! Primary collimator Secondary collimators Absorbers Protection devices Tertiary collimators Triplet magnets Experiment Beam Primary halo particle Secondary halo Tertiary halo + hadronic showers hadronic showers ITER-CERN WS - J. Wenninger

26. Collimator settings at 7 TeV 26 1 mm Opening ~3-5 mm The collimator opening corresponds roughly to the size of Spain !  For colliders like HERA, TEVATRON, RHIC, LEP collimators are/were used to reduce backgrounds in the experiments ! But the machines can/could actually operate without collimators !  At the LHC collimators are essential for machine operation as soon as we have more than a few % of the nominal beam intensity ! ITER-CERN WS - J. Wenninger

27. Collimator robustness 27  Around ~2001 when the MPWG started its work, the LHC collimation system consisted of Copper collimators : Excellent for beam stability (low resistivity) Good for collimation itself (density).  A single mis-injection would have damaged the collimators !!! >> Failures were not considered in the design !!!!  A review of the collimation system requirements indicated that a major re- design was needed !! Collimation project & collimation WG were launched. Work in close collaboration with MPWG. Robust collimator design based on Carbon collimators – ‘phase 1’.  The phase 1 collimator will not allow nominal beams due to beam instability issues (Carbon resistivity). >> Phase 2 collimator design in progress. 15/12/2008ITER-CERN WS - J. Wenninger

28. Damage levels 2815/12/2008ITER-CERN WS - J. Wenninger

29. Beam induced damage test 29 25 cm >> organized a controlled beam experiment:  Special target (sandwich of Tin, Steel, Copper plates) installed in an SPS transfer line.  Impact of 450 GeV LHC beam (beam size σ x/y ~ 1 mm) Beam The effect of a high intensity beam impacting on equipment is not so easy to evaluate, in particular when you are looking for damage : heating, melting, vaporization … >> very little experimental data available ! 15/12/2008

30. Damage potential of high energy beams 30 A B D C ShotIntensity / p+ A1.2×10 12 B2.4×10 12 C4.8×10 12 D7.2×10 12 Controlled experiment with 450 GeV beam to benchmark simulations: Melting point of Copper is reached for an impact of  2.5×10 12 p, damage at  5×10 12 p. Stainless steel is not damaged with 7×10 12 p. Results agree with simulation. Effect of beam impact depends strongly on impact angles, beam size… Based on those results LHC has a limit for safe beam at 450 GeV of 10 12 protons ~ 0.3% of the total intensity ~ 0.1 MJ Scaling the results (beam size reduction etc) yields a limit @ 7 TeV of 10 10 protons ~ 0.003% of the total intensity ~ 0.02 MJ 15/12/2008ITER-CERN WS - J. Wenninger

31. When the MPS is not fast enough… 15/12/2008ITER-CERN WS - J. Wenninger31 At the SPS the MPS was been ‘assembled’ in stages over the years, but not following a proper failure analysis. As a consequence the MPS cannot cope with every situation! It is now also covered by the MPWG but would require new resources… Here an example from …. 2008 ! The effect of an impact on the vacuum chamber of a 400 GeV beam of 3x10 13 p (2 MJ). Vacuum chamber to atmospheric pressure, Downtime ~ 3 days.

32. Full LHC beam deflected into copper target 32 Target length [cm] vaporisation melting Copper target 2 m Energy density [GeV/cm 3 ] on target axis 2808 bunches The beam will drill a hole along the target axis !! 15/12/2008ITER-CERN WS - J. Wenninger

33. Beams and damage 15/12/2008ITER-CERN WS - J. Wenninger33 Beam typeNo. protons Safe @ 450 GeV Safe @ 7 TeV Comment Probe bunch2x10 9 YES !!‘YES’ Nominal bunch1x10 11 YESNOSafe for collimators Nominal injection (288 bunches) 3x10 13 NO Safe for collimators at 450 GeV(*) Full beam4x10 14 NO At injection commissioning can be done safely with one bunch. At 7 TeV even the smallest bunch is just about safe. (*) : also tested with 450 GeV beams (same time as damage test). Note that a first test resulted in mechanical deformations that led to an improved design (that was retested with beam).

34. Lessons from the 19 th September incident 15/12/2008ITER-CERN WS - J. Wenninger34 An severe incident occurred on 19 th September during the last powering tests of one LHC sector (sector 34):  At 8.7kA a resistive zone developed in the dipole bus bar between dipoles.  Most likely an electrical arc developed which punctured the helium enclosure.  Large amounts of Helium were released into the insulating vacuum.  Rapid pressure rise inside the LHC magnets –Large pressure wave travelled along the accelerator both ways. –Self actuating relief valves opened but could not handle all. –Large forces exerted on the vacuum barriers located every 2 cells. –These forces displaced several quadrupoles by up to ~50 cm. –Connections to the cryogenic line damaged in some places. –Beam ‘vacuum’ to atmospheric pressure >> Repair of ~ 50 magnets. >> Indicates that the collateral damage due to beam impact can be much more severe that anticipated  consolidation under way !

35. Failure studies 3515/12/2008ITER-CERN WS - J. Wenninger

36. Simulations 3615/12/2008ITER-CERN WS - J. Wenninger  Many failures simulations were performed under the guidance of MPWG members.  They resulted in : Correct requirements for protection systems. Design changes and new developments. Typical example : Current decay curves of power converters are used to asses criticality of magnetic circuits. PHD - A. Gomez

37. Simulation result examples 37  The evolution of the beam parameters, here beam orbit, is used to evaluate REACTION times for internal interlocks and for beam diagnostic systems (beam loss monitors). Orbit along the ring Orbit around collimators Collimator jaw PHD - A. Gomez

38. Simulation result example 38  Using a certain transverse beam distributions (usually nominal size with Gaussian shape) it is possible to reconstruct the beam lost at various locations versus time to evaluate REACTION times for internal interlocks and for beam diagnostic systems (beam loss monitors). PHD - A. Gomez

39. Failure studies outcome 3915/12/2008ITER-CERN WS - J. Wenninger

40. Beam loss monitors 40  Ionization chambers to detect beam losses: –N 2 gas filling at 100 mbar over-pressure, voltage 1.5 kV –Sensitive volume 1.5 l  Requirements (backed by simulations) : –Very fast reaction time ~ ½ turn (40  s) –Very large dynamic range (> 10 6 )  There are ~3600 chambers distributed over the ring to detect abnormal beam losses and if necessary trigger a beam abort !

41. FMCMs 41  Simulations indicated absence of redundancy and very short reaction times for BLMs for failures of some normal-conducting circuits in the LHC.  Led to the development (together with DESY/Hamburg) of so-called FMCMs (Fast Magnet Current change Monitor) that provide protection against fast magnet current changes after powering failures: Very fast detection (< 1 ms) of voltage changes on the circuit. Tolerances of ~ 10 - 4 on DI/I achievable. The hardware is based on a DESY design. BIS interface resistive magnet Fast Magnet Current change Monitor Power Converter VIPC626 VME Crate CPU + CTRP (or TG8) Voltage Divider & Isolation Amplifier RS422 link

42. FMCM Test Example 42 Transfer line dipole PC: >> Steep step programmed into the PC reference to simulate failure FMCM interlock trigger time:  I < 0.1 A  I/I < 0.01% - specification : 0.1% Zoom around step time 15/12/2008ITER-CERN WS - J. Wenninger

43. Beam Interlock System 4315/12/2008ITER-CERN WS - J. Wenninger

44. Interlock System Overview 44 Beam Interlock System Beam Dumping System Injection BIS PIC essential + auxiliary circuits WIC QPS (several 1000) Power Converters ~1500 AUG UPS Power Converters Magnets FMCM Cryo OK RF System Movable Devices Experiments BCM Beam Loss Experimental Magnets Collimation System Collimator Positions Environmental parameters Transverse Feedback Beam Aperture Kickers FBCM Lifetime BTV BTV screensMirrors Access System DoorsEIS Vacuum System Vacuum valves Access Safety Blocks RF Stoppers BLM BPM in IR6 Monitors aperture limits (some 100) Monitors in arcs (several 1000) Timing System (Post Mortem) CCC Operator Buttons Safe Mach. Param. Software Interlocks LHC Devices SEQ LHC Devices LHC Devices Timing Safe Beam Flag Over 10’000 signals enter the interlock system of the LHC !! 15/12/2008ITER-CERN WS - J. Wenninger

45. 45 Beam Interlock System BIS Dump Kicker Beam ‘Permit’ User permit signals Actors and signal exchange for the beam interlock system: ‘User systems’ : systems that survey equipment or beam parameters and that are able to detect failures and send a HW signal to the beam interlock system. Each user system provides a HW status signal, the user permit signal. The beam interlock system combines the user permits and produces the beam permit. The beam permit is a HW signal that is provided to the dump kicker (also injection or extraction kickers) : absence of beam permit  dump triggered ! Hardware links and systems 15/12/2008ITER-CERN WS - J. Wenninger

46. Beam Interlock System 46 User Interfaces User Permit #1 #14 #2 Beam Interlock Controller copper cables User System #1 User System #2 User System #14 front rear Beam Permit Loops (F.O.)  Unique Hw solution for connecting any user system (= interlock) via a copper cable. Fiber optic variant for long links (>1.2km)  BIC (Beam Interlock Controller) boards embedded in VME chassis.  Beam Permit Loops with Frequency signals connect the BICs with the corresponding kicker system (extraction, injection, dump).  In operation at the SPS and the SPS/LHC transfer lines since 2006. Inputs are:  maskable (with safe beam)  unmaskable

47. Architecture of the LHC BEAM INTERLOCK SYSTEM Beam-1 / Beam-2 are Independent! - fast reaction time (~  s) - safe - limited no. of inputs - Some inputs maskable for safe beam intensity Up to 20 Users per BIC system: 6 x Beam-1 8 x Both-Beam 6 x Beam-2 Connected to injection IR2/IR8: -In case of an interlock (=NO beam permit), the beam is dumped & injection is inhibited. - It is not possible to inhibit injection ALONE. 47

48. 48 BIS Reaction Times User System process a failure has been detected… beam dump request Beam Dumping System waiting for beam gap 89μs max Signals send to LBDS t2t2 t3t3 Beam Interlock system process ~70μs max. t1t1 > 10 μ s USER_PERMIT signal changes from TRUE to FALSE Kicker fired t4t4 all bunches have been extracted ~ 89μs Achievable response time ranges between 100  s and 270  s (between the detection of a dump request and the completion of a beam dump) 15/12/2008ITER-CERN WS - J. Wenninger

49. In action… 4915/12/2008ITER-CERN WS - J. Wenninger

50. First Emergency Dump 50 First “Emergency Dump” on Thurs 11 th at 22:45:08  On 11th September 2008 during operation with circulating beam.  At 22:45:08, beam 2 was dumped by the LBDS triggered by the BIS.  The dump was caused by a water fault in the DC cables in the main quadrupole circuit in LHC sector 81.  This event allowed to address the performance of the interlock / machine protection systems at a very early state, as well as to understand the functionality of the post mortem (transient data) recording 15/12/2008ITER-CERN WS - J. Wenninger

51. First Dump 51 Data from Beam Interlock System Beam Interlock Controller at IP8 received dump request at 561.437 ms Beam Interlock Controller at IP6 received dump request - 50  s later (anti clockwise signal) - 180  s later (clockwise signal) Beam Dump 561.523 ms

52. Post-mortem System 15/12/2008ITER-CERN WS - J. Wenninger52 As indicated on the previous slide the Post-mortem data is very important.  The diagnostics of failures is essential to: Understand what happened. Assess the performance / correct functioning of the MPS. For critical systems like beam dumping system, the PM analysis is MANDATORY to ensure that the system is ‘as good as new’.  At the LHC all equipment systems provide post-mortem data:  Circular buffers that are frozen on fault/beam abort.  Accurate timestamps, down to  s for fast systems.  Data relevant for understanding of failures.  Buffer depth and granularity dependent on system. Typical for beam diagnostics is turn by turn (sometimes bunch by bunch). >> ‘Expected’ data volume for LHC : 2-5 GBytes

53. Software interlocking 15/12/2008ITER-CERN WS - J. Wenninger53  In very large accelerators it is not always possible to cover all failure mechanisms with a hardware system: needs something more flexible. Example : At the LHC the integrated bending field of horizontal steering magnets may bias the beam energy and cause problems during beam aborts.  Provide flexibility to quickly add new interlocks (provided they are not too time critical).  Need to survey the integrity of the settings even with a MCS system: Comparison of data and digital signatures between front end computers and DB. >> Software Interlock System to survey the control system components relevant for machine protection as additional protection layer, with possibility to abort beam if necessary.

54. MPS settings control 15/12/2008ITER-CERN WS - J. Wenninger54  A Critical Settings Management (MCS) system has been developed for the LHC (and for CERN in general) to be able to control MPS settings (for example Beam loss monitor thresholds…) through the central controls database without loss of security.  MCS provides:  Critical settings that can only be changed by authorized groups of persons.  Parameters are visible to everyone that has access to the control system.  Authentication and Authorization of the user.  Verification that values of critical parameters have not changed since the authorized person has updated them:  Data transfer errors.  ‘Hacking’.  Data corruption – radiation, data loss during reboots…

55. Critical settings control 15/12/2008ITER-CERN WS - J. Wenninger55 Based on the concept of public & private key.  User logs in.  The critical data receives a digital signature.  Data and digital signature are: Send to the front-end system which verifies the data validity. Stored together in the DB - avoid direct DB access, reference for checks.

56. Documentation 15/12/2008ITER-CERN WS - J. Wenninger56  All presentations, minutes of meetings etc are accessible from the machine protection web site : http://lhc-mpwg.web.cern.ch/lhc-mpwg/ which is however only accessible from INSIDE CERN.  MP commissioning documents for SPS and LHC are on 2 other sites: https://sps-mp-operation.web.cern.ch/sps-mp-operation/ https://lhc-mp-operation.web.cern.ch/lhc-mp-operation/