1. 1 LHC Timing Requirements and implementation J.Lewis for AB/CO/HT 21 st /Sep/06 TC presentation

2. 2 Initial observations on LHC timing Basic-Periods don’t mean a lot Telegrams don’t mean a lot Cycle means even less No super-cycles Little or no PPM/Multiplexing The LHC timing is machine safety critical, so it must be very simple/reliable and hardware monitored Response time to operational requests must be very rapid <<100ms UTC is important LHC timing directly controlled by the High Level LSA sequencer [LSEQ] in real time! It’s a Collider

3. 3 Basic LIC & LHC Requirements Respond to commands from the High Level LSA Sequencer [LSEQ] for LHC events in < 100ms  Provide an accurate UTC time reference Pilot the LHC Injector Chain [LIC] to fill the LHC Produce the LHC timing from external events and tables loaded by LSEQ Distribute the safe beam parameters and flags very reliably Minimize impact on the existing controls infrastructure

4. 4 Respond to commands from the High Level LSA Sequencer [LSRQ] Implement a suitable API which… Selects Pilot/Intermediate/Nominal [PIN] LIC beams Controls LIC to deliver a set of CPS batches [1..4] to a target RF bucket and Ring with correct characteristics Sends LHC event(s) on request from LSEQ Loads, runs N times, starts, stops and aborts concurrent asynchronous LHC event tables on LSEQ requests Distributes repeatedly at 1Hz some machine parameters such as MODE

5. 5 About the LHC Timing, observations LHC MTG is highly interactive, the intelligence is delegated almost entirely to LSEQ Runs/Stops/Aborts several independent asynchronous concurrent event tables on LSEQ request Sends event(s) on LSEQ request Sends event(s) such as “post-mortem” and “injection- warning” on external hardware trigger Observe that most data on the LHC timing cable comes from the outside world Looks much closer to the LEP than to other cycling machines in the LHC Injector Chain [LIC] Almost completely decoupled from LIC

6. 6 Pilot the LIC for LHC Filling

7. 7 1Hz repeated information on the LHC GMT cable By information I mean data transmitted both by events and by the LHC telegram at 1Hz The telegram is a snap shot taken each basic-period, that is each UTC second Events arrive asynchronously and can be subscribed to The event payload contains relevant information for that event ** nHz - means distribute at n Hertz AT- means distribute at arrival time & at nHz ** Payloads carry information useful for the event. Payloads may be set to any value by the LSEQ in an event send request. This mechanism is one way to trigger power-converter equipment groupings where for example the Start-Ramp would control which equipment is actually started. This usage is between OP and the equipment specialists

8. 8 Information on the LHC GMT cable BTC1 & BTC2Circulating Beam Type (per Ring) – 1Hz These values will change soon after the injection has completed and will be distributed as soon as the results of the injection are known. This value must be fully controlled through the API as it is impossible for the timing system to know whether the injection has been successful or not. It is controlled by LSEQ based on measurements taken in the LHC ring. The beam type is a 16 bit encoding of… The Number of circulating bunches. Range 1..2808 The Bunch intensity. Example 4 x 10 11 The Beam Current The Bunch Spacing. Values 25ns or 75ns The Particle Type. Values Protons or Ions

9. 9 Information on the LHC GMT cable RF-Injection-Parameters These parameters are distributed from the API with very low latency, they are controlled by the LSEQ. These parameters must be established in the RF system 450ms before the first CPS batch is extracted towards the SPS. They must also remain stable for at least one second after the SPS extraction towards the SPS has taken place. All these parameters are RESET to zero one second after the SPS Beam-Out event or whenever the LHC Mode is not “Filling”.

10. 10 Establish RF injection parameters 450ms before first CPS extraction

11. 11 Information on the LHC GMT cable RF-Parameters BTNINext injection Beam Type – “AT” Obviously the next injected beam type is determined by the settings in the injector chain and by nothing else. The LSEQ may request a certain type of beam to be injected, but if the requested value does not correspond to the actual beam type being provided by the injector chain, then the request can not be fulfilled and no injection can take place. This value is thus inherited from the injector chain BKNINext injection RF Bucket –“AT” There are 35640 RF buckets around the LHC ring. It is essential that this parameter is established before RF re- synchronization starts between the CPS and the SPS RF systems, namely 450ms before CPS extraction towards the SPS RNGINext injection Ring –“AT” This parameter determines the value of the SPS beam destination in the DEST group of the telegram. Various ways to do this are possible. Its an OP decision. AT: Arrival Time & 1Hz

12. 12 Information on the LHC GMT cable Safe Beam Parameters The safe beam parameters must be delivered to the users over the GMT. Failure to deliver these parameters correctly and in time may result in a beam dump **. The Beam energy is measured by independent systems and transferred to the LHC central timing where one of them is sent over the GMT. A hardware monitor then compares these energy measurements against the GMT value and removes the beam permit if they differ by a threshold value, this hardware module also has a watch dog. Similar mechanisms exist for the intensities. The SBF is calculated by both systems and compared in hardware. **For example the Beam Energy is received by a CTRV module, with timeout, that delivers this value directly over the P2 VME bus to hardware involved in Beam Transfer.

13. 13

14. 14 Information on the LHC GMT cable Safe Beam Parameters ENG1 & ENG2Beam Energy (per Ring) –“AT” These values are sent over dedicated timing cables from two independent measurement systems for each ring (4 Values in all) both to the LHC MTG and to the hardware monitor module (Controls Interlock SBF Generator CIG). If any of these 3 values differ for a ring, or if they are not transmitted within the watch dog time out period, the CIG removes the interlocks and the BIS then may remove the Beam Permit and dump the beam. INT1 & INT2Beam Intensity (per Ring) –“AT” The beam intensities are measured by BI and sent over dedicated timing cable as with Energy. Any failures will cause the BIS to remove the Beam Permit Flag. The intensity will be encoded as N x 10^10 where N is the value transmitted in the INT parameters.

15. 15 Information on the LHC GMT cable Safe Beam Parameters BPF1 & BPF2Beam Present flag (per Ring) – 10Hz The beam present flags are calculated by the LHC MTG from the energy and intensity values, they are transmitted over the GMT cable with a 100ms period. See SBF below PRF1 & PRF2Beam Permit flag (per Ring) – 10Hz The beam permit flags are simply forwarded from the BIS at 10Hz. A transition of this flag from 1 to 0 provokes a postmortem event to be sent. Failure scenarios need study SBF1 & SBF2Safe Beam flag (per Ring) – 10Hz The safe beam flags are calculated in the LHC MTG. As with the BPF they are transmitted over the GMT with a 10Hz frequency at millisecond 0, 100, 200, 300 …1000. The CIG module compares its own calculated values against that on the GMT with a 100ms watchdog, any differences reset the Beam Permit interlocks.

16. 16 Information on the LHC GMT cable MODELHC Machine Mode – 1Hz The LHC machine mode described in the FS is controlled through the API by the LSEQ and will appear immediately on the LHC timing cable. See ** ** Monitoring this by hardware is not a good idea as it would require VME access to the interlock generator.

17. 17 Information on the LHC GMT cable FNUMFill number – 1Hz This is controlled across the API by the HLLSA and may be used to tag postmortem data. At some point in the mode sequence the LSEQ will decide to issue a new fill number which is subsequently distributed as a 16 bit value at the ready telegram frequency. BPNM Basic-Period Number – 1Hz The basic period of the LHC machine has been chosen to be one second, and to be coincident with the Pulse Per Second. The basic- period number is reset to zero when the Mode changes to Start of pre- injection. Note also that the millisecond modulo is also zeroed at PPS time, so the LHC machine time in the run is defined by BPNM + Millisecond in BP PTY1 & PTY2Particle Type (per Ring) – 1Hz In the distant future it may be possible to collide Ions and Protons. If this is in fact the case, then each Ring could have a different particle type. This value is inherited by the LHC MTG from the injector chain at the time of injection. Logically it could be treated or even be part of the Beam Type.

18. 18 Other events on the LHC GMT Post-Mortem Hardware triggered from the BIS via the Beam Present Flag transitions LHC Injection forewarning Hardware triggered by the SPS extraction forewarning event Other hardware triggered events as needed Events sent on request from the LSEQ Event tables repeating under LSEQ control Ready BP and TGM

19. 19 Implementation decisions Safe beam parameter distribution will be hardware monitored A FESA Class will implement the API for the LSEQ across reflective memory The basic-period will be equated with the UTC second Telegrams will be a snap shot of LHC parameters taken each second Telegram values will also be events that can be subscribed to New approach is needed for the event tables, and best implemented in new hardware design

20. 20 LHC MTG 2.2 G-Bit / S optical link 64Mb Reflective memories CMW Server LHC MTG GMT LHC Clocks: 40.00 MHz GPS clock 1PPS (1Hz) clock Basic period clock Event Tables Safe Params Energy/Ring Intensity/Ring BIS Beam permit Flags External Events LSA High level Sequencer LSA Core FESA LHC API Slave/Master

21. 21 Multi-tasking CTG module Same idea as the BST master card Multiple independent asynchronous tasks running event tables under LSEQ control, looks like a collection of CTG cards all driving the same GMT cable Load/Unload table Run table N times Run table for ever Stop table Abort table LHC GMT MT-CTG 16

22. 22 Multi-tasking CTG module Instruction sample ADDR,Add,11,Reg,Reg,Reg SUBR,Subtract,12,Reg,Reg,Reg LORR,Or,13,Reg,Reg,Reg ANDR,And,14,Reg,Reg,Reg XORR,XOr,15,Reg,Reg,Reg ADDV,Add,16,Lit,Reg,Reg SUBV,Subtract,17,Lit,Reg,Reg LORV,Or,18,Lit,Reg,Reg ANDV,And,19,Lit,Reg,Reg XORV,XOr,20,Lit,Reg,Reg WDOG,Watch Dog,25,Adr JMP,Jump,26,Adr BEQ,Branch Equal,27,Adr BNE,Branch Not Equal,28,Adr BLT,Branch Less Than,29,Adr BGT,Branch Greater Than,30,Adr BLE,Branch Less Than Equal,31,Adr BGE,Branch Greater Than Equal,32,Adr BCR,Branch Carry,33,Adr

23. 23 Multi-tasking CTG module 16 Task Control Blocks typedef enum { ILLEGAL_OP_CODE= 0x01, ILLEGAL_VALUE= 0x02, ILLEGAL_REGISTER= 0x04, WAITING= 0x08, STOPPED= 0x10, RUNNING= 0x20, } Task_Status; typedef enum { EQ= 0x1, /* Last instruction result was Zero */ LT= 0x2, /* Last instruction result was Less Than Zero */ GT= 0x4, /* Last instruction result was Greater Than Zero */ CR= 0x8, /* Last instruction result Set the Carry */ } Processor_Status; typedef struct { unsigned longPc;/* Program counter */ Task_StatusTaskStatus;/* Status of Task, see above */ Processor_StatusProcessorStatus;/* Processor status word */ } Task_Block;

24. 24 Master Slave configuration

25. 25 Putting it all together 3 Sequences [PIN] loaded in LIC CBCM Pilot Intermediate Nominal LIC Beams for LHC require the mode to be filling to execute, otherwise they spare ** The LIC telegrams are extended to contain the Beam-Type **This may not be the best way, anyway this is an operational decision. There are many ways Sequences and Normal/Spare can be used

26. 26 Nominal Filling 635 2395 SPS Extraction 18 basic-periods = 21.6 Sec ~3.6 Sec CBCM-Forewarning Establish Batches Request Batch 2 Batch 3Batch 4 Batch 1 7.2 Sec 14.4 Sec Last possible moment TCLP For PSB Injection Bch3 Linac 1S LHC bunch Test Bunch Quality Go/No Go Last possible moment Bch3 For CPS Extraction to D3-Line Next LHC RF Bucket established 3030 3S Nominal LHC injector filling CBCM OSD

27. 27 Simple Use Case: Fill LHC Pilot-Intermediate-Nominal [PIN] 1.SPS operator selects CBCM Sequence PIN This is done in the usual way using the Sequence Manager, this action determines the Beam-Type. All LHC beams are in spare 2.LSEQ Selects mode = Filling Until the mode is set filling, all LHC beams are spared, probably the SPS is in economy mode (There are other possibilities) 3.LSEQ Requests PIN Request: Beam-Type, CPS-Batches, LHC Ring, RF Bucket If the LSEQ were to ask for another beam type than what the LIC is programmed for an error is returned. Also it must ask in good time else an error is returned and no beam is delivered. 4.CBCM delivers PIN The BIS is in charge of the transfer and suppresses the LHC injection if needed

28. 28 Implications for controls Very little or no classic multiplexing Basic-Period is 1S This means that we can get the time since the start of the pre- injection mode I.E. LHC Machine-Time by taking the BPNM and adding the millisecond modulo to it No cycles or super-cycles For cycle stamps we can use the basic-period stamp instead as a way to stamp acquisitions No meaning for the USER telegram group We can have a USER group, but it would probably always contain the same value Payloads do not contain the USER, but they are not zero, they contain corresponding values like Energy, Mode, Beam-Type, Intensity, Power Converter groups, Flags etc. Telegrams still exist, but it’s better to subscribe to the event if you need it as soon as it changes The LHC timing will be implemented in its own TGM network with its own set of Ctims LHC central timing is completely driven from LSA

29. 29 Schedule By January 2007 Running prototype ready for testing installed No master/slave capability initially LIC Part may not be fully functional Extensive reliability tests must be done over months Ref: LHC Slow Timing System – High Level Operational Requirements Functional Specification [Draft] Mike et All