1. The CERN LHC central timing A vertical slice Pablo Alvarez Jean-Claude Bau Stephane Deghaye Ioan Kozsar Julian Lewis Javier Serrano

2. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

3. R1 LHC CNGS Linac PSB CPS SPS D3 Dump TI8 Dump TI2 Dump SPS Dump R2 LHC TCLP The LHC Proton Injector Chain Strongly time coupled

4. CERN accelerator network sequenced by central timing generator ….…. PSB CPS SPS LHC Experimental area Experimental area Experimental Area

5. CBCM Sequence Manager

6. PSB1 PSB2 PSB3 PSB4 CPS Batch 1CPS Batch 2CPS Batch 3CPS Batch 4 SPS Cycle for the LHC SPS injection plateaux The LHC Beam LSA Beam request: RF bucket Ring CPS batches Extraction Forewarning LHC Injection plateaux Injection Extraction The LHC timing is only coupled by extraction start-ramp event Extraction Forewarning

7. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

8. Timing Distribution Overview RS485 drop nets distribute timing around the CERN accelerator complex Long distance transmission over optical fibers One timing network for each accelerator Hundreds/Thousands of timing receiver modules distributed around the complex One timing generator drives one timing network

9. Timing Frames Timing frames are Manchester encoded at 500kbit using a 1MHz clock. Each frame carries 32 data bits, parity, start and stop bits. One frame transmitted each 125µs, 8 per millisecond. The frame data is broken into bit fields –4 Bits Accelerator [A] –4 Bits frame Type [T] –8 Bits Code[CODE] –16 Bits Payload [PAYLOAD] Some frames are recognized by the hardware and cause special treatment –Two UTC frames carry the time of day in their payload –Millisecond frames are always sent in phase with the PPS –Telegram frames are stored in double buffers –Event frames cause counters to be loaded and triggered and may produce bus interrupts Millisecond ATCODEPAYLOAD 8 frames

10. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

11. LHC Central Timing Generation CBCM Controlling Injector Chain Gateway FESA API LSA Timing Service SPS destination request R1,R2 CPS Batch Request 1,2,3,4 LIC Timing HIX.FW1K LHC Timing Inhibits Requests Interlocks SIS TI8/TI2 & SPS Dump Inhibits LHC Fill Requests: Bucket Ring Batches Master ON/OFF CTR LHC Fill Requests: Bucket Ring Batches Reflective Memory Link CBCM Sequence Manager LIC Sequence TI8/TI2 Dump Request TI8/TI2 Dump LHC User Request LHC User Normal Spare LSA changes Allowed LSA Master SEX.FW1K SPS.COMLN.LSA_ALW Central Timing Overview

12. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

13. LHC Gateway Implements FESA API 2Gbit/S Token ring VMIACC-5595 Single Mode Hub 64Mbyte VMIPMC-5565 Reflective memory LHC Timing generator A LHC Timing generator B Reflective Memory Reflective memory: A and B must always be in the same state. If no restrictions for switch over Protects token ring Identical except ID event

14. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

15. GPS One pulse per Second GPS Symmetricom XLI PLL One pulse per Second Phase locked 10MHz Basic Period 1200/900/600 ms Advanced (100us) One pulse per Second Synchronized 1KHz (slow timing clock) Phase locked 10MHz Phase looked 40 MHz Event encoding clock 40MHz PLL Synchronization module in each timing generator crate RS485 Timing MTT Multitask Timing Generator MTT UTC time (NTP or GPS) Event tables External events UTC Time and GPS CERN UTC Time Set once on startup & on Leap Seconds RS485 Timing CTR PPS 10 MHz 1 KHz 40MHz Delay Control System CERN UTC Time 25ns steps Timing receiver Symmetricom CS4000 portable Atomic Clock

16. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

17. Multitask Timing Generator MTT Hardware multitasking for 16 tasks –32 local registers per task –218 Global registers –6 Memory mapped IO registers Timing frame out register VME P2 in register … Host processor access to all registers Implements general purpose CPU –Op-codes are triadic: AND SrcREG,SrcREG,DstREG – AND 0x7,VMEP2,TMP Arithmetic and logical Move indexed, literal, register Wait value, relative Conditional branch Interrupt host Tasks defined from host via Task Control Block –PC –PC Offset –Processor Status Word Command and status registers allow host access to running tasks

18. MTT hardware module See: The LHC central timing hardware implementation P. Alvarez, J. Lewis, J. Serrano CERN, Geneva, Switzerland This conference

19. MTT External Events Task

20. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

21. Controls Timing Receiver CTR Hybrid PLL Delay Frame Decoder TGM UTC Content Addressed Memory - Triggering Counter Configurations Loader 22Bit 50MHz Counters Bus Interrupt Output Timing 40MHz Lookup Load Trigger Start Previous External Event Clock Previous 40MHz harmonic External Modes Once Multi-pulse Burst Trigger Event frame Wildcard Telegram Action Output Bus interrupt Output TTL/TTL bar Pulse width CTRV – CTRI – CTRP formats

22. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

23. Reads thresholds @1kHz-24Bit 10 8 Safe Machine Parameters Controller for LHC Energy A I_beam1 & 2 Reads status Energy B BEM BCT “A” BEM Management Critical Settings (CTRV) CTRV CTRx LSA BCT “B” EXP Line driver LHC Timing Generator SMP @ 10Hz 16Bit 10 10 (Flags, E & Int.) BLM CTRV BLM CTRV BIS CTRV Kickers Timing Network Events, UTC, & Telegrams (including SMP) Safe Machine Parameters Distribution BIS CTRx EXP Flags TTL Hw Output If length > 5m

24. Talk layout LHC Injector Chain Timing Timing Distribution Overview Central Timing Overview Reflective Memory UTC Time and GPS The Multitask Timing Generator The Controls Timing Receiver Module Safe Machine Parameters Hardware Checking LHC Software Architecture and Event Tables

25. LSA and FESA The FESA API is implemented on the LHC timing gateway Accesses timing generators across reflective memory Implements –Load or Unload event table –Get running tables list –Set event table run count and synchronization event –Stop or Abort event table –Set telegram parameters –Send event –Read status of tasks and MTT module

26. Garbage Collect Load object Initialize TCB Run task Assemble task Translate and Merge HX.Start-Ramp 1.01 0x5 HX.Start-Freq 0 0x05 waitr MSFR,1001 movv 0x14020005 EVOUT movv 0x14030005 EVOUT Event Table Processing FESA API LSA Event Table Table Compiler Translation Event Table template Table Task Assembler Object Code Reflective Memory Task Loader MTT Program Memory Hot Standby Data Process

27. Conclusion –Event tables model LHC machine processes. –Reuse of existing timing boards was facilitated by using FPGAs and writing new VHDL. –The LHC timing is monitored by hardware. –Reflective memory has increased reliability.