1. the FIRESTORM project Real-Time distribution of
magnetic field values using White Rabbit
the FIRESTORM project
Maciej Lipinski
BE-CO-HT
BE-CO Technical Meeting

2. Agenda Context of today’s presentations: White Rabbit
Distribution of magnetic field value
Current BTrain system
New BTrain system 2

3. Agenda Context of today’s presentations: White Rabbit
Distribution of magnetic field value
Current BTrain system
New BTrain system 2

4. What is a White Rabbit Network?
Ethernet network
White Rabbit
Ethernet Network
Switch
Switch
Switch 3

5. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement 3

6. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement 3

7. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
Initially created as next generation network for control & timing (GMT)
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement 3

8. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
Initially created as next generation network for control & timing (GMT)
Versatile solution for control, synchronization, acquisition
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement 3

9. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
Initially created as next generation network for control & timing (GMT)
Versatile solution for control, synchronization, acquisition
Applications: China,
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement
Large High Altitude Air Shower Observatory 3

10. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
Initially created as next generation network for control & timing (GMT)
Versatile solution for control, synchronization, acquisition
Applications: China, Finland, Netherlands
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement
Large High Altitude Air Shower Observatory
SuperGPS 3

11. What is a White Rabbit Network?
Time
Control data
Ethernet network with unique features:
Deterministic data delivery
Sub-ns accuracy of synchronization
Real-time network based on standards
Initially created as next generation network for control & timing (GMT)
Versatile solution for control, synchronization, acquisition
Applications: China, Finland, Netherlands and many others (see) including CERN
White Rabbit
Ethernet Network
Switch
Switch
Switch
sensors
actuator
measurement
Large High Altitude Air Shower Observatory
SuperGPS 3

12. White Rabbit applications at CERN
Applications so far:
CERN Neutrinos to Gran Sasso (CNGS)
LHC Instabilities (LIST)
Synchronization of AD with LIC
WR-BTrain
Applications possibly in the future:
OASIS (based on WRXI)
General Machine Timing (GMT)
Beam Synchronous Timing (BST)
RF distribution
ProtoDUNE 4

13. White Rabbit applications at CERN
Applications so far:
CERN Neutrinos to Gran Sasso (CNGS)
LHC Instabilities (LIST)
Synchronization of AD with LIC
WR-BTrain
Applications foreseen in the future:
OASIS (based on WRXI)
General Machine Timing (GMT)
Beam Synchronous Timing (BST)
RF distribution
ProtoDUNE 4

14. White Rabbit applications at CERN
Applications so far:
CERN Neutrinos to Gran Sasso (CNGS)
LHC Instabilities (LIST)
Synchronization of AD with LIC
WR-BTrain
Applications foreseen in the future:
OASIS (based on WRXI)
General Machine Timing (GMT)
Beam Synchronous Timing (BST)
RF distribution
ProtoDUNE 4

15. Agenda Context of today’s presentations: White Rabbit
Distribution of magnetic field value
Current BTrain system
New BTrain system

16. Distribution of magnetic field value
Operation of circular accelerators is based on
accelerating RF cavities
bending magnets
Changes of frev(t) and B(t) must be coordinated
In CERN accelerators, B(t) is the master value
I(t) is increased to achive desired B(t)
frev(t) follows changes of B(t)
Value of B(t) depends on I(t), their dependency is not always easy to model
The knowledge of the value of B(t) in real-time is essential for longitudinal and transverse beam control
The value of B(t) is distributed using the BTrain system in LEIR, PSB, PS, SPS, AD and ELENA
Accelerating
RF cavity
frev (t)
Bending
magnet
B (t)
Magnets
power
converter V
I(t) 5

17. Agenda Context of today’s presentations: White Rabbit
Distribution of magnetic field value
Current BTrain system
New BTrain system

18. Currently used BTrain system
preset
marker trigger pulse
Measurement:
Peaking strip provides a marker signal: t1
Flux loops provides rate of change of field: Ḃ
Distribution:
B(t):
Pulses of 24V and 1us on 2 coax cables UP / DOWN-train
Each pulse represents a step of 0.1G
Max 500kHz, max slope 5T/s
Bdot
Analogue signal over coax cable
Needed by RF in PS
Marker
Pulse to mark start of integration
Bflat top
time
B(t)
Bflat bottom t1 t2
marker 1
marker 2 (optional)
coil measurement Vcoil ~ Ḃ
UP-train
DOWN-train
preset B(t1) 6

19. Currently used BTrain system
Measurement:
Peaking strip provides a marker signal: t1
Flux loops provides rate of change of field: Ḃ
Distribution:
B(t):
Pulses of 24V and 1us on 2 coax cables UP / DOWN-train
Each pulse represents a step of 0.1G
Max 500kHz, max slope 5T/s
Bdot
Analogue signal over coax cable
Needed by RF in PS
Marker
Pulse to mark start of integration
Why upgrade?
Obsolete measurement technology
Critical electronic components are ageing, unique and unrepairable
0.05G resolution required in PS
Peak slope of 6T/s required by PSB
Voltage spikes and noise on Bdot
Timing problems with marker
Based on non-standard equipment, thus little support from CO
Not uniform across accelerators
Limited diagnostics 6

20. Agenda Context of today’s presentations: White Rabbit
Distribution of magnetic field value
Current BTrain system
New BTtrain system

21. FIRESTORM project to renovate BTrain
Field In REal-time STreaming from Online Reference Magnets
Project is carried out (named) by Magnets, Superconductors and Cryostats (MSC) Group in collaboration with EPC, RF, BI and CO.
Renovated BTrain system, called FIRESTORM:
addresses the shortcomings of the current BTrain systems
consolidates the existing BTrain systems under a common hardware/software platforms
Design principles:
Uniform system across the whole CERN complex
Modern digital off-the-shelf technology and/or CERN-developed components
Industrially produced sensors
Abstraction from machine control system and timing signals (no built-in dependencies upon reset, marker window, special cycle types ...)
Allow operation in // with old systems
MSC Group has chosen for distribution of information in the renovated BTrain system White Rabbit (CO-provided off-the-shelf technology)
Open hardware ? Software ? 7

22. White Rabbit in FIRESTORM (WR-BTrain)
Ethernet
Frame
Distributes digitally absolute values of B and Bdot in Ethernet frames (BTrain frame)
Distributes simultaneously B from different sources
Transmission supports resolution down to G
Allows rates >500kHz and infinitely fast ramps thanks to absolute value
Guarantees latency of max 10us 8

23. WR-Btrain Network 9

24. Some other WR-compatible card, e.g. VXS Switch
WR-BTrain Node
WR-BTrain node
FPGA
Simple VME64x
FMC Carrier
(SVEC)
Simple PCIe
FMC carrier
(SPEC)
VME FMC Carrier
HPC-DDR3
(VFC-HD)
Some other WR-compatible card, e.g. VXS Switch
FPGA
FPGA
FPGA
FPGA
CO-supported
hardware 10

25. support eventually by TE-MSC-MM
WR-BTrain Node
WR-BTrain node
FPGA
User logic
Ctrl Tx B/I-Frame
support eventually by TE-MSC-MM
Tx Streamers
CO-supported
WR PTP core
Simple VME64x
FMC Carrier
(SVEC)
Simple PCIe
FMC carrier
(SPEC)
VME FMC Carrier
HPC-DDR3
(VFC-HD)
Some other WR-compatible card, e.g. VXS Switch
FPGA
FPGA
FPGA
FPGA
CO-supported
hardware 10

26. WR-BTrain Network 11 WR-Btrain Node’s FPGA Tx Streamers
User logic
FPGA
Ctrl Tx B/I-Frame
Tx Streamers
WR PTP core
Switch monitoring
Nodes monitoring
WR-Btrain Node’s FPGA
WR-Btrain Node’s FPGA
WR PTP core
WR PTP core
Rx Streamers
Rx Streamers
Ctrl Tx B/I-Frame
Ctrl Tx B/I-Frame
User logic
User logic 11

27. WR-BTrain Network and its config in PS12

28. WR-BTrain status 13 PS: Installed since 2014
Clients: POPs, RF, BI, labs for development
Tested with RF only in 2015 (old B)
Tested with POPS and RF in 2017
ELENA:
Installed since 2016
No legacy BTrain system
Clients: currently LLRF, maybe later power converters
Ongoing tests with beam
LEIR,
Installed
PSB:
Partially installed 13

29. WR-BTrain plans FMC card FMC carrier 14 Deployment plans: PS
2017: progressive testing in operation
2018: operation with production system (new B and WR-BTrain)
ELENA:
2017: operation with prototype
2018: operation with production system
AD:
2017: installation & tests
PSB, LEIR, SPS:
2017/2018 (Run2): installation & test
2019/2020 (LS2): tests with prototype
2021: operation with production system
Technical plans:
WR-BTrain release
Diagnostics (host and SNMP)
Fixed latency
WR as BE-CO service
Legacy output (Bup/Bdown)
Seamless network redundancy
WR-BTrain FMC (I/F on pins)
FMC card
FMC carrier 14

30. Summary BTrain is a critical system in PSB, LEIR, PS, SPS, AD, ELENA
The current BTrain needs renovation to meet future requirements
MSC Group leads this effort within the FIRESTORM project
White Rabbit was chosen as a suitable and CO-provided technology
White Rabbit will soon be critical to the operation of most accelerators
CO-provided White Rabbit services (next presentation) will be essential in BTrain installations
WR-BTrain is an example of a smooth collaboration between MSC, EPC, RF, BI and CO Groups. 15

31. Summary BTrain is a critical system in PSB, LEIR, PS, SPS, AD, ELENA
The current BTrain needs renovation to meet future requirements
MSC Group leads this effort within the FIRESTORM project
White Rabbit was chosen as a suitable and CO-provided technology
White Rabbit will soon be critical to the operation of most accelerators
CO-provided White Rabbit services (next presentation) will be essential in BTrain installations
WR-BTrain is an example of a smooth collaboration between MSC, EPC, RF, BI and CO Groups.
Thank you
Any questions? 15

32. Backup slides 16

33. WR-BTrain status and plans17

34. 18