1. Control Systems for the APTM and GRID
Joao Paulo Martins
Control Systems Engineer - ICS
European Spallation Source ERIC

2. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

3. Layer Architecture of the ESS Control Systems
Introduction
Layer Architecture of the ESS Control Systems

4. Experiment workstations other human interfaces
Layer Architecture of the ESS Control Systems
ESS Main Control Room
Experiment workstations
other human interfaces
System
GUIs
Equipment
GUIs
Physics
Apps
Config.
Tools
Data on
Demand
PSS
GUIs
NICOS
DMSC
High Performance
Data Network
EPICS
EPICS
IOC
EPICS
IOC
EPICS
IOC
EPICS
IOC
EPICS
IOC
Data on
Demand
EPICS
Archiver
Timing System
Event
Generator
Machine Protection
Fast Beam
Interlock System
Personnel Safety Systems
PLC
PLC
Ether
CAT
MicroTCA
System
MicroTCA
System
PSS
PLC
Fieldbus
Device signals
Remote
I/O
Controller
Device
Electronics
Device
Electronics
Controller
Remote
I/O
Remote
I/O
Pump
Flowmeter
Motor
Sensor
Chopper
Power Supply
Switch
Sensor
Sensor
Mech. Movement RF
Proton
Neutron
Temp
Electricity
Magnetic
H2O Flow
He Flow
etc.

5. other human interfaces
Architecture of the APTM/Grid Control Systems
ESS Main Control Room
other human interfaces
System
GUIs
Equipment
GUIs
Physics
Apps
Config.
Tools
Data on
Demand
High Performance
Data Network
EPICS
EPICS
IOC
EPICS
IOC
Data on
Demand
EPICS
Archiver
Timing System
Event
Generator
Machine Protection
Fast Beam
Interlock System
Personnel Safety Systems
Ether
CAT
MicroTCA
System
Fieldbus
Device signals
Controller
Device
Electronics
Motor
Sensor
Mech. Movement
Temp
Current

6. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

7. Prototype tests in Japan

8. Prototype tests in Japan
Installation and remote operation of the APTM/Grid prototype at J-PARC using as much as possible “ESS environment”;
MicroTCA as main component: DAQ and CPU;
Beckhoff EtherCAT for motion control and temperature reading;
EPICS for operation and data collection;
Lot of experience gained;
Robustness of the MicroTCA crate confirmed;
Motion Control IOC interface and operation needs refinements;
EPICS IOCs instances should be easily manageable;
Timing Events
MicroTCA System
APTM blades
Digitizer
Timing
Motion Control Software
Control System Software
APTM grid
External
clients
Motors / encoders
EtherCAT I/O Modules

9. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

10. Hardware Project FMC-PICO-1M4 selected as digitizer;
20 bits current measurement within 1 uA 1 MSPS;
4 channels per FMC;
IOxOS IFC1410 FMC carrier;
ICS supported platform;
No support for CAENELS FMC-PICO-8 yet. [On-going activity with Lodz in-kind];
Needs custom firmware implementation for interlock functions;
Concurrent Technologies MTCA CPU;
Based on 4-core Intel i7 processor (new version based on Intel Xeon improves performance);
Front-panel Ethernet ports enables its use as EtherCAT master;
MRF MTCA-EVR-300
Standard timing component for MicroTCA crates, fully integrated in EPICS;

11. Hardware Project Preliminary proposal of MicroTCA equipment:
One 12-slot 9U MicroTCA crate for all 4 APTM units:
10 slots for digitizer unit, 2 slots for CPU and Timing: 80 analog inputs;
One 6-slot 3U MicroTCA crate for the GRID:
4 slots for digitizer unit: 32 analog inputs;
Alternative proposal:
Separate APTM hardware in different crates;
Offload CPU consumption (although maybe only one is enough);
Local CPUs of the IFC1410 can also be used for data acquisition;

12. Hardware Project Motion Control Hardware Beckhoff EtherCAT modules;
Motion Control Hardware Standards:
EtherCAT master can be implemented using regular Network Interface Card: MicroTCA CPU;
Motors / encoders

13. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

14. Software Project
EPICS IOC for data acquisition and operation control have access to the digitizer and timing cards via PCIe drivers;
Main functionalities: configuration of the timing triggers, analog data acquisition, beam profile measurement (GRID);
EPICS device support layer implemented based on areaDetector framework;
areaDetector will be the common framework for digitizers in EPICS at ESS;
Support for IOxOS cards in areaDetector currently under development;
Single IOC to control timing and data acquisition. Motion Control runs in separate IOC for now – Scenario with single IOC for all systems will be evaluated;

15. Software Project External clients APTM/GRID IOC
Beam profile measurement;
APTM blades measurement;
Acquisition triggers configuration;
Motion Control IOC
Motor record;
Sequencer;
Temperature reading;
AreaDetector
TimeSeries
Plugin
AreaDetector
ROI
Plugin
AreaDetector
Stats
Plugin
Other AD plugins
MRFIOC2
Device Support
ECMC
Open-Source Motion Control
Framework
areaDetector (asyn) device support for IFC1410
Developed at ESS
TSC kernel driver
MRF kernel driver
Etherlab EtherCAT master driver
PCIe
PCIe
EtherCAT

16. Software Project EPICS Environment EEE -> E3
Same concept as EEE (dynamic loading of shared libraries) but different implementation;
E3 build system can accept external code “as it is”;
Strict dependencies: integrator should specify which versions will be used in compilation and IOC startup. No automatic loading anymore;
Standard set of EPICS extensions already supported: autosave, iocStats, recSync (Channel Finder). Standard template for a all these IOC services;
procServ and systemd services used to run IOCs – currently under tests;
EPICS 7: version already supported in E3. The new “qsrv” engine directly maps the DBR types (used in EPICS 3 database files) to new Normative Types;
For more complex data structures – structured PVs – changes on database files are needed;
BI requested a standard template of structured data for the waveforms acquired in digitizers systems: to be done;

17. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

18. Machine Protection Interface
Fast Beam Interlock System (FBIS) Interface:
Data process on the FPGA should trigger interlock functions;
How to do this? FFT? Integrator? Envelope detection? Bad condition should be detected within 10 µsec?
Signals are divided in different digitizer cards. AMC-to-AMC backplane communication needed – avoid CPU;
Support for FMC-PICO for IOxOS FPGA design kit should be relatively simple. Currently under development by in-kind partner (Lodz);
FBIS interface under development: MTCA RTM – to be available in September-19;
IFC1410
AMC Carrier
FBIS RTM
FBIS SCU
FPGA

19. Summary Introduction; Prototype tests in Japan; Hardware project;
Software project;
Machine Protection interface;
Conclusion;

20. Evaluation of the requirements
Current design evaluation
Depends on custom FPGA design to implement the measurement and interface to FBIS. Should be verified by extensive tests, but 10 microseconds is not a big challenge for a FPGA;

21. Evaluation of the requirements
Current design evaluation
Already verified with different IOCs configurations: Struck, AMC-PICO-8, IFC1410 – areaDetector or NDS3-based;
FMC-PICO sampling rate is 1 MSPS. Waveforms for every pulse can respond at 14 Hz on EPICS level;
IFC1410 has 2 banks of 512 Mbytes. Considering 8 1 MSPS (32 bits per sample), we have room to store ~ 16 seconds continuously;
Should be verified;
Sampling at 1 MSPS gives us ~ 100 intrapulse samples. Averaging is allowed;
Sampling at 1 MSPS gives us ~ 5 intrapulse samples. Averaging is allowed;

22. Conclusion
Hardware project (based on IFC1410) still needs some firmware implementations. Integration of the FMC-PICO can be demonstrated until Jun-19;
FBIS functionalities still at an early stage; Interlock conditions can be simulated and tested with current FPGA platforms. Vertical integrated tests with FBIS will need more time – not achievable in Jun-19;
ICS should carry on performance tests with different CPU/AMC configurations to support the decision of how many crates will be used;
Control system software (EPICS IOCs) design already tested in J-PARC; No major changes foreseen – areaDetector as main component;
EPICS 7 IOC should be the goal – achievable by Jun-19;

23. Conclusion
Obrigado!