1. 1 BROOKHAVEN SCIENCE ASSOCIATES Control System Overview L. R. Dalesio EPICS Collaboration March 11, 2008

2. 2 BROOKHAVEN SCIENCE ASSOCIATES Outline Requirements Standard Control System Components Development Opportunities Embedded Device Control Client/Server Architecture for High Level Applications Relational Database and tools for all configuration parameters Near Term Plans Scope of Work and Budget Concluding remarks

3. 3 BROOKHAVEN SCIENCE ASSOCIATES Control System Requirements – 1 of 3 Bunch Length 1-40 psecs 2.6 usec ring revolution Top off every 1 minute Top off bunch train 140-300 nsec Top off damping time 10-50 msecs (no extraction) Synchronous reading/writing (latency?jitter?future settings?rate?) Manual control of orbit trims, quadrupoles, sextupoles, and insertion devices are asynchronous ~10 Hz write/read is suitable for “ turning knobs ” for a power supply 5 Hz updates to operators of up to 1000 chosen parameters Archive up to 6000 parameters at a rate of 2 Hz continually What is done elsewhere? Must scale to support 150,000 physical I/O connections and 400,000 computed variables 99.99% availability 24/7

4. 4 BROOKHAVEN SCIENCE ASSOCIATES Control System Requirements – 2 of 3 Transient Recording Take coherent turn by turn orbit data for up to 800 channels 1024 turns Latch the last 10 seconds of data from all parameters in the storage ring Others? Beam line needs 1 msec archiving over 1 minute for temperatures and positions Provide data for all control aspects 5 KHz RF Feedback on beam phase 10 kHz orbit feedback, (100 usec loop time) 300 BPMs (10 per cell) 2 * 120 Corrector PS in 90 I/O Controllers (IOC) 20 msec equipment protection mitigation (too long?) 1 Hz model based control 10 kHz power supply read backs triggered from timing sys 10 ’ s of Hz Data Collection for RF loop correction. 80 psecs pulse to pulse timing jitter. During top off, some beamlines will need 1.1 - 1.8 psecs of timing jitter Machine physics thinks this number should be < 2 psecs Provide electron detector as event for beam line?

5. 5 BROOKHAVEN SCIENCE ASSOCIATES Control System Requirements – 3 of 3 First need dates 2008/03 – vibration tests on girders done using stand alone software. Some vibration sensors will be used online. 2008/06 – support libera BPM evaluation Where do the rest of the diagnostics fit Where does the conventional facility demonstrator fit Where does the power supply test stand fit 2008/08 – X-9 upgrade for NSLS beam line 2008/09 – low level RF lab support 2009/06 – support water systems 2009/09 – support thermocouple measuring 2009/09 – support vacuum test stands 2010/03 – support high power RF Control Room Must be able to be relocated Control room in the LOB for commissioning Control room re-located with accelerator team for operation

6. 6 BROOKHAVEN SCIENCE ASSOCIATES Nomenclature Standard Psy:PI-Ssy:SI-Tsy:TI Sg:SgI-SD Control System Naming Examples S:C30-VA:G1 Pos:1-Opn S:C20-FE:B Pos:1-Cls S:C20-VA:G1 E:1-RB S:C20-VA:G4 S:C24-RF:G1 S:C24-RF:G1 P S:C24-VA:G1 P System Device Signal Comment cell 30, vacuum, girder 1, sector gate valve A (upstream), open cell 20 BM front end fast valve 1st sputter ion pump at C20 girder #1 2nd aluminum chamber at C20 girder #4 rf shielded bellows at C20 girder #4 TMP station at 2nd SC cavity insulating vacuum cold cathode gauge at 2nd SC cavity insulating vacuum cold cathode gauge at 2nd SC cavity beam vacuum

7. 7 BROOKHAVEN SCIENCE ASSOCIATES Control System Software Standards EPICS based Embedded Real-Time Operating System choices: RTEMS which is in use at LCLS, Spear, and CLS or VxWorks which is in use at APS, SNS, Diamond, and SLS. Linux Workstations Use of standard EPICS engineering tools: Extensible Display Manager, Channel Archiver, Striptool, Alarm Handler Evaluate the possibility of participating in the CSS development Visual Database Configuration Tool w/ Modifications for Table Entry Use of physics applications: Matlab Middle Layer Toolkit (MMLT) which is in use at LBL, Spear, Diamond, SLS, CLS, and ASP, and eXtensible Accelerator Language (XAL) which is in use at SNS and Elegant which is in use at APS. Use of Beam Line Applications: Matlab in place of SPEC, BluIce, IDL, SynApps

8. 8 BROOKHAVEN SCIENCE ASSOCIATES Control System Architecture Shared Memory Ethernet – EPICS Channel Access Protocol Events/ / Timing Data PLCPLC CNETCNET I/OI/O Field I/O ….. Instrumentation Ethernet ALSALS NETNET Field I/O ….. PLCS,Slow I/O, High Reliability, Low Accuracy, High Density Vacuum, PPS, MPS, Non FOFB PS, Cryo., Facility control CPUCPU EVREVR I/OI/O I/OI/O MTRSMTRS Read Remote Ethernet and serial devices. High density IO, Motor Control Position Control, Image Acquisition, Non FOFB Diagnostics Operator stations: Displays, Archiving, Alarm Management, Strip charts, Save/Restore Utility CPUCPU EVGEVG C EL L Timing Master CPUCPU EVREVR C EL L BPM IOCs CPUCPU EVREVR C EL L PS IOCs BPM PS Fast Equipment Protection Signal

9. 9 BROOKHAVEN SCIENCE ASSOCIATES Control Hardware Standards for Test Stands Dell Linux development workstations PLC Solutions Allen BradleyControl Logix Allen BradleyFlexIO Building Automation ALSNeeds fast Ethernet based interface board VME crates Rittan 4 slotVME64x-2U4S-PS300C-SM4,395.00 Rittan 7 slotVME64x-4U7S-PS900C-SM5,510.00 Weiner 9 slotVME195xPO6,050.00 Weiner 21 slotVME6023/611_JL7,271.00 CPU Boards MotorolaMVME 55003,000.00 PP4103,500.00 Motor Controllers Hytec8601 with motor driver32 axis~1,000/axis Newport XPS Intelligent motion controllers

10. 10 BROOKHAVEN SCIENCE ASSOCIATES Areas For Technical Development Embedded Controllers need an open standard for high speed, deterministic functions. Work with other labs and board manufacturers to develop one. High Level Applications currently tie together functions through data or file structures. To make the components of High Level Applications modular and distributed, a client/server architecture is needed. Relational Databases support data management through the life of a project. Development of adequate tools to enter and report this data is required early.

11. 11 BROOKHAVEN SCIENCE ASSOCIATES Control Hardware For Beam Synchronous Applications 2 GB Shared Memory – Communicates Cell Data @ 20 usecs Ethernet – EPICS Channel Access Protocol Events/ / Timing Data – Beam Trip Event, Synch Data Event CPUCPU EVGEVG C EL L Timing Master Physics Applications: Correlation plots, orbit calculations, beam dump data CPUCPU EVREVR C EL L BPM IOCs CPUCPU EVREVR C EL L PS IOCs BPM PS Fast Equipment Protection Signal Compute Fast Orbit Feedback

12. 12 BROOKHAVEN SCIENCE ASSOCIATES Embedded Controllers - Per Cell BPM Controller RF Clock Distribution 45 MHz Fiducial Distribution 1 Hz BPM Controller BPM Controller BPM Controller BPM Controller BPM Controller 100 MB ENET PS Controller PS Controller PS Controller PS Controller VME I/OC EPICS Core Controller VME Interface VME I/OC EPICS Next Cell Core Controller VME Interface VME I/OC EPICS Prev Cell Core Controller VME Interface 2 GBit ENET +T0 + - Settling time on BPMs +0.0 usecs - BPM to Compute Controller 384 bits = 64 bits * 6 BPMs 4.9 usecs = 384 bits over 100 MBit enet +4.9 usecs - Compute Controllers to each other 10,290 bits = 30 nodes * 384 bits 10.5 usecs = 10,290 bits over 2 GB enet +15.4 usecs – Compute local matrix 0 usecs + 15.4 usecs - Communicate t Power Supply Controllers 24 = 4 PS * 4 bytes each 3.5 usecs +18.9 usecs – loop complete settling time for magnets communicate diagnostic waveforms etc… +200 usecs – start again

13. 13 BROOKHAVEN SCIENCE ASSOCIATES Embedded Controllers - Approach Develop a prototype cell controller: Redundant 2 Gbit communication paths for peer to peer communication RF timing signals Verify communication and timing jitter meet requirements Develop the interface from the cell controller to a processor for integration into EPICS In parallel: Develop the inexpensive device controller with redundant 100 MBit controllers Develop the 100 Mbit receive and transmit circuits for the Cell Controllers Develop the EPICS interface to the Cell Controller

14. 14 BROOKHAVEN SCIENCE ASSOCIATES Embedded Controllers - Status FY 08 Purchase order in place with LBL (Alex Ratti and Larry Doolittle) Develop a prototype cell controller: –Redundant 2 Gbit communication paths for peer to peer communication –RF timing signals –Verify communication and timing jitter meet requirements Develop the interface from the cell controller to a processor for integration into EPICS FY 09 Develop the inexpensive device controller with redundant 100 MBit controllers Develop the 100 Mbit receive and transmit circuits for the Cell Controllers Begin integration of these device controllers –Libera –Power supply control –LLRF control

15. 15 BROOKHAVEN SCIENCE ASSOCIATES L attice O ptics Beam Resp. Matrix ( S ) Model Data Measured Orbit Differences State Data Application / Family Control System / Channels I/O Controllers Beam Resp. Matrix Diff’s Optics Deviations EPICS Client/Server Gradient Errs & Corrections Name Mapping Mid Level Model Client/Server Configuration Parameters Physics Applications Conversions Need to develop Existing Need to port Computed Data High Level Applications – Client/Sever

16. 16 BROOKHAVEN SCIENCE ASSOCIATES High Level Applications - Approach Install an operational NSLS II simulation using Matlab Middle Layer Toolkit Define an interface library for physics applications to communicate to the configuration parameters, calibrations, model data, and measured data. Port some subset of the available packages (Matlab Middle Layer Toolkit, Elegant, XAL, Dimad, and SAD) onto the interface library one application at a time. Develop a client/server implementation of the interface library. Use an RDB to store configuration parameters and calibrations.

17. 17 BROOKHAVEN SCIENCE ASSOCIATES Relational Database Support all configuration data Provide tools as they are needed throughout the project. Make the RDB the source of all configuration data and derive it from there for all uses.

18. 18 BROOKHAVEN SCIENCE ASSOCIATES Relational Database - Approach Use the IRMIS database as the basis for our component and wiring database which is currently in use at APS, SNS, CLS, Diamond, and SLS. Extend the database to support Optics early through a contract with Tech-X in conjunction with our physicists and control group. This contract includes a name mapper and lattice input / report tool Develop tools through contracts, collaboration, and the hire of a second RDB expert. We are preparing SOWs for Naming Tool and Wiring Input Schedule the time and put in the project rigor to use the RDB and tools as new portions of the project are needed.

19. 19 BROOKHAVEN SCIENCE ASSOCIATES WBS Dictionary: Hardware and software for the global controls of the linac, booster ring, transport lines, storage ring, network, relational database, data archiving, timing, fast feedback, and physics application support. WBS 1.03.05 Global Controls System ~ 10% of Accelerator WBS Description Direct $ FTEs/yrTotal FTE CostsTotal Material 1.03.05/01 – 1.03.05.09 Apps: Diagnostics, RF, Power Supply, Vacuum Subsystem Integration of: Insertion Devices, Beam lines, Safety Systems, Facility75.3 M1.9 M 1.03.05.12 Timing1.51.2 M0.9 M 1.03.05.11 & 1.03.05.13 High Level Applications64.8 M0 1.03.05.10 Accelerator Control Room75K0.5 M 1.03.05.11 Network.50.3 M1.3 M

20. 20 BROOKHAVEN SCIENCE ASSOCIATES Near Term Plan Start R&D Contact Nearly in place for development of open-source embedded device controllers with LBL team (Larry Doolittle and Alex Ratti) RDB server is purchased and we are ready to Install IRMIS Tech-X contract ready to go out the door to extend IRMIS to support Optics Position open for someone to Develop RDB tools for entry and reports Installed NSLS II Simulation through IOCs Installed Matlab Middle Layer Toolkit for Commissioning Two positions in place to Start Development of API for High Level Applications (one starts in May, the other position should be filled by March) Make Preliminary Hires RDB job ad is placed. A candidate has been indentified and could start by March 15 th.. High Level Applications Engineers – Guobao Shen starts May 10, 2 nd position starts by March 15 th.. 2 positions are open for project engineers 1 position is open for an embedded systems expert Completed preliminary design Completed extensions to Visual DCT for Spreadsheet based input

21. 21 BROOKHAVEN SCIENCE ASSOCIATES Concluding Remarks Global requirements are complete Equipment control S/W and H/W infra-structure is selected. A collaboration of LBL and SLS EEs along with NSLS II team are prepared to begin the open-source, embedded device controller development RDB tables and tools have begun building on the success of IRMIS. High Level Application environment is selected. The Diamond simulation running under EPICS is installed along with the Matlab Middle Layer Toolkit as a starting point at KEK by Guabao Shen. The API for High Level Applications is being developed in conjunction with MMLT, SDDS, and XAL authors. Staff Acquisition is a critical activity and we have made good progress.