CLS: Control System E. Matias, R. Berg, G. Wright, T. Wilson, T. Johnson, R. Tanner and H. Zhang Canadian Light Source University of Saskatchewan Invited Talk – Particle Accelerator Conference 2005 Knoxville, Tennessee

CLS Control System Heritage Saskatchewan Accelerator Laboratory (SAL) operated from the late 1960s until –Control system evolved from PDP-8 -> PDP-11 -> VAX -> NeXT and Sun workstations. –IO was based on CAMAC with two CAMAC data highways. –Some Micro84 PLCs. Control System was locally developed running on BSD UNIX. Was it open source? Yes, but only because so much of it was locally developed. What to do with the grass field?

CLS Control System Heritage 1999 March 31 funding for CLS was approved. Nuclear physics program was discontinued. The existing Linac would need to be reconfigured and refurbished. Linac Controls: –CAMAC hardware would need to be replaced. –Power supplies would need to be upgraded. –RF control would need to be redesigned. –The old computer hardware would need to be replaced. We need to make some design choices....

CLS Control System Principles System design based on highly distributed control. Extensive use of single board computers (originally used in SAL). Target lifetime of 15+ years. Data communication over Ethernet when possible. System must be user-friendly. The accelerator and beamline systems must be maintainable by a small team. Reliability and availability of beam are critical to the success of the facility. Building an open source control system was not the initial goal, it was the outcome. Accelerator complex must be complete by Dec and the first phase of beamlines by Dec The project must come in on budget.

The options: (1) EPICS or (2) Isagraph/Virgo. EPICS was selected, since it had: –large built up accelerator and beamline user community; –availability of suitable drivers and utilities; –credibility with the CLS user community; and –good design. EPICS Extensions selected include: –EDM, –Accelerator Toolbox, –Gateway and –Data Archiver. EPICS extensions that were locally developed: –assortment of drivers, –IOC Auto-Save-Restore, –simple beamline scanning program, and –SQL Alarm Management Database. Selection of a Distributed Control Platform

The options: (1) Sun or (2) Linux. Linux was selected, since it had: –better hardware availability, and –fairly equivalent reliability levels. EPICS Extensions selected include: –EDM, –Array Display Tool, –StripTool, and –Knob Manager. We are now starting to deploy touch screens running Linux/EDM. Selection of an Operator Workstation Platform

PLC hardware extensively used –Telemecanique Momentum –Siemens S7/300 –Siemens S7/400 –Siemens S7/400 F (safety critical applications) Software Written using IEC Programming Language primarily Graphical Function Block Notation PLC Control

Selection of a Hard Real-Time Operating System The Options: (1) RTEMS and (2) VxWorks. RTEMS was selected, since it had: –good experience from SAL, –additional flexibility with single board computers, and –high level of reliability. IOCs are CLS/SIL embedded controllers (approx 150) based on the MC MHz. Processor. Pros and Cons: –No dynamically loaded libraries; must be linked prior to download. –Large number of IOCs (separation of function but more points of failure).

The options: (1) Matlab, (2) SciLab, or (3) root. Matlab was selected primarily because of the availability of the accelerator toolbox and staff experience. Matlab is commercial, the accelerator toolbox is open source. Software originates from ALS and SPEAR III. Augmented with other CLS specific utilities. Also being used as a commissioning tool for beamlines. Special care is required to maintain consistency with other parts of the control system. Selection of an online scripting/analysis package

Single board computers (EPICS/RTEMS) used for: –stepper motors, –power supply control, –vacuum equipment monitoring, –radiation monitors, and –other RS-232 devices. PLC hardware/software used for machine protection. Industrial PCs with VME used for diagnostics. Linux servers used for high- level control, network services and EPICS/PLC interface. Implementation

Linux and VME Using VME hardware connected to a Linux PC. SIS1100 PCI card fiber optic link SIS3100 VME module Maps VME backplane to IOC memory. Advantages: –PC can be physically separated from VME crate. –More than one VME crate per PC. –Multiple applications can access the same crate. –High throughput 25 to 80 Mbytes/sec block transfer. Work ongoing on RTEMS support.

Provides fiber optic signal distribution of triggers. VXI based hardware IOC running EPICS on RTEMS. Operator Interface implemented using Glade. Glade was selected for the table and file handling capabilities. Timing System

Beamline Controls are based on the same software and hardware as the accelerator systems. Each beamline is on a separate virtual network. The EPICS Gateway provides links between the different networks. Matlab is used for scripting. Beamlines

Interface to Administrative Systems Administrative (financials and work- management) databases are based on MS- SQL. freeTDS was used to provide an interface into these systems. The control-system is a source of alarm-data that can be used to trigger preventative maintenance activities. This integration is still a work in progress.

VLANs for: each beamline, machine control, development, office, visitors VME Crate (Reflective Memory) MicroStep EROC IOC RTEMS Field Dev. RS-232 Devices OPI Linu x IOC Step Controller RTEMS Motors MicroStep OPI Linu x OPI Linu x Touch Panel OPI Linux Network Server (bootp, dhcp, auto restore) Linux Data Archive Server Linux Alarm Server MS-Win MS-SQL Server MS-Win PowerEdge IOC Linux PS Boards IOC RTEMS Power Supplies EROC IOC RTEMS Field Dev. Ethernet Devices PLC & GPIB Field Dev. MagnetsMotors 1Gig Bridge IOC Linux Field Dev. Profibus PLC System Architecture

Lessons Learned The use of open source did not compromise reliability or availability. Configuration management is critical to the success of the system. Layering of different software packages requires care to ensure the system is reliable. Additional flexibility in available in managing upgrades compared to commercial software. Dependence on suppliers continuing to support the product is not an issue. Our, building automation system is a case in point, where the vendor has discontinued the product and we are now replacing it with EPICS and PLC hardware.

Winter at the CLS