1. EPICS Overview EPICS Training INFN October 13, 2008 Presented by: Andrew Johnson Based on previous versions by Marty Kraimer, Ned Arnold, John Maclean, Kay Kasemir, Bob Dalesio and probably a few other people too.

2. October 13, 2008EPICS Training INFN2 Sources Main WWW site  www.aps.anl.gov/epics/ www.aps.anl.gov/epics/ Documents/Training This talk taken from the following training presentations  SSRF, APS, PSI, SNS, USPAS2007, USPAS2003  Marty Kraimer, Ned Arnold, John Maclean, Kay Kasemir, Bob Dalesio  The talk from PSI did not list authors.

3. October 13, 2008EPICS Training INFN3 What is EPICS? A Collaboration A Control System Architecture A Software Toolkit

4. October 13, 2008EPICS Training INFN4 What is EPICS? A Collaboration  Began in 1989 between LANL/GTA & ANL/APS (Bob Dalesio & Marty Kraimer)‏  Over 150 license agreements were signed before EPICS became “open source”  Recent EPICS collaboration meetings 100+ Attendees 30+ Institutions 75+ Presentations  List server; tech-talk: the collaboration in action  Collaborative efforts vary Assist in finding bugs Share tools, schemes, and advice

5. October 13, 2008EPICS Training INFN5 What is EPICS? Early Major Collaborators  America ANL, LANL, LBL, ORNL, SLAC, JLAB  Europe DESY, BESSY, PSI  Asia KEK Recent Major Collaborators  America TRIUMF, CLS, BNL  Europe DIAMOND Light Source  Asia/Australia J-PARC, Australian Synchrotron, IHEP, SSRF, TLS Many Other Collaborators/Users

6. October 13, 2008EPICS Training INFN6 What is EPICS? EPICS For EPICS, client and server speak of their Channel Access role i.e. Channel Access Client & Channel Access Server Client Server CA A Collaboration A Control System Architecture  Network-based “client/server” model (hence the EPICS logo)‏

7. October 13, 2008EPICS Training INFN7 What is EPICS? Channel Access clients are programs that require access to Process Variables to carry out their purpose The “service” that a Channel Access server provides is access to a Process Variable* StripToolMEDM Process Variables Process Variables CAS CAC * A Process Variable (PV) is a named piece of data.

8. October 13, 2008EPICS Training INFN8 What is EPICS? Vocabulary EPICS  Experimental Physics and Industrial Control System Channel Access  The communication protocol used by EPICS Process Variable  A piece of named data referred to by its PV name  The primary object of the Channel Access Protocol Channel  A synonym for Process Variable Channel Access Server  Software that provides access to a Process Variable using the Channel Access Protocol Channel Access Client  Software that requests access to a Process Variable using the Channel Access Protocol

9. October 13, 2008EPICS Training INFN9 What is EPICS? Vocabulary IOC – Input Output Controller  A computer running iocCore, a set of EPICS routines used to define process variables and implement real-time control algorithms - iocCore uses database records to define process variables and their behavior Soft IOC  An instance of iocCore running as a process on a “non-dedicated” computer (i.e. a computer that is performing other functions as well)‏ Record  The mechanism by which a Process Variable is defined in an IOC (using iocCore )‏  Dozens of record types exist, each with it’s own attributes and processing routine that describe its functionality

10. October 13, 2008EPICS Training INFN10 What is EPICS: Vocabulary EPICS Base  IOC, 'hard' and 'soft'  Database (records)‏  Channel Access: Server and Client  Previously Driver/Device support: Now available as modules  Previously Sequencer: Now unbundled.  Extensions  Client tool that is not part of base  EDM, StripTool, … Module  IOC support that is not part of base. Mostly device/driver support  AsynDriver, etc, etc.

11. October 13, 2008EPICS Training INFN11 What is EPICS? Process Variable  A Process Variable (PV) is a named piece of data associated with the machine (e.g. status, readback, setpoint, parameter)‏  Examples of PV names and values: S1:VAC:reading 3.2e-08 torr LINAC:BPM4:xPosition -0.323 mm BOOSTER:gateValvePosition ‘OPEN’ S3:DIPOLE:PS:setPoint 123.4 Amps APS:Mode ‘Stored Beam’ BL3:HISTOGRAM {3, 8, 1, 2, 56, 44, 32, 43, 3, 5, 1}

12. October 13, 2008EPICS Training INFN12 What is EPICS? Process Variable  A Process Variable is a named piece of data with a set of attributes  Examples of Attributes: Alarm Severity (e.g. NO_ALARM, MINOR, MAJOR, INVALID)‏ Alarm Status (e.g. LOW, HI, LOLO, HIHI, READ_error)‏ Timestamp Number of elements (array)‏ Normal Operating Range Control Limits Engineering Unit Designation (e.g. degrees, mm, MW)‏

13. October 13, 2008EPICS Training INFN13 What is EPICS? A Control System Architecture  Network-based “client/server” model where the basic data element is a Process Variable  The Channel Access Protocol defines how Process Variable data is transferred between a server and client  The entire set of Process Variables establish a Distributed Real-time Database of machine status, information and control parameters Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS Process Variables CAS

14. October 13, 2008EPICS Training INFN14 What is EPICS? By default, Channel Access traffic is constrained to a single subnet, but configuration options can direct traffic elsewhere Physical hierarchies can be implemented using switches, routers, and gateways Gateway

15. October 13, 2008EPICS Training INFN15 What is EPICS? Any tool/program/application that abides by the Channel Access protocol could be described as “EPICS Compliant”. My Accelerator Simulator Code (PVs)‏ CAS My Special Data Collection Program CAC EPICS can be viewed as a “toolkit” of EPICS compliant programs. One can select the appropriate tool for their need or develop their own. StripToolMEDM CAC iocCore (PVs)‏ LabView (PVs)‏ CASCACCASCAC

16. October 13, 2008EPICS Training INFN16 How does it do it? Power Supply Beam Position Monitor Vacuum Gauge Computer Interface Process Variables: Channel Access Server S1A:H1:CurrentAO S1:P1:x S1:P1:y S1:G1:vacuum Channel Access Client

17. October 13, 2008EPICS Training INFN17 Where does it do it? Power Supply Beam Position Monitor Vacuum Gauge Computer Interface Process Variables: Channel Access Server S1A:H1:CurrentAO S1:P1:x S1:P1:y S1:G1:vacuum iocCore Channel Access Client Remote Control & Monitoring Access SecurityAlarm Detection Data Trending/Archiving Automatic Sequencing Alarm Logging/Reporting Data Conversion/Filtering Closed-loop Control Operational Constraints Data Trending/Archiving Modeling/ Simulation/Analysis Automatic Sequencing Configuration Control

18. October 13, 2008EPICS Training INFN18 Canonical Form of an EPICS Control System Commercial Instruments IOC CAS Channel Access IOC Software EPICS Database Sequence Programs Custom Programs Real-time Control Client Software MEDM ALH StripTool TCL/TK Perl Scripts OAG Apps Many, many others CA Server Application Custom Chassis/Panels Technical Equipment Process Variables

19. October 13, 2008EPICS Training INFN19 Typical Realizations of an EPICS System IOC CAS Commercial Instruments Custom Chassis/Panels Technical Equipment All IOCs were based on vxWorks (mostly VME)‏ Most CAS Apps were based on Unix or Windows

20. October 13, 2008EPICS Training INFN20 RTEMS Typical Realizations of an EPICS System IOC CAS With Release 3.14, the operating system limitations for iocCore have been removed. Commercial Instruments Custom Chassis/Panels Technical Equipment

21. October 13, 2008EPICS Training INFN21 Introducing the IOC Input Output Controller A computer running software called “IOC Core” The computer can be:  VME based, running vxWorks (only choice until Release 3.14) or RTEMS  PC running Windows, Linux, RTEMS  Apple Mac running OSX  UNIX Workstation running Solaris, OSF Usually has Input and/or Output devices attached An EPICS control system must contain at least one Channel Access Server (usually an IOC)‏ An IOC loads one or more databases, which tell it what to do  What Process Variables to serve  What I/O devices to connect to  What other Pvs to connect to  What processing to do

22. October 13, 2008EPICS Training INFN22 IOC Database  Configuration instead of Coding  'iocCore' software loads and executes 'Records'  Example Assignment:  Read some temperature sensor  Open/close a valve when value is above resp. below some threshold

23. October 13, 2008EPICS Training INFN23 The Example in simplified Code Sensor temp = open_device(…); Valve valve = open_device(…); Loop: if (temp.value() > 10) valve.open(); else valve.close(); delay(1.0);

24. October 13, 2008EPICS Training INFN24 What we omitted Error checking Code comments Apply some smoothing to the temperature reading to filter noise. Send current temperature and valve state to network clients (operator display). Attach a time stamp to the data, so that network clients can see for example when the valve was last opened. Send warnings if the temperature is close to the threshold, or an alarm when way above. Allow runtime changes of the threshold from the remote operator interface. Allow runtime changes to the scan rate. Maybe allow runtime changes to the device address?

25. October 13, 2008EPICS Training INFN25 This IOC 'Database’ does all that At first glance, this might look much worse than the code, but…  that was simplified code.  there's no way the full source code for the above would fit on one screen.  after learning more about the database (~2 days), this becomes much more readable than somebody else's custom code for the same functionality.

26. October 13, 2008EPICS Training INFN26 Some Detail on an EPICS 'Record' Configuration instead of Programming "SCAN=1 second" instead of starting periodic thread, delaying until next multiple of 1 second, locking required resources, … "SMOO=0.5" configures the smoothing algorithm. Almost any field in any record is accessible via network at runtime  Change scan rate, smoothing, … record(ai, temp)‏ { field(DESC, "Read Temperature")‏ field(SCAN, "1 second")‏ field(DTYP, "XYZ ADC")‏ field(INP, "#C1 S4")‏ field(PREC, "1")‏ field(LINR, "typeJdegC")‏ field(EGUF, "100")‏ field(EGUL, "0")‏ field(EGU, "deg C")‏ field(SMOO, "0.5")‏ field(HIGH, "15")‏ field(HSV, "MAJOR")‏ field(MDEL, "0.1")‏ }

27. October 13, 2008EPICS Training INFN27 IOC Database  A single record often handles the scanning, signal conditioning, alarming of a temperature, pressure, or similar analogue reading.  Combined with binary and computational records, it can express most of the data flow logic for a front-end computer  Avoiding the pitfalls of real-time, multi-threaded and networked programming. One can have thousands of records in one IOC.  Well suited for systems with high signal counts, like vacuum or water systems with relatively simple logic but many, many sensors and valves.  kHz-rate processing with record chains is doable  Of course limited by CPU. Not 1000s of kHz rate-records…

28. October 13, 2008EPICS Training INFN28 EPICS Base: IOC Toolkit Database, Sequencer, ChannelAccess. Plus: portability libraries  Linux, OS X, Win32, vxWorks, RTEMS, others.  Basis for most of the "extensions"  Technically, the sequencer is provided separately. Probably the main difference between EPICS and other systems, where the support for front-end computers is often only helper libraries.

29. October 13, 2008EPICS Training INFN29 Record types Classified into four general types Input: e.g.  Analog In (AI)‏  Binary In (BI)‏  String In (SI)‏ Algorithm/control: e.g.  Calculation (CALC)‏  Subroutine (SUB)‏ Output: e.g.  Analog Out (AO)‏  Binary Out (BO)‏ Custom: e.g.  Beam Position Monitor  Multi Channel Analyzer

30. October 13, 2008EPICS Training INFN30 Some record types Analog in Analog out Binary in Binary out Calculation Calculation out Compression Data fanout Event Fanout Histogram Motor Multi bit binary input Multi bit binary output PID control Pulse counter Pulse delay Scan Select Sequence String in String out Subarray Subroutine Waveform

31. October 13, 2008EPICS Training INFN31 Record Processing Record processing can be periodic or event driven Periodic: Standard scan rates are…  10, 5, 2, 1, 0.5, 0.2 and 0.1 seconds  Custom scan rates can be configured up to speeds allowed by operating system and hardware Event driven: Events include  Hardware interrupts  Request from another record via links  EPICS Events  Channel Access Puts

32. October 13, 2008EPICS Training INFN32 IOC Summary Channel Access LAN Sequencer Device/Driver Support I/O Hardware IOC Database

33. October 13, 2008EPICS Training INFN33 Device/Driver Support Software layer that interfaces records to hardware  EPICS web site lists many existing support modules.  Well defined Interfaces allow adaptation of new or custom hardware.  A recordType can interface to many different types of devices. AsynDriver is a facility that supports many different devices  Robust support for GPIB and Serial devices.  Also supports synchronous devices.

34. October 13, 2008EPICS Training INFN34 The Sequencer Runs programs written in State Notation Language (SNL)‏ SNL is a ‘C’ like language to facilitate programming of sequential operations Fast execution - compiled code Programming interface to extend EPICS in the real-time environment Common uses -Provide automated start-up sequences like vacuum or RF where subsystems need coordination -Provide fault recovery or transition to a safe state -Provide automatic calibration of equipment

35. October 13, 2008EPICS Training INFN35 EPICS Sequencer  Adds state-machine behavior to the IOC program Elevator_Simulation ss Elevator { state floor1 { when (floor2_call)‏ { } state goto2 } state goto2 { …

36. October 13, 2008EPICS Training INFN36 Channel Access Allows other programs (CA Clients) to see and change values of Process Variables in an IOC (CA Server)‏ CA Clients may  Put (write)‏  Get (read)‏  Monitor data of Process Variables IOCs are both CA clients and CA servers. They can interact with data in other IOCs A CA Client can connect to many servers A CA Server may serve many clients A very efficient and reliable protocol CA Client CA Server Workstation IOC Network

37. October 13, 2008EPICS Training INFN Channel Access Commands Process Variables: Channel Access Server S1A:H1:CurrentAO S1:P1:x S1:P1:y S1:G1:vacuum Channel Access Client Who has a PV named “S1A:H1:CurrentAO”? I do. What is its value? 25.5 AMPS Change its value to 30.5 “connection request” or “search request” OK, it is now 30.5 30.5 is too high. It is now set to the maximum value of 27.5. You are not authorized to change this value Notify me when the value changes It is now 20.5 AMPS It is now 10.5 AMPS It is now 0.0023 AMPS “put” or “caput” “get” or “caget” “set a monitor” “post an event” or “post a monitor” “put complete” or CA Server CA Client

38. October 13, 2008EPICS Training INFN Some CA Clients (from the EPICS Website - incomplete)‏ ALH: Alarm Handler BURT: Backup and Restore Tool CASR: Host-based Save/Restore CAU: Channel Access Utility Channel Archiver (SNS) Channel Watcher (SLAC) EDM: Extensible Display Manager (ORNL) JoiMint: Java Operator Interface and Management INtegration Toolkit (DESY) Knobs: Knob Manager und KnobConfig, eine Schnittstelle zu SunDials MEDM: Motif Editor und Display Manager StripTool: Strip-chart Plotting Tool and many more …

39. October 13, 2008EPICS Training INFN Example: ALH

40. October 13, 2008EPICS Training INFN Alarms and Colours Record not known White block or Pink Not connected Device not reachableWhiteINVALID ErrorRedMAJOR WarningYellowMINOR Everything okGreenNO_ALARM

41. October 13, 2008EPICS Training INFN Example: ALH

42. October 13, 2008EPICS Training INFN Some CA Clients (from the EPICS Website - incomplete)‏ ALH: Alarm Handler BURT: Backup and Restore Tool CASR: Host-based Save/Restore CAU: Channel Access Utility Channel Archiver (SNS) Channel Watcher (SLAC) EDM: Extensible Display Manager (ORNL) JoiMint: Java Operator Interface and Management INtegration Toolkit (DESY) Knobs: Knob Manager und KnobConfig, eine Schnittstelle zu SunDials MEDM: Motif Editor und Display Manager StripTool: Strip-chart Plotting Tool and many more …

43. October 13, 2008EPICS Training INFN Examples: StripTool

44. October 13, 2008EPICS Training INFN44 Popular CA Server Applications IOC Core PV Gateway CA Server Interface to LabVIEW CA Server Interface to PC Image Acquisition Systems Gateway CAS CAC (PVs)‏ LabVIEW CASCAC (PVs)‏ PC Image Capture CASCAC

45. October 13, 2008EPICS Training INFN45 How fast is EPICS? Can be fast or slow, it depends how you use it! Use the correct tool for the job; Database, sequencer, custom code (ioc) or custom code (client)‏ Ultimately speed depends upon hardware Some benchmarks*: 253,00033MHz68040vxWorksMVME167 25100,000450MHzPPC750vxWorksMVME5100 18100,0002.4GHzP4LinuxPC 233MHz 300MHz Speed 10,000 20,000 Rec/sec 27PIILinuxPC 20PPC604vxWorksMVME 2306 %CPUCPUOSMachine Database design and periodic scanning effect apparent system speed * Extrapolated from benchmark figures courtesy of Steve Hunt (PSI) and L.Hoff, (BNL)‏

46. October 13, 2008EPICS Training INFN IOC Core: Channel Access Services Sequencer Channel Access Client LAN/WAN Operator Interface Channel Access Client Database Links Channel Access Client Channel Access Server EPICS Process Database Performance: 68040 over 10 Mbit Ethernet Gets Propagation Delay2 mS Throughput7.8K /sec Puts Propagation Delay1 mS Throughput17K /sec Monitors Propagation Delay Dependent Throughput15K / sec (Typically 10% channels have monitors)‏ (memory use in IOC - 2 Meg / 60 connections)‏ (30% network load for 15K monitors / second)‏ Increase bandwidth with Routers, Bridges, Higher speed networks and EPICS gateway TCP/IP & UDP

47. October 13, 2008EPICS Training INFN The Learning Curve for EPICS is difficult Installing EPICS Setting up the application environment to automatically build databases Setting up the IOC to boot from the workstation Installing the new drivers Knowledge of how to debug the application - is needed by everyone Learning to use the process database Choosing and learning which client tools to use This learning curve can be eased by receiving training from other laboratories, having one of your employees work and train at an EPICS site, or reading the documents and using the software support document to determine the collaboration member supporting your platform.