1. SLAC 1st April 2010 Paul Scherrer Institut Ernst Johansen The „Pie in the Sky“

2. Agenda Unified Processing Architecture Motivation History Technology Feasibility Proposal Cost Discussion ma.a3uUs

3. Motivation PSI Motivation We will soon operate four Machines (2x proton, 2x electron) Maintenance will become a huge Challenge Focus on synergies between all machines We need very long lifetime for the control solutions Personal Motivation Bring in +15 years of experience in designing Industrial Control Systems I like do design systems on a “decent” platform

4. Unified Processing Architecture ? GPAC PPC440, OS = ? IOC PPC, Intel VxWorks, Linux VPC PPC405, DSP No OS, Xilkernel UPA Common CPU, OS Common FW SLS, HIPA DSP, uC Use Cases PLC Siemens, Rockwell, Wago, Beckhoff

5. Platforms Religion…. They somehow do it all… don’t they ? Its very expensive to change them You need alliances to be strong… Technology Are there any new technologies around ? How to apply these to the VME64x platform ? Significant Improvements PCI Express FMC (VITA57.1-2008) Larger, low-power FPGA How to apply PCI Express, FMC and larger FPGA’s to VME64x ?

6. Design – VME Bottleneck CPU FPGA ≈250MB/s (2Gbps) VME64x CPU DSP

7. Design – Additional Bandwidth at Low-Cost CPU FPGA PCIe 4x 8Gbps Duplex GEthernet 2x 2Gbps Duplex SFP 4x 15Gbps Duplex VME64x Card

8. Design – Scalable Bandwidth CPU FPGA CPU FPGA CPU FPGA N x 8Gbps 10G Ethernet Switch PCI Express, Fiber Optics N x 2Gbps N x 15Gbps COTS N = 1.. 21 VME

9. Improved P0 – VME64x Compatible Improved P0 2mm Legacy Compatible High-Speed Links SFP 7Gbps SPF PCIe VersaLink

10. FMC – High-Speed I/O Defined PSI standard, but… VITA57.1-2008 FPGA Mezzanine Cards is here Supports the PSI Use Cases Broad industrial acceptance 69 x 76.5 mm 2 (small) Examples 4DSP – 8 channel ADC 250Msps @ 14-bit – 4 channel ADC 125Msps @ 16-bit – 1 channel ADC 5Gsps @ 8-bit – 4 channel DAC 1000Msps @ 16-bit Curtiss-Wright – 4 channel DAC 500Msps @ 16-bit Proposal VITA57.1-2008 compatibility Support for extended PCB size where required

11. Hardware Design Are there any reusable existing products on the market? What has to be extended ?

12. IOxOS Swiss Company Gland by Geneva Experts VME, PCIe, Board Design Products VME64x High-Performance VME64x Bridge Design Kits Virtex-5, Virtex-6 Status NDA Signed Offer Design UPA Sell Into research community

13. 01.04.10PSI, CPU Intel Atom No ECC AMCC APM83290 ~21W Marvell MV78200 ARM, 6W, immature Linux Freescale P2020 Dual 1.2GHz PowerPC, <8W Freescale P4040 Quad 1.5GHz PowerPC, ~20W Freescale P4080 Octal 1.5GHz PowerPC, ~ 30W Broadcom, NetLogic, Cavium… P2020RDB

14. Design IOxOS TOSCA II Design Kit Based on mature TOSCA product Designed for Reuse: VME64x, PCIe Bridge, IDMA, DDR3 Innovative PCIexpress 2.0 Network-on-Chip Architecture Supports Virtex-6 CXT – Cost efficient 40nm process VHDL Simulation Framework Linux Drivers and Test Framework Excellent documentation All VHDL source and hardware design files available !! Freescale P2020RDB Design Kit P2020 Dual Core CPU ECC on memory 2x Ethernet, USB 2.0, UART, NOR and NAND Flash Long term availability Low power 45nm process

15. Unified Processing Architecture – IOxOS TOSCA II HW+FW reuse P2020RDB HW+BSP reuse

16. Unified Processing Architecture – HW Benefits High-Reliability Dual Core IOC ≈250USD Fast VME64x ≈100USD FMC (Digitizers) XMC (PCIe) PMC (PCI) PCI Express (P0) High-End Workstation Large and fast FPGA ≈300USD Ethernet Remote debugging & download P2 Legacy Support 3.3V I/O FPGA

17. IOxOS Thermal Design Rear IO PMC / XMC / FMC Cooler

18. Software Design Are there any reusable existing products on the market? What has to be extended ?

19. 3S - CoDeSys SP V3 (OEM) CoDeSys IEC61131-3 De-facto Industrial Standard Used by several hundred companies Supports all IEC61131-3 Languages Ported to Linux Build-In EtherCAT Stack Status Offer Porting to UPA

20. EtherCAT – Medium-Speed I/O Bus Master Based on standard Ethernet Very low protocol overhead Key Data Performance – 1.000 Digital-I/Os in 30 µs – 200 Analog-I/Os (16 Bit) in 50 µs – 100 Servo-axis in 100 µs Suitable PCI / PMC replacement Suitable IP-module replacement Technology http://www.beckhoff.de

21. Design Open Source Real Time Linux with PREEMPT_RT patch P2020RDB Linux BSP, SMP configuration IOxOS TOSCA II Linux Drivers Test Framework EPICS Linux Port CoDeSys IEC61131-3 with EtherCAT Stack Linux PREEMPT_RT

22. Capabilities - Linux RT on P2020 Similar to Emerson MVME4100 performance, but has additional Second CPU core for real-time DSP Real-Time PREEMPT_RT on P2020 in SMP-Mode (One operating system – two cores) Patch is easy to apply and well supported Moved all Processes to PPC0 Moved all Interrupts to PPC0 100% PPC0 load (cyclictest) Executed EPICS on PPC0 Executed cylictest on PPC1 PPC1 Latency Results Min 5usAvg 5usMax 40 us Conclusions Guarding one CPU enables hard real-time behavior running EPICS in parallel. Preliminary results shows that 10kHz @ 50% CPU load is feasible.

23. Unified Processing Architecture – Software EPICS Channel Access EPICS IOC CoDeSys IEC61131 Matlab M  C IOxOS TOSCA II EtherCAT I/O RF FMC VME 64x IDMA DRAM Legacy IO

24. Potential for Cost Reductions Cooperate with Industry to find a Common Platform Sell Platform into different Markets PECTRON PSIIOxOS SwissFELRF Applications Process Automation

25. PECTRON – Power Electronics Control Status Swiss start-up company Share NRE Sell into Power Electronic Market Crate ELMA 2-Slots PLC CoDeSys SoftPLC IEC61131-3 Programming EtherCAT Beckhoff

26. VITA57.1-2008 FMC – Partner Strategy Cooperate with Industry Utilize latest available technology Curtiss-Wright NDA Next Generation FMC ADC 250Msps @ 16-bit ??? 4DSP Next Generation FMC 6-channel ADC 250Msps @ 16-bit ???

27. Conclusions We need a long term strategy for our control systems UPA enables one single framework for PLC, IOC, Medium-Speed- and RF applications VME64x is the most stable specification and compatible with SLS and HIPA  Retrofit Higher integration we can considerably reduce the cost of VME64x The UPA has low component count and ECC – suitable for high-reliability applications With UPA we can achieve near to zero long-term royalties With strategic sourcing a lifetime of 20 years is feasible (if we want to) We get complete design in source code – including hardware and VHDL!!! There are potential partners to get into a win-win situation!!!

28. The „Pie in the Sky“