1. Fast Orbit Feedback - BPM to PS Yuke Tian Control Group, Accelerator Division Photon Sciences Directory Brookhaven National Lab EPICS Collaboration Meeting, BNL, 2010

2. Outline NSLS-II orbit feedback system requirements Fast orbit feedback system architecture Overall architecture BPM FOFB data measurement BPM FOFB data delivery – fiber SDI link FOFB calculation – compensation for each eigenmode Corrector setpoints to power supply system Progress Summary EPICS Collaboration Meeting, BNL, 2010

3. NSLS-II orbit feedback system requirement Energy 3.0 GeV Circumference 792 m Number of Periods 30 DBA Length Long Straights 6.6 & 9.3m Emittance (h,v) <1nm, 0.008nm Momentum Compaction.00037 Dipole Bend Radius 25m Energy Loss per Turn <2MeV Energy Spread 0.094% RF Frequency 500 MHz Harmonic Number 1320 RF Bucket Height >2.5% RMS Bunch Length 15ps-30ps Average Current 300ma (500ma) Current per Bunch 0.5ma Charge per Bunch 1.2nC Touschek Lifetime >3hrs NSLS-II technical Requirements & Specifications EPICS Collaboration Meeting, BNL, 2010

4. NSLS-II orbit feedback system requirement Long IDShort ID Dispersion Region Lattice function for one cell EPICS Collaboration Meeting, BNL, 2010

5. NSLS-II orbit feedback system requirement In short insertion,, hence must hold orbit stable to Orbit stability requirements can be met using orbit feedback. EPICS Collaboration Meeting, BNL, 2010 NSLS-II orbit stability requirement

6. NSLS-II orbit feedback system requirement Long term - Years/Months Ground movement Season changes Medium - Days/Hours Sun and Moon Day-night variations (thermal) Rivers, rain, water table, wind Synchrotron radiation Refills and start-up Sensor motion Drift of electronics Local machinery Filling patterns Short - Minutes/Seconds Ground vibrations Traffic, Earth quakes Power supplies Injectors Insertion devices Air conditioning Refrigerators/compressors Water cooling Beam instabilities in general Noises source need to be suppressed EPICS Collaboration Meeting, BNL, 2010

7. NSLS-II orbit feedback system requirement Typical noises source frequency in light source EPICS Collaboration Meeting, BNL, 2010

8. NSLS-II orbit feedback system requirement Beam motion from long term group motion Floor motion around the ring. Maximum movement is 107 µm, and the RMS around the ring is 36 µm. Electron beam motion (vertical) without feedback loop; maximum is 600 µm. Lihua Yu EPICS Collaboration Meeting, BNL, 2010

9. Orbit feedback system architecture Overall architecture: slow and fast correctors EPICS Collaboration Meeting, BNL, 2010

10. Fast orbit feedback system in one cell Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010

11. NSLS-II two tier device controller architecture Orbit feedback system architecture Cell 1 Cell 2 Cell 3 Cell 16 Cell 30 Cell 29 Storage Ring Cell 15 Cell 17 SDI link EPICS Collaboration Meeting, BNL, 2010

12. Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010 NSLS-II FOFB latency/bandwidth ● BPM group delay: less than 50 us (estimate) ● BPM data from 8 BPMs transferred to cell controller: 3.5us ● BPM data distribute among the storage ring 6us (12us if one link broken) ● FOFB calculation with separate mode compensation: 3.5us ● Set power supply by PS SDI link: 5.4 us (11us if one link broken) ● Fast corrector PS bandwidth (10 degree phase shift): 8KHz ● Corrector magnet/chamber bandwidth: 1KHz (measured with Inconel beampipe, 10 degree phase shift)

13. BPM FOFB data measurement Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010 Joseph Mead

14. BPM FOFB data measurement Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010 Kiman Ha

15. BPM FOFB data delivery – fiber SDI link Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010 Joseph De Long

16. FOFB calculation – compensation for each eigenmode Orbit feedback system architecture Fast orbit feedback system is a typical multiple-input and multiple-output (MIMO) system. For NSLS-II, there are 240 BPMs and 90 fast correctors. Tranditional singular value decomponsition (SVD) based FOFB treats each eigenmode the same. The reality is, a MIMO system will have different frequency response for different eigenmode and thus it is desirable to design different compensation to each eigenmode. The challenge is to finish the large computation within the time budget of FOFB system. NSLS-II FOFB system takes advantage of our two-tier communication structure and the parallel computation capability of FPGA to do the compensation for each eigenmode in FPGA. EPICS Collaboration Meeting, BNL, 2010

17. Orbit feedback system architecture A simple SISO feedback system Controller C(z) Plant H(z) EPICS Collaboration Meeting, BNL, 2010

18. Orbit feedback system architecture Fast orbit feedback system algorithm (MIMO system) Controller R -1 =VΣ -1 U T Accelerator R=UΣV T Compensator (PID etc) R: response matrix R -1 : reverse response matrix FOFB baseline algorithm Offline operation: kick each corrector  measure all BPM and get response matrix R  calculate R -1 with SVD (10KHz) operation: measure/distribute all BPM data  calculate corrector setpoints  set correctors EPICS Collaboration Meeting, BNL, 2010

19. Orbit feedback system architecture FOFB baseline calculation Each of the corrector setpoint is calculated from a 1xM vector and Mx1 vector multiplication. If 8 BPM/cell, M=240. For each corrector plane EPICS Collaboration Meeting, BNL, 2010

20. Orbit feedback system architecture Problems for baseline algorithm The ill-conditioned response matrix will cause numerical instability Solution: 1) Truncated SVD (TSVD) regularization: sharp cut off 2) Tikhonov regularization Each of the corrector setpoint is calculated from a 1xM vector and Mx1 vector multiplication. If 8 BPM/cell in NSLS-II, M=240. Total: ~ 240 MAC. For each corrector plane EPICS Collaboration Meeting, BNL, 2010

21. Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010

22. Orbit feedback system architecture EPICS Collaboration Meeting, BNL, 2010

23. Orbit feedback system architecture Problems for baseline algorithm To suppress high frequency contribution to the integrated amplitude, it is desirable to compensate each mode in frequency domain. So far, all the FOFB has an assumption that each mode has the same frequency response (same bandwidth). UTUT Accelerator R=UΣV T Qi(z -1 ) V EPICS Collaboration Meeting, BNL, 2010 Compensation for each eigenmode

24. Orbit feedback system architecture : Compensator for mode i. :eigentspace components of Calculation with the compensation for each mode EPICS Collaboration Meeting, BNL, 2010

25. Orbit feedback system architecture 1. Calculate the eigenvector components (d 1,d 2,…d N ) for BPM displacement Total calculation: 1xM vector times Mx1 vector. Do this N times (for each ). This is N times larger calculation than gain-only FOFB calculation. For NSLS-II, N=90. Total MAC: 240 * 90 = 21,600 2. For each, design compensation. 3. Output modes (V) times. 1xN vector times Nx1 vector. Challenge: need to finish the calculation within a few microsecond.  FPGA is perfect for this task. It can carry out the calculation (Matrix calculation and DSP compensation) in parallel. Calculation with the compensation for each mode EPICS Collaboration Meeting, BNL, 2010

26. Implementation of fast orbit feedback Calculation with the compensation for one mode Dual Port RAM Dual Port RAM Dual Port RAM MUX X Compensation (PID, Notch filter etc) + From control system Eigenvector Select Active Eigenvector BPM data From SDI link EPICS Collaboration Meeting, BNL, 2010

27. Implementation of fast orbit feedback EPICS Collaboration Meeting, BNL, 2010 Single mode component compensation (floating point module)

28. Implementation of fast orbit feedback EPICS Collaboration Meeting, BNL, 2010 Single mode component compensation (floating point module) Simulated BPM data Simulated input matrix Single mode component Single mode component with compensation

29. Implementation of fast orbit feedback EPICS Collaboration Meeting, BNL, 2010 Single mode component compensation (compare fixed point and floating point module)

30. Implementation of fast orbit feedback SystemGenerator: floating point with fixedx point EPICS Collaboration Meeting, BNL, 2010

31. Implementation of fast orbit feedback EPICS Collaboration Meeting, BNL, 2010 Single mode component compensation (compare FPGA fixed point and Matlab floating point calculation) Calculation difference for single mode component Calculation difference for single mode component with compensation

32. Implementation of fast orbit feedback EPICS Collaboration Meeting, BNL, 2010 Single mode component compensation (FPGA hardware co-simulation and Matlab floating point calculation)

33. Implementation of fast orbit feedback Compare FPGA hardware co-simulation results and Matlab floating point calculation EPICS Collaboration Meeting, BNL, 2010

34. Implementation of fast orbit feedback Flexible frequency compensation in FPGA for each eigenmode EPICS Collaboration Meeting, BNL, 2010 60Hz notch filter 200KHz low path filter

35. Implementation of fast orbit feedback Calculation for corrector setting (one plane) One modeTotal Speed(240+12)*10 = 2520ns2520 + (90+8)*10 =3500ns Accuracy<10ppm BRAM Resource (Virtex5FX70) 1% of BRAM33% of BRAM DSP ResourceDepend on the compensation design. Need lots of DSP resource to compensate for all eigenmodes. Virtext5 (Virtex 6) provides many DSP resources to achieve single mode compensation requirement. EPICS Collaboration Meeting, BNL, 2010

36. Corrector setpoint to power supply system Orbit feedback system architecture Master (PSC master, Cell controller) PSC TX RX TX RX TX RXTX RX TX RX TX RX TXRX TX RX TX RX TXRX TX RX TX RXTX RX Power supply SDI link RX TX RX TX RX TX RX TX RX TX EPICS Collaboration Meeting, BNL, 2010  Redundant 100Mbps link to deliver FOFB calculation results (corrector setpoints) to PSC  Dynamic ID assignment  Flexible package size  Simply cabling between PSCs and cell controller.

37. Progress EPICS Collaboration Meeting, BNL, 2010 BPM – finished first revision SFP for fiber SDI link RF signal inputs V5 FPGA 4 ADC channels 256MB DDR2 GigE to EPICS IOC

38. Progress EPICS Collaboration Meeting, BNL, 2010 Cell Controller – finished first revision SFP for fiber SDI link V5 FPGA 256MB DDR2 16 isolated TTL inputs 12 50Ohm TTL outputs 4 analog outputs 2 PS SDI links GigE to EPICS IOC

39. Progress EPICS Collaboration Meeting, BNL, 2010 Power supply controller – in production 1 = JTAG connectors – Programming to FPGA and CPLD. 2 = RS232 port – Communication to PC for diagnostic and software development. 3 = DDR2 memory modules – PS diagnostic data, CPU memory. 4 = SDI connectors – Communication between PSC master (or cell controller) and PSC slaves. 5 = Fiber transceiver – Communication with PSI. 6 = Ethernet connector – Communication to EPICS IOC for PSC master. 7 = FPGA (Spartan3A) 8 = CPLD(8a) & SPI memory(8b) – Dual boot and remote programming functions. 7a 1 2 3 4 5 6 8a 8b 7

40. Progress BPM hardware (DFE and AFE) passed the first revision. The second revision is underway. Cell controller (DFE and IO board) passed the first revision. BPM FOFB data measurement is tested with lab simulated data and real beam data. BPM FOFB data delivery (fiber SDI link) is tested. FOFB calculation is tested. Power supply SDI link is tested. Power supply controller is in production. We are working on the FOFB system integration. It includes integration of BPM, cell controller and power supply controller units. EPICS Collaboration Meeting, BNL, 2010

41. Summary NSLS-II tow tier structure provides fast and deterministic data transfer for BPM data (BPM to cell controller), and power supply setpoints (cell controller to power supply controller). Cell controller is the central data concentrator/generator that has all the necessary data for FOFB calculation and needs no extra data shuffling. All BPM and power supply data bits are put “on wire” once. NSLS-II FOFB is taking advantages of the two tier structure and the parallel computation capability of FPGA to implement a unique FOFB algorithm that can carry compensation for each eigenmode. NSLS-II will be the first facility to implement such FOFB approach. NSLS-II FOFB hardware, firmware and software design will be fully open to the community. For more hardware and firmware details, please go to tomorrow’s “Open Hardware Development Workshop”. EPICS Collaboration Meeting, BNL, 2010

42. Acknowledgments FOFB algorithm Lihua Yu Igor Pinayev BPM/ Cell controller development Kurt Vetter Joseph Mead Joseph De Long Kiman Ha Alfred Dellapenna Yong Hu Guobao Shen Om Singh Bob Dalesio PSC and PS design Wing Louie John Ricciardelli George Ganetis