LHC RT feedback(s) CO Viewpoint Kris Kostro, AB/CO/FC

Sept. 21, 2005 Kris Kostro, AB/CO/FC 2 Outline Recapitulation of what was already done by CO Where can we go from here and what remains to be done

Sept. 21, 2005 Kris Kostro, AB/CO/FC 3 CO efforts for RT orbit feedback Before 2003: Studies by Thijs Wijnands and Pedro Ribeiro Jens Andersson (fellow) –SPS orbit feedback server “version 1”, successfully used during MD in It was specifically made for this task. –Feedback server framework. SPS orbit server “version 2”, using this framework, successfully tested during MD in –Network/server performance tests using server framework End 2004 it was decided to freeze the work for at least one year –Infrastructure was not yet ready for more tests (FEC, BPM, PO gateways)

Sept. 21, 2005 Kris Kostro, AB/CO/FC 4 Feedback Server Framework “Triggers” activate a sequence of “modules” Modules perform operations on the state –Multiply vector v with matrix M –Receive or send network data –Log data CMW management interface Triggers and modules setup by a configuration file Written in C++, runs on LynxOS and Linux Main motivation: Flexibility, decouple physics calculations from feedback infrastructure Do some work on the LHC orbit feedback in absence of infrastructure and final interfaces

Sept. 21, 2005 Kris Kostro, AB/CO/FC 5 Server framework Config file 10 Hz triggerData importy=Ax Data Correction output Data Beam outReset correctionsCorrection output Data TriggerModule Trigger Module

Sept. 21, 2005 Kris Kostro, AB/CO/FC 6 Feedback server conclusion Good prototyping tool, used with –SPS orbit feedback –Orbit feedback performance tests –Ralph’s simulation of orbit response

Sept. 21, 2005 Kris Kostro, AB/CO/FC 7 Performance tests to assess feasibility of orbit feedback Used feedback server framework Standard PC gateway hardware –Realistic processing complexity 1000x1000 matrix multiplication Extrapolate network behavior from tests of smaller scale –Split over fewer sources 10 or 20 sources (vs ~68) –But realistic data volume 1000 inputs (BPMs), 1000 outputs (correctors) –PCR/Point 8: Upgraded technical network –No correction data sinks

Sept. 21, 2005 Kris Kostro, AB/CO/FC 8 Test configuration PCR and Point 8 Provisional 10/25/50 Hz time tick (CTRP & CTRI) 10 test sources, test data collection Prevessin site, PCRFerney, surface at point 8 Orbit server 5 frontends Timing

Sept. 21, 2005 Kris Kostro, AB/CO/FC 9 Performance tests conclusion Server performance –1000 inputs, 1000 outputs –10 Hz OK, 50 Hz possible, >50 Hz more complicated Network/system performance –10 sources, 99.9% input sets within 6 ms –No signs of scaling problems so far Collect dataProcess dataSend data <6 ms <12 ms <16 ms

Sept. 21, 2005 Kris Kostro, AB/CO/FC 10 Where can we go from here Thanks to the feedback server framework we can start immediately where we stopped in 2004 with new interfaces and infrastructure. Feedback server framework is at least a good prototyping tool for orbit and other future LHC feedbacks. At least partial infrastructure must be in place and interfaces with equipment defined before serious continuation of the feedback tests is possible. Possible scenario: 1 FT SW engineer from CO, common project with OP –CO responsible for server & communication –OP responsible for physics computation and GUI Start at least 1.5 year before the pilot beam from LHC

Sept. 21, 2005 Kris Kostro, AB/CO/FC 11 What remains to be done for orbit Finalize interfaces –BPM’s: FESA? Standard CMW or special UDP? –Power converters: standard CMW or special UDP? –External interfaces to acquire orbit data? –Matrix changes Synchronization –Timing infrastructure and triggering of feedback loop Server functionality and robustness –Hotswap on matrix change – handling BPM and corrector errors –Post Mortem

Sept. 21, 2005 Kris Kostro, AB/CO/FC 12 What remains to be done for orbit (cont.) Continue to assess network performance –Include (more) data sinks –Study perturbations by simultaneous network load –Faster network interfaces (Gigabit Ethernet) for the server? Miscellaneous –Study, together with BDI, impact of other simultaneous operation of BPM (bunch-per-bunch orbit acquisition) on the feedback Hardware for the server(s)

Sept. 21, 2005 Kris Kostro, AB/CO/FC 13 Basic principle 68 Beam Position Monitor frontends –Up to ~1200 BPMs 80 power converter gateways –Up to ~750 correctors Server … Acquire Calculate Correct Acquire … t

Sept. 21, 2005 Kris Kostro, AB/CO/FC 14 Measurement Illustration of what is being measured –a) Time to (trigger and) collect BPM data –b) Time to get a correction result ready Not included –Time to send the corrections out, and for them to be applied Timing event Server #1, Point 8 #2, Point 8 #3, Point 8 #4, Point 8 #5, Point 8 #6, PCR #7, PCR #8, PCR #9, PCR #10, PCR a b t Data processing Input data set complete Calculation complete

Sept. 21, 2005 Kris Kostro, AB/CO/FC 15 Distribution at server Introduction – Server – Tests – Future –a) Time to receive all data –b) …and also compute corrections

Sept. 21, 2005 Kris Kostro, AB/CO/FC 16 Distribution, log scale –After 6 ms, 99.9+% input sets ready at the server –After 12 ms, 99.9+% output sets ready at the server

Sept. 21, 2005 Kris Kostro, AB/CO/FC 17 Number of sources Investigate scaling Comparing 3, 5, 10, 20 sources of UDP data –Constant: in total 1000 data entries, 25 Hz

Sept. 21, 2005 Kris Kostro, AB/CO/FC 18 Networking Technical network –1000 entries (value, sigma, ID; 20 bytes) At 25 Hz, during 4 ms every 40 ms: ~5 Mbyte/s = ~40 Mbit/s bursts That is, 40% of the servers’ 100 Mbps bandwidth …but only 4% in average, or 0.4% of gigabit backbone –Will the future background load change the picture? 20 kB Transfer rate t 40 ms 4 ms 5 Mbyte/s 0.5 Mbyte/s Approximate per plane – total would be twice the amount. However, 20 bytes can be reduced to 10 or even less.

Sept. 21, 2005 Kris Kostro, AB/CO/FC 19 Distribution, CMW CMW also performs well –Not a CMW/UDP comparison UDP: floats, CMW: doubles UDP: including names, CMW: no names Preparation work done for both CMW and UDP