1. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski1

2. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski2 Outline Requirements of DAQ framework for CBM The CMS XDAQ framework XDAQ on the CBM IB-Cluster Summary and further evaluation

3. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski3 DAQ for CBM Triggerless data transport until filter farm B-net: 1000 x 1000 nodes, high speed connections Linux may run on all DAQ nodes (even FPGAs) Test cluster with InfiniBand: small „demonstrator“ set-up within next year

4. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski4 Data Acquisition framework requirements Configuration of multiple nodes (connections and algorithms, database?) Controls and monitoring Data transport (format, protocol, hardware drivers?) Message logging Error handling and failure recovery (ARMOR?) Modular architecture, extensability! (sub-detector tests) User interface?

5. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski5 Standard DAQ framework for LHC CMS experiment (Orsini, Gutleber) http://xdaqwiki.cern.chhttp://xdaqwiki.cern.ch C++ libraries on Linux, modular packages Each node: context, xdaq process with embedded xdaq applications Configuration: XML Data transport: I2O protocol (Intelligent IO) Communication: http, cgi; SOAP messages The CMS XDAQ framework

6. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski6 XDAQ features continued... http://xdaqwiki.cern.chhttp://xdaqwiki.cern.ch State machines (sync/async FSM) Message logger, error handler Monitoring tool Hardware access library (HAL) Front End Driver (FED kit, for CMS!) Job Control (task handler for node control) others: exceptions threads infospace data (de)serializers,... The CMS XDAQ framework

7. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski7 XDAQ: messaging and peer transport peer transport i2o tcp fifo http SOAP

8. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski8 XDAQ: I2O messaging and peer transport

9. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski9 XDAQ: web server hyperdaq web interface

10. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski10 XDAQ executive with applications Common XML configuration file: Each node knows all applications on all nodes! Unique addressing by context (url) and application id

11. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski11 XDAQ: SOAP messaging xrelay: send SOAP control messages example commands: Configure, Enable, Halt, Reset,.. any new commands may be defined web interface not suited as real control system UI! XDAQ applications may exchange SOAP messages with other UI (Labview?)

12. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski12 XDAQ on the CBM InfiniBand cluster Installation problems: XDAQ distribution not 64 bit save! Code adjustments required kernel modules for CERN linux excluded! drivers for CMS hardware (FED) excluded coretools done, worksuite partially,…  Basic evaluation possible!

13. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski13 XDAQ tests on CBM InfiniBand cluster Tested features on IB cluster: http://wiki.gsi.de/cgi-bin/view/Daq4FAIR/DaqClusterSoftwareXdaqStatus http://wiki.gsi.de/cgi-bin/view/Daq4FAIR/DaqClusterSoftwareXdaqStatus hyperdaq webserver (main test controls ui) xml configuration setup control variable access and export (infospace) SOAP messaging (test controller application) state machines (sync./async. with web/SOAP ui) monitoring application (very raw ui!) multithreading (workloop framework) exceptions, data wrappers, message logging, toolbox,… i2o data transport (tcp roundtrip example)

14. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski14 I2O roundtrip on CBM InfiniBand cluster Standard XDAQ roundtrip benchmark: sender node posts I2O frame to receiver node receiver callback sends frame back to sender sender callback sends frame back again, etc… perfmeter class records bandwidth, latency,… web display of results on sender node (java!) Code independent of transport implementation! XML configuration file does set up (tcp ethernet, tcp IB, fifo)

15. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski15 XDAQ executive MyRoundTrip HyperDAQ I2O roundtrip on CBM InfiniBand cluster sender receiver XDAQ executive MyRoundTrip token() EnableAction() default() I2O callback web application state machine tcp over IB

16. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski16 XDAQ pseudo roundtrip with udapl rdma Developed udapl roundtrip in xdaq application: Use of C++ wrapper for udapl (S. Linev) XDAQ roundtrip example, but without I2O no package return in receiver callback, but IB-RDMA write-read from sender measurements with XDAQ perfmeter tool  XDAQ compatible with UDAPL library (multithreading!)  Overhead compared with plain UDAPL!

17. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski17 XDAQ executive MyDAPLRoundTrip HyperDAQ XDAQ pseudo roundtrip with udapl rdma sender receiver XDAQ executive MyDAPLRoundTrip Benchmark() ConfigureAction() EnableAction() default() state machine workloop (thread) web application ConfigureAction() state machine allocated mem uDAPL/RDMA

18. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski18 I2O roundtrip on CBM InfiniBand cluster 955 MB/s plain RDMA 813 MB/s XDAQ with RDMA 234 MB/s XDAQ, I2O, TCP over IB penalty of XDAQ framework? measurement? procedure? minimum latency: 55 µs 30 µs 4 µs

19. 7. CBM collaboration meetingXDAQ evaluation - J.Adamczewski19 Summary and Outlook CBM DAQ system needs software framework XDAQ offers many required features Tests on CBM IB cluster: SOAP controls, web server, monitor, i2o over tcp XDAQ works together with uDAPL library (IB) To do: I2O implementation of uDAPL – performance? Configuration and control system? XDAQ scalability? Job Control? Implementations for DAQ demonstrator hardware?