1. NA49-future Meeting, January 26, 20071Ervin Dénes, KFKI - RMKI DATE the DAQ s/w for ALICE (Birmingham, Budapest, CERN, Istanbul, Mexico, Split, Zagreb collaboration) and its possible application at NA49-future

2. NA49-future Meeting, January 26, 20072Ervin Dénes, KFKI - RMKI ALICE Experiment Counting rooms Cavern Computing Center DDLs 10Gb Ethernet Machine: LHC @ CERN, Point 2 ALICE detectors: 17 Online Systems: ECS, DAQ, TRG, HLT, DCS Readout: ~400 DDLs @ 2125 Mbps Storage rate: max. 1.25 GB/s Events: 2.5 MB (p-p), 87 MB (Pb-Pb) DAQ Software: DATE V5 DAQ

3. NA49-future Meeting, January 26, 20073Ervin Dénes, KFKI - RMKI GDC ALICE DAQ architecture CTP LTU TTC FERO LTU TTC FERO LDC BUSY Rare/All Event Fragment Sub-event Event File Storage Network TDS PDS L0, L1a, L2 262 DDLs EDM LDC Load Bal. LDC HLT Farm FEP DDL H-RORC 10 DDLs 10 D-RORC 10 HLT LDC 123 DDLs TDS DS Event Building Network 329 D-RORC 175 Detector LDC 50 GDC 25 TDS 5 DSS

4. NA49-future Meeting, January 26, 20074Ervin Dénes, KFKI - RMKI Trigger Level 0,1 Trigger Level 2 High-Level Trigger Decision Data ALICE online logical model Detector Digitizers Front-end Pipeline/Buffer Decision Readout Buffer Decision Subevent Buffer Event Buffer Storage Local Data Concentrator (LDC) Global Data Collector (GDC) Data transfer Detector Data Link (DDL) DAQ Read-Out Receiver Card (D-RORC) ALICE Terminology Transient Data Storage (TDS) Permanent Data Storage (PDS) Decision Event Building

5. NA49-future Meeting, January 26, 20075Ervin Dénes, KFKI - RMKI ALICE Detector Data Link

6. NA49-future Meeting, January 26, 20076Ervin Dénes, KFKI - RMKI LDC DDL architecture PCI Bus Front-End Read-Out DAQ Read-out Receiver Card (D-RORC) Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) Destination Interface Unit Detector Data Link (DDL) : - Source Interface Unit - Transmission media - Destination Interface Unit Standard detector/DAQ interface 100 MHz 64-bit PCI card

7. NA49-future Meeting, January 26, 20077Ervin Dénes, KFKI - RMKI DAQ Read-Out Receiver Cards (D-RORC) PCI-X adapter for 2 DDL PCI master: autonomous DMA

8. NA49-future Meeting, January 26, 20078Ervin Dénes, KFKI - RMKI Readout System Performance Motherboard with dual Xeon CPUs @ 2.4 GHz Six PCI-X slots, 4 bus segments (3+1+1+1), 2 controllers Linux OS ALICE Data-Acquisition software (DATE)

9. NA49-future Meeting, January 26, 20079Ervin Dénes, KFKI - RMKI Performance: 6 D-RORCs Testing the fully populated PC using data source internal to PCI interface Interoperability test Measure the maximal input bandwidth PCI #6 PCI #5 PCI #4 PCI #3 PCI #2 PCI #1 Segment #1 #2 #3 #4 Controller #1 #2 1 Ch 2644644245287921045840 264232141.3264 261.3140 Aggregate Bandwidth [MB/s] Normalized Bandwidth [MB/s/Ch]

10. NA49-future Meeting, January 26, 200710Ervin Dénes, KFKI - RMKI DAQ software DDL softwarewe can freely use DAQ framework (DATE)does we need it? Performance Monitoring (AFFAIR) Data quality monitoring (MOOD) Information dissemination Documentation: 4 DAQ sw packages + ECS: fully documented. User’s guide released and printed Regular DAQ trainings (70 people in last 3 years) Linux Linux SLC4 now. Process defined to produce, distribute, install new versions Code management system: CVS. Release packaging and distribution: RPM (~20 MB) Automatic installation of Linux and DATE on DAQ nodes

11. NA49-future Meeting, January 26, 200711Ervin Dénes, KFKI - RMKIRORC Memory Management’s Principle Firmware data length transfer status Ready FIFO Transfer status possible values: ffffffff ffffffff unloaded (set by sw) 00000000 00000000 loaded, no DTSTW (set by fw) else else loaded, DTSTW (set by fw) Free FIFO block size Index of Ready FIFO start address Page aligned continuous user memory outside Linux’s memory space, reachable for DMA and user as well. (physmem module necessary) Free blocks for events or event fragments Free FIFO / Ready FIFO depth: 128 Maximum block size: 2Mbyte

12. NA49-future Meeting, January 26, 200712Ervin Dénes, KFKI - RMKI The Free FIFO D-RORC PC memory bank Firmware readout page address Free FIFO PC CPU Allocation of free pages

13. NA49-future Meeting, January 26, 200713Ervin Dénes, KFKI - RMKI Direct Memory Access D-RORC Firmware PC memory bank DDL No involvement PC CPU

14. NA49-future Meeting, January 26, 200714Ervin Dénes, KFKI - RMKI The Ready FIFO D-RORC PC memory bank readout DDL Ready FIFO Firmware addresspage status addresspage status addresspage status Delivery of filled pages PC CPU

15. NA49-future Meeting, January 26, 200715Ervin Dénes, KFKI - RMKI DDL Software All functions accessible as interactive commands or API Script-based interpreter for sequence of operations: Sending command to the FEE Reading FEE status printing the status comparing the status polling the status Downloading data into the FEE from a file Reading data from the FEE writing data into a file comparing data with data in a file TPC configuration: < 0.3 s DDL performance Parallelism FERO DDL definepedestal_addr0x1FFF defineenable_pedestal0x2C resetSIU write_commandenable_pedestal write_blockpedestal_addr pedestal.hex%x read_and_check_blockpedestal_addrpedestal.hex%x definepedestal_addr0x1FFF defineenable_pedestal0x2C resetSIU write_commandenable_pedestal write_blockpedestal_addr pedestal.hex%x read_and_check_blockpedestal_addrpedestal.hex%x D-RORC LDC

16. NA49-future Meeting, January 26, 200716Ervin Dénes, KFKI - RMKI DATE Run Control Control Configuration Main parameters Display SMI State of all nodes Main counters of all nodes Compatible with ALICE ECS

17. NA49-future Meeting, January 26, 200717Ervin Dénes, KFKI - RMKI DATE Configuration Database Database content DATE Roles Actors of DATE system: LDCs, GDCs… Trigger Trigger masks Detectors Front-end equipment of LDCs Event building control Event building rules Banks Memory banks to operate DATE

18. NA49-future Meeting, January 26, 200718Ervin Dénes, KFKI - RMKI DATE Infologger (1/2) Online view Online selection of visible fields Online selection of message displayed

19. NA49-future Meeting, January 26, 200719Ervin Dénes, KFKI - RMKI DATE Infologger (2/2) Offline view Queries in the message archives Selection of visible fields and of message displayed

20. NA49-future Meeting, January 26, 200720Ervin Dénes, KFKI - RMKI Data quality monitoring: MOOD MOOD: Monitoring Of Online Data DATE + ROOT environments MOOD framework Interfaces to detector code Applications: Raw data integrity Detector performance

21. NA49-future Meeting, January 26, 200721Ervin Dénes, KFKI - RMKI Performance monitoring: AFFAIR System view / Aggregated performances Event building bandwidth Event numbers Individual nodes view CPU usage Input/Ouput Node status

22. NA49-future Meeting, January 26, 200722Ervin Dénes, KFKI - RMKI Event Building Switch ALICE baseline: TCP/IP over switched Ethernet Computing Data Challenges Event-Building Switch CERN frame contracts Needs of IT and experiments Good prices On site maintenance (company+IT) Force 10 Model 1200

23. NA49-future Meeting, January 26, 200723Ervin Dénes, KFKI - RMKI Qualification of event-building switch

24. NA49-future Meeting, January 26, 200724Ervin Dénes, KFKI - RMKI Storage Arrays Performance Local disk buffer at Pt2 Storage network: Fibre Channel currently FC 2G or 4G (Brocade,QLogic) 16 ports switches with 4 uplinks for interconnect

25. NA49-future Meeting, January 26, 200725Ervin Dénes, KFKI - RMKI Storage Arrays Performance Transient Data Storage Storage arrays (e.g. Infortrend A16F 2 FC 2G ports, 16 SATA II HD) Performance for a RAID 5 set of 5 disks

26. NA49-future Meeting, January 26, 200726Ervin Dénes, KFKI - RMKI DDL and DATE for NA49-future? (1/2) Data transfer DDL and D-RORC produced and used for longtime Match speed requirements Few (< 5) PCs are enough for the task We have the knowledge how to implement to NA49-future About 1 man-year of work Problems with sub-detectors using VME In ALICE exist VME boards where DDL SIU cards are attached, but interfaces are specific to the given ALICE sub-detector Or the design VME – DDL interface is necessary for TOF and CAMACs Interconnect with the trigger system Trigger “box” has to be designed

27. NA49-future Meeting, January 26, 200727Ervin Dénes, KFKI - RMKI DDL and DATE for NA49-future? (2/2) DAQ software Software (DATE V5) released and documented. Process defined to produce, distribute, install new versions. Everything (ECS, monitoring, etc.) included Can we get it from ALICE? They give it only with support They can not support us during this year (integration with 17 ALICE sub-detectors) Modifications necessary if TOF and CAMACs are used without the DDL Is it shooting at sparrows with a high caliber gun ? DAQ fabric hardware Event building switch is necessary Simple storage network has to be added

28. NA49-future Meeting, January 26, 200728Ervin Dénes, KFKI - RMKI Thank you

29. NA49-future Meeting, January 26, 200729Ervin Dénes, KFKI - RMKI Detector Data Link (DDL) Detector readout: fast data transfer to PC memory Electronics configuration: pedestals download Interface and data-transfer detector/DAQ Radiation tolerant card All components are radiation tolerant including FPGA (ACTEL) 10 year doses in ALICE (at TPC inner radius): total ionising dose1.6 krad neutron fluence3.9·10 11 cm -2 charged hadron fluence8·10 9 cm -2 ACTEL ProASIC adopted as baseline

30. NA49-future Meeting, January 26, 200730Ervin Dénes, KFKI - RMKI DATE V5 Software ALICE Data Acquisition and Test Environment (DATE): software framework for the ALICE DAQ system  Latest DATE version V5:  DATE V5.x kits: RPM package (~20 MB), User ’ s Guide (~470 pages)  Platforms: IA32 family, SLC4 using a 2.6 kernel  Development: ~20 packages, mostly in C, using CVS  Auxiliary Software: DIM, SMI, MySQL, Tcl/Tk, libshift, ROOT, VMEbus driver  Testing: reference system, on-line data challenges, test beams DATE V5 Packages