DAQ for Test&Commissioning at Point 2 Klaus Schossmaier CERN PH-AID ALICE TPC Collaboration Meeting Heidelberg, Germany February 2004

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Planning Parameters  System: DAQ system for Test&Commissioning (= LHCC milestone)  Size:12 DDLs for TPC 8 DDLs for others  Place:LHC Point 2  Date: Q  People:ALICE DAQ team

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DAQ Installation at Point 2 SXL HallSX Hall UX25 (Experimental Area) PX24/CR1 (DAQ) PX24/CR2 (HLT) SR Hall Access Shaft DDL LAN (ALICE sub-detector assembly) (Networking) PX24/CR3 (DCS) PX24/CR4 (Misc.) LAN ACRWR1WR2

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Installation at PX24/CR1 10 LDCs (Local Data Concentrator): rackmount PCs U1, U2, U4 height equipped with 20 D-RORC cards  DDL readout, sub-event building  HLT interface 4 GDCs (Gobal Data Collector): rackmount PCs U1 height equipped with FC cards  full-event building 1 DSS (DAQ Server Services): rackmount PC U4 height hot-swap SCSI disks  RunControl, Logger  On-line Monitoring 1 KVM switch1 FC switch 1 TDS (Transient Data Storage): rackmount disk array U2 height IDE of FC disks  event buffering GE switches PDS (Permanent Data Storage) CERN computer center 20 DDLs DDL patch panels HLT (High Level Trigger) PX24/CR2

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Installation Issues  Hardware/Mechanical issues:  Equipment: rackmount PCs, switches, disk array, …  Rack assembly, DDL installation, cabling, …  Infrastructure: power supply, networking, cooling, …  Software/Testing issues:  Operating System: Linux + customizations  DAQ Software: DATE + configuration  Standalone tests  Integration tests  Reference System in our AID lab:  small, complete DAQ system with 6 DDLs  to spot problems in advance  to tackle problems during operation

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Reference Setup – Front KVM switch: Raritan Paragon UMT2161 4x LDCs: - 1U, 2U, 4U height - dual Xeon - 6x D-RORC cards  DDL 2x GDCs: - 1U height - dual Xeon - Qlogic QLA2310F cards 2x disk arrays: - Infortrend IFT DotHill SANnet II - Fibre Channel DSS: - 4U height - quad Xeon - 3x 36GB SCSI disks L3 rack 2x GB Ethernet switch: 3COM SuperStack 3 Fibre Channel switch: Broacade SilkWorm 3800 DDLs: - to the DDL lab

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Reference Setup - Rear Power distributor Cat5 cables: - Ethernet - KVM - RJ45 connectors Fiber Channel cable: - 2 Gbit/s multimode - LC-LC connectors L3 rack Mounting rails (!)

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DDL & D-RORC DDL link card  Uniform half-CMC card for both SIU (= Source Interface Unit) and DIU (= Destination Interface Unit)  Gbit/s components (Agilent) for 200 MB/s bandwidth  bridge up to 200 meters DDL patch panel with D-RORC  Two D-RORC types: 1 DDL link card plugged in, 2 integrated DDL channels  PCI 64 bit, 66 MHz  ALTERA FPGA, IP core from PLDA  200 MB/s throughput with DATE

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Rackmount PCs Evaluation/Benchmarking itemLDCGDCDSS Rackmount height1U, 2U, 4U1U4U CPU chipset motherboard dual Xeon 2.4 GHz E7501 Supermicro X5DPE-G2 dual Xeon 2.66 GHz E7501 Supermicro X5DPI-G2 quad Xeon 1.5 GHz ServerWorks GC-HE Intel SRSH4 Main memory2 GB DDR 266 MHz 4 GB DDR 266 MHz Ethernetdual Gigabit Ethernet on-board Gigabit and Fast Ethernet on-board Disk40 GB IDE hot-swap 36 GB SCSI disks PCI slotsup to 6 PCI-X for D-RORC cards 1 PCI-X for Fibre Channel card 6 PCI-X 2 PCI 5V Linux installationRed Hat 7.3  Red Hat 7.3 (rather old)

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Disk Array Systems front view (IFT-6330) rear view (IFT-6330)  Infortrend IFT F2D:  12x 300 GB IDE-based disks  dual 2 Gbit/s Fibre Channel ports  Embedded manager via front panel, RS232C, or IP  2x logical volumes: each one has 1.1 TB, RAID5, with 5 disks  DotHill SANnet II 200:  12x 36.7 GB FC-based disks  dual 2 Gbit/s Fibre Channel ports  RAID configurations via RS232C

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DAQ Software: DATE  DATE (= Data Acquisition and Test Environment) is the software framework of the ALICE DAQ.  Runs on Linux platforms (e.g. Red Hat 7.3)  Released (latest is version 4.7) and documented  DATE evolves with requirements and technology  Features/Packages:  Dataflow: DDL readout, event building  System configuration and Run control  Monitoring: on-line performance, on/off-line data quality  Utilities: message logger, control/configure front-end electronics, …

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Readout with DATE TPC RCU 3 with SIU D-RORC with DIU DDL  FEC  LDC  Step 0: Hardware is in place  Step 1: Run Linux with the physmem and the rorc driver  Step 2: Install DATE with a general configuration for the LDC  Step 3: Set up configuration file to enter parameters like D-RORC serial number, channel number, page size, checks, etc.  Step 4: Start/Stop the acquisition  Step 5: Analyse the raw data files, e.g. /tmp/run , /tmp/run , …

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DATE Run Control

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb FEE Configuration/Control TPC RCU 3 with SIU D-RORC with DIU DDL  FEC  LDC  Step 0: Hardware is in place  Step 1: Run Linux with the the rorc driver on the LDC  Step 2: Write a script (ASCII file) for the FeC2 interpreter  Step 3: Run this script, e.g. type FeC2 123 example.fec2 # send command, 19 bit information write_command 0x10f # read status, 19 bit address, 19 bit reply read_and_print 0x100 “Register XYZ: 0x%x” # download data block, 19 bit address, file write_block 0x600 pedestal.hex “%x” # verify data block, 19 bit address, file read_and_check_block 0x600 pedestal.hex “%x” stop_if_failed -1 S/N of D-RORC

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb On-line Monitoring  MOOD Framework:  DATE + ROOT environment  Interfaces to detector code  MOOD Applications:  Visualization, e.g. TPC sector test data (Hall 167)  Data integrity  Detector performance  Execution:  DSS machine

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DAQ Performance  ALICE Computing Data Challenge:  1750 MB/s event building without recording, 5 days  280 MB/s event building and recording to CASTOR, 7 days  D-RORC readout of one DDL channel with DATE: 206 MB/s readout only 118 MB/s readout with event building

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb DAQ Installations DAQ SystemLocationDDLLDCGDCNetworkingStorage Development / EvaluationAID lab244Gb Ethernetdisk DDL DevelopmentDDL lab210Gb Ethernetdisk Detector Groups (e.g. SDD, HMPID, TOF, Muon, TRD) Institutes110Fast/Gb EthernetDisk TPC Sector Test § PS T10210Fast Ethernetdisk or CASTOR ALICE Data ChallengesCERN IT080 Gb EthernetTDS + CASTOR Reference SetupAID lab642Gb EthernetTDS Test&CommissioningPoint Gb EthernetTDS + CASTOR Final SystemPoint Gb EthernetTDS + CASTOR § Given that the RCU 3 is fully functional in time, see Joachim’s presentation.

DAQ Test&Commissioning at Point 2TPC Collaboration meeting, Feb Summary + Future Work  Objective:  DAQ system for Test&Commissioning at Point 2  Surface tests of ALICE sub-detectors (e.g. TPC sector)  Operational in Q  Installation Layout:  20 DDL links between the PX24/CR1 counting room and the SXL mounting hall  L3 racks: DDL patch panels, 15 rackmount PCs, 1 disk array, switches for Gigabit Ethernet, Fiber Channel and KVM  DAQ Reference System in the AID lab  Future work concerning TPC and DAQ:  Bergen: Development of RCU 3 firmware for the DDL interface  Luciano’s lab: Integration test of RCU 3 with the DDL  TPC sector test at T10 § : DAQ chain with RCU 3 + DDL + DATE