The Application of DAQ-Middleware to the J-PARC E16 Experiment E Hamada 1, M Ikeno 1, D Kawama 2, Y Morino 1, W Nakai 3, 2, Y Obara 3, K Ozawa 1, H Sendai 1, T N Takahashi 4, M M Tanaka 1, S Yokkaichi 2 1 High Energy Accelerator Research Organization (KEK) 2 RIKEN Nishina Center 3 The University of Tokyo 4 Research Center for Nuclear Physics, Osaka University 2015/4/14CHEP20151 Track 1 Session: #3 (Data acquisition and electronics) April 14, :00 – Village Center

Outline 1.DAQ-Middleware overview and features 2.DAQ System at E16 Experiment requirements and architecture 3.DAQ System Performance method and result 4.Summary and Future Plan 2015/4/14CHEP20152

1. DAQ-Middleware 2015/4/14CHEP20153

What DAQ-Middleware Is A framework for network based DAQ software –Easy to use, configure and develop Target –Medium-scale experiments –Test benches (sensors, electronics) 2015/4/14CHEP20154

DAQ-Middleware Architecture Develop DAQ system by configuring DAQ components 2015/4/14CHEP20155 Dispatcher Gatherer ・・・・・・ Read-out -modules PC XML System Configuration Daq Operator Logger Monitor Command / Status software unit that achieve DAQ functions

DAQ Component 2015/4/14CHEP20156 InPort OutPort Service Port (command/status) Logics (for data handling) Data InPort OutPort Service Port (command/status) (for data handling) Data + = DAQ-Middleware provides users writes Logics

DAQ Component Features 2015/4/14CHEP20157 READOUT Flexibility Reusability Scalability logic Ring Buffer ・・ ・ Users can flexibly change DAQ component combination. Users can improve performance by adding new PCs and deploying DAQ components. Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer.

List of DAQ-Middleware Users 2015/4/14CHEP20158 DAQ-Middleware Working J-PARC MLF Neutron beam lines Experiments [19 experiments] - Material and Life Science - J-PARC MLF Neutron/Muon - DAQ system of Depth-resolved XMCD (KEK PF) - Elementary Particle/Nuclear Physics - J-PARCE16 Experiment - CANDLES - SuperNEMO (planning) Test benches for sensors and electronics [9 test benches] –Liquid Argon TPC –SOI Pixel Detector –ILC CCD Vertex –ADC-SiTCP ・ 2D gaseous detector with readout ASIC using printing technologies and so on….

2. DAQ System at E16 Experiment 2015/4/14CHEP20159

Requirements 2015/4/14CHEP MB data receive 660MB data receive not receive 1spill not receive Trigger rate fluctuates due to beam rate variation. Event size per one event is almost constant. Estimation of data transfer to DAQ PCs Data rate per spill660MB/spill Trigger rate(average) 1kHz (max) 2kHz Instantaneous data rate (average) 330MB/s (max) 660MB/s

System Architecture 2015/4/14CHEP Function ・ Event build ・ Analyze data & show the result Read-out-moduleNetwork switch Storage PC Network switchMonitor PC HDD Function ・ Read data ・ Save data on each HDD ・ Send data of a part of event to Monitor PC

DAQ Component Configuration 2015/4/14CHEP Storage PC Monitor PC Gatherer Merger Dispatcher Logger Filter Merger Eventbuilder Monitor ComponentFunction GathererRead data MergerReceive data from multiple Gatherers DispatcherSend data to Logger and Filter LoggerSave data on each HDD FilterSend data of a part of event to Merger of Monitor PC ComponentFunction MergerReceive data from multiple Filters EventbuilderEvent build MonitorAnalyze data & show the result

DAQ Component Features 2015/4/14CHEP READOUT Flexibility Reusability Scalability logic Ring Buffer ・・ ・ Users can flexibly change DAQ component combination. Users can improve performance by adding new PCs and deploying DAQ components. Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer.

If data volume increase by detector upgrade, we add Storage PC. Advantage of DAQ Component Features 2015/4/14CHEP Flexibility & Scalability Reusability Ring Buffer Using DAQ component which is prepared, we saved time and effort for development. We can use no event time effectively. SampleDispatcher, Logger Gatherer, Monitor DAQ system receive process receive process

3. DAQ System Performance 2015/4/14CHEP201515

Purpose & Method 2015/4/14CHEP Emulator Storage PC Network switchMonitor PC HDD Emulator test data send data once every 10 events Purpose Evaluation of a total throughput using a DAQ-Middleware for the J-PARC E16 experiment Emulators run with 1 cycle per 6 seconds. During one cycle, emulators send test data for 2 seconds and do not send test data for 4 seconds. Monitor PC shows data value of a part of data regularly. We changed the number of emulators to change transfer data volume and measured processing data speed. Method

Result The result met the requirements of the E16 experiment! 2015/4/14CHEP requirement throughput (Trigger rate is always max [2kHz]. ) HDD write limit requirement throughput (Trigger rate is always average [1kHz]. ) During sending test data, one event data size per one emulator is 14kB. 2kHz 1kHz

4. Summary and Future Plan 2015/4/14CHEP201518

Summary and Future Plan DAQ-Middleware is a framework for network based DAQ software. DAQ component has following features. –Flexibility –Scalability –Reusability –Ring Buffer We have developed DAQ system for E16 experiment by using DAQ-Middleware. The requirements from E16 experiment have been met. In the future, we are going to connect DAQ system to read out module, and evaluate the system. 2015/4/14CHEP201519

Backup 2015/4/14CHEP201520

J-PARC E16 Experiment DAQ system overview 2015/4/14CHEP201521

Evaluation Environment 2015/4/14CHEP HDD PC1 PC2 PC3 CPUIntel(R) Xeon(R) 2.67GHz 6Cores Memory24GB Network1Gbps x 5 OSScientific Linux 6.4 HDD Hitachi HDS724040ALE6 4TB PC1 CPUIntel(R) Xeon(R) CPU E GHz 6Cores Memory32GB Network1Gbps x 5 OSScientific Linux 6.6 HDD Hitachi HDS724040ALE6 4TB PC2 CPUIntel(R) Xeon(R) CPU E GHz Memory8GB Network2Gbps OSScientific Linux 6.6 PC3 Cisco Catalyst 2960G

HDD write speed check 2015/4/14CHEP201523

SRS (Scalable Readout System) 2015/4/14CHEP SRS is general purpose multi-channel readout system. User can choose variety of frontend chips. FECADC FEC Front-End Card FEC Front-End Card FEC Front-End Card Eurocrate Front-End ASIC SRU

Evaluation by using SRS 2015/4/14CHEP SRS ADC/EFC ADC/FEC send test data. Maximum transfer speed of test data is 1Gbps + 1Gbps. We configured that DAQ PC could process all of data. Storage PC Network switchMonitor PC HDD test data send data once every 10 events

Storage PC Performance 2015/4/14CHEP Emulator Storage PC Test PC emulator CPU Intel(R) Xeon(R) 2.67GHz 6Cores Memory 24GB Network 1Gbps x 10 OS Scientific Linux 6.4 SSD Intel SSD520Series 240GB only receive data Throughput of 1 Storage PC is 1000MB/spill. write data on SSD test data trigger rate = 2kHz 1 event size = 14kB

DAQ Component & Configuration Example 2015/4/14CHEP InPort OutPort Service Port (command/status) Logics (for data handling) Data InPort OutPort Service Port (command/status) Logics (for data handling) Data += DAQ-Middleware provides user writes Monitor DaqOperator Gatherer Online Monitor only Examples of DAQ component combination Logger DaqOperator Gatherer Logging only Dispatcher Logger Monitor DaqOperator Gatherer Online Monitor & Logging