1. Data Acquisition

2. GLUEX GLUEX high intensity running, 5x107 γ/s :
July 2016 review highlighted issues with running at high intensity.
Event size dependency on intensity would produce unacceptable data rates.
GLUEX put in a lot of work to understand triggers and thresholds. Reviewed again in May 2017.
The focus this year - improving stability and throughput at high rates.
Spring 2016 the maximum stable throughput was ~300 Mbyte/s
Fall 2016 this was increased to ~1 Gbyte/s to disk (1.2 without disk).
Spring 2017 – two parallel data streams, build rate ~1.4 Gbyte/s per stream.
Summer 2017 – improve throughput within EB – lower load on systems.
Proposal
July 2016 estimate
May 2017 estimate
L1 trigger rate
100kHz
150kHz
90kHz
Average Event Size
15kB/event
52kB/event
17kB/event
Data rate (from EB)
1.5GB/s
9GB/s
<2GB/s
Date rate to tape (uncompressed)
0.15GB/s
2.25GB/s
Data volume on tape
0.42PB/y
6.75PB/y
4.8PB/y
Data volume on tape (Total)
2.3PB
37PB
26PB
Num. crates exceeding 1Gbit/s
none
>35
<=3

3. Improve DC to SEB flow
We had a nice fat pipe, 40 Gbit/s Infiniband but were struggling to get above 10 Gbit/s Why?
Answer:
“Under the hood” the OS assigns a single compute core to do the sending, this core is fully loaded!
Solution – allow DC to use multiple TCP connections in parallel.

4. Parallel Event Building
On the left is the scheme used in GLUEX so far.
For high intensity running the SEB and ER/disk are bottlenecks, this was anticipated.
Reconfigure to use two SEB and two disks, throughput doubles as expected.

5. Data compression
Gagik Gavalian implemented a compressed format, HIPO, for use by CLAS12
HIPO file layout is very similar to EVIO file format.
Carl Timmer (my group) and Gagik are implementing a compressed EVIO file format.
Writing keep up with GLUEX and CLAS12 rates and not to impact data rate.
HIPO is in areas simpler than EVIO – need to support use by all halls.
The new EVIO library must still read older data files.
Didn’t want to rush something out for the upcoming run.
Will be an option for Fall 2018.
Can the machine running the ER for GLUEX compress as well?
William Gu, my group is investigating a hardware compression accelerator…
Commercial or home brew?

6. Shameless plug We are hosting the 2018 IEEE Real Time Conference!
June 11th through15th with workshops 9th & 10th.
Local chair – David Abbott.
Location – Williamsburg.