Niko Neufeld CERN PH/LBC

Detector front-end electronics Eventbuilder network Eventbuilder PCs (software LLT) Eventfilter Farm up to 4000 servers Eventfilter Farm up to 4000 servers UX85B Point 8 surface subfarm switch TFC x 100 Gbit/s subfarm switch Online storage Clock & fast commands 8800 Versatile Link 8800 Versatile Link throttle from PCIe40 Clock & fast commands 6 x 100 Gbit/s ECS Online upgrade status - Niko Neufeld2

Full event-building of every bunch-crossing (40 MHz)  no bottle-neck for event-building in the system DAQ, ECS and TFC relying on the same universal hardware module PCIe40 New DAQ with very challenging I/O in the event-builders and the network Cost-effectiveness dictates very compact system concentrated in new data-center, connected via long- distance links to the detector FEs ECS and TFC continued smoothly from current system (modulo the significant changes in the TFC due to the trigger-less nature of the read-out) Online upgrade status - Niko Neufeld3

Most compact system achieved by locating all Online components in a single location Power, space and cooling constraints allow such an arrangement only on the surface: containerized data- centre Versatile links connecting detector to readout-boards need to cover 300 m Online upgrade status - Niko Neufeld4

CPPM, Bologna, CERN (supporting)

PCIe40: the universal hardware module, a PCIe Gen3 x 16 card with up to 48 optical transceivers TELL40: a PCIe40 with the DAQ firmware with up to 48 GBT receivers SOL40: a PCIe40 with the ECS/TFC firmware (replacing SPECS and various TFC modules) with 48 GBT transceivers

Online upgrade status - Niko Neufeld7 slides by JP. Cachemiche

Online upgrade status - Niko Neufeld8

9

10 slide by F. Pisani

Online upgrade status - Niko Neufeld11

Bologna INFN and U., CNAF, CERN

Performances tests performed at CNAF with a test bed similar to the CERN one Exploiting the best performances required some tuning Bind processes according to NUMA topology and switch off power saving modes Very close to saturation ∼ 52.5 Gbit/s!!! Online upgrade status - Niko Neufeld13 A.Falabella et al

Extensive tests need to be done on a bigger cluster We aim at the new CINECA Galileo TIER-1 cluster Possible to test on a scale similar to the LHCb Upgraded DAQ network The cluster is in production from the last week of January 2015 First tests in few weeks managed by CNAF team ModelIBM NeXtScale Cluster Nodes516 Processor 2 8-core Intel Haswell 2.40GHz per node RAM 128 GB/node, 8 GB/core Network InfiniBand with 4x QDR switches Online upgrade status - Niko Neufeld14

CERN (thanks to technical coordinatioe team for their help!)

In real conditions First test – loop-back of AMC40 Soon – with front-end prototype Watch out for bit-errors / verify optical margin Use a MiniDAQ setup Online upgrade status - Niko Neufeld16

PCIe40 Patch cord court MPO(2m-5m) Câble ‘longue distance’ (300m) MPO-MPO Adaptateur Cassette MPO vers 12x LC ou SC Câble ‘longue distance’ (300m) Patch panel rack en souterrain x12 Config 1: Config 2: (repartition de charges) Fan Out court (2m-5m) PCIe40 MPO-MPO Adaptateur A B slide by L. Roy Online upgrade status - Niko Neufeld17

Online upgrade status - Niko Neufeld18

Online upgrade status - Niko Neufeld19

no error (12 links for 4 weeks) over 700 m More links and different tests to follow Online upgrade status - Niko Neufeld20

Annecy, CPPM, CERN + sub-detector experts

TFC & ECS firmware (SOL40) DAQ firmware (for MiniDAQ and PCIe40) Prototype ECS for MiniDAQ Global firmware framework