1. Takeo Higuchi (KEK); CHEP.20120521.pptx High Speed Data Receiver Card for Future Upgrade of Belle II DAQ 1.Introduction – Belle II DAQ Experimental apparatus parameters SuperKEKB acceleratorMaximum luminosity8x10³⁵ /cm²s Belle II detector Expected raw data size1MB + 80kB Maximum L1-trigger rate30 kHz Dead-time fraction< 3% The anticipated faint signatures of the new physics effect are detected with a large number of e⁺e⁻ collisions and a dedicatedly configured hermetic and granular detector. Belle II detector and Belle II DAQ Near-detector signal digitizers and optical TX Opt TX Far-site optical RX ~700 RocketIO optical links COPPER readout system COPPER boards (x~180) ~20MB/s per link at most HSLB cards (x~700) x4 RX-card slots. Pipeline buffer (4MB). PrPMC slot for online processing. Local and PCI buses bridged by PLX9054 bus-bridge chip. x2 GbE data-output ports. EVB (PC) HLT ~⅓ rate reduction Storage ~1.5GB/s Max link speed =3Gbps. Virtex-5 FPGA. 2.New Data Receiving System PXD2-layer DEPFET pixel detector SVD4-layer silicon vertex detector CDCCentral drift chamber TOPTime of propagation counters ARICHRing-image Cherenkov counters ECLElectromagnetic calorimeters KLMK L and μ detector TRGHardware trigger information Opt RX T. Higuchi et al., IEEE Trans. Nucl. Sci. 52, 1912 (2005). ~20m fibers Design concept of the COPPER board ~ Opt RX..PrPMC FIFO 9054 Local busPCI bus COPPER board (VME9U size) A new data receiver card DDR3 memory (up to 4GB) Virtex-6 (XC6VLX75T) Status indicators (LED) x2 optical input ports (SFP+) x2 LVDS output ports (RJ45) PCIe x4 Gen2 New data receiver card (PCI Express) Optical RX→The receiver card itself COPPER FIFO→On-card DDR3 Local/PCI buses→PC’s motherboard bus PrPMC→PC’s CPU COPPER GbE→PC’s GbE port Design concept of the new receiver card Use of commodities saves costs for software R&D and efforts for system maintenance. DDR3SFP+#1SFP+#2 PCIe x4 Gen2 Virtex-6 Memory I/F Generator FIFO (1k x 328b)FIFO (1k x 256b)FIFO (1k x 4b) AURORA 8b10b FIFO (1k x 288b) AURORA controller FIFO (1k x 288b) AURORA controller DDR3 controller DMA controller PCIe controller Integrated block LVDS outputs to notify the DDR3 getting full. 3.Performance Studies Prototype study Virtex-6 evaluation card (ML605) FMC Mezzanine card with x4 SFP+ Loop back New card’s performance study Loop back New data receiver card DMA PCIe x4 Gen2 Readout PC Corei7 3.4GHz x2 (PCIe x8 Gen3) 4GB main memory Scientific Linux 6.0 Optical link performance † BER < 1.8x10⁻¹³ DMA performance † BER < 1.9x10⁻¹³ Throughput = 635MB/s † 4MB per transaction The COPPER board can handle >30 kHz L1 trigger rate with a 90% dead-time reduction than Belle I proved in the Belle I DAQ operation. The COPPER system is widely adopted in many HEP experiments. The great success of the COPPER system: L1-asynchronous DAQ by FIFOOnline data reduction by CPU Disadvantage of the COPPER system: As the COPPER system embeds everything needed for the DAQ, the system became too complicated for the average Belle II users to start up and to maintain. Simple-minded data receiverMore use of commodities Block diagram of the new data receiver card.GbE Studies in cooperation with Sansei System Inc. DMA PCIe x8 Gen1 DELL 760MT Core2 Duo 3.0GHz PCIe x16 4GB main memory CentOS 5.5 16kB image pre- stored in ML605 Opt TX 8b10b Opt RX PC Loop back DMA/PCIe Optical link + DMA performance † Throughput = 470MB/s † Average over 8192 transactions We continue system R&D with a milestone settled in the middle of 2017, when the peak luminosity will exceed 2x10³⁵ /cm²s. Takeo Higuchi, Nobu Katayama, High Energy Accelerator Research Organization (KEK) Kavli IPMU, University of Tokyo