S.Anvar, V.Gautard, H.Le Provost, F.Louis, K.Menager, Y.Moudden, B.Vallage, E.Zonca, on behalf of the KM3NeT consortium 1 IRFU/SEDI-CEA Saclay F Gif-sur-Yvette, France 1 Supported by the European Commission through FP6 and FP7 TWEPP-11, Vienna, Austria, September 2011

The KM3Net Detector Digital Optical Module (DOM) 31 PMTs/DOM DOMs 320 lines Technical Design Report (ISBN ) Conceptual Design Report (ISBN )

The DOM processor board HV PMT base Signal collection board Processor board & Software designed at CEA- IRFU Power conversion board Heat conductor PMT DOM designed at NIKHEF institute (Amsterdam, Netherlands)

RTOS DDR2 Memory (64 MB) Flash Memory (32 MB) FPGA Bitstream / Processor Boot The KM3Net prototype Offshore Processor Board PPC440 processor Slow Control Slow Control Task Slow-Control for the Storey (I2C, SPI) 1Gb/s Ethernet Link To shore station Data Task Data Readout Logic Readout System On Chip (RSOC) / VIRTEX-5 SC Protocol Logic Clock /Command Extraction PMTs x 31 small PMTs 31 x 1 GHz TDC 31 TOT Data

Firmware TDC Development 1 GHz Sampling PLL 500 MHz /500 MHz 125 MHz Recovered ETHERNET XILINX VIRTEX-5 ISERDES Readout MHz ‘1’ Detection & Coarse Time Stamping TOT LVDS PMT asynchronous input 8-bit 0 Packetized and Sent over ETHERNET TDC #16-bit Coarse Time Stamp #24-bit Event type/TDC number [Originally designed by A.Zwart (NIKHEF) / small PMTs test bench for ALTERA]

Clock distribution Transmit Reference MHz STANDARD 1000BASE-X ETHERNET Phase and Latency Measurements Command (CMD) insertion Custom Logic On Shore VIRTEX-6 Board IDLE CMD DATA Command(CMD) extraction Custom Logic Off Shore VIRTEX-5 RSOC 1.25 Gbps PPC440 Bus Recovered MHz / Bit slice analysis (known latency) CMD IDLE DATA Event Time Stamping Embedded 62.5 MHz

Transmit/Receive clock skew Tx/Rx clock skew measured on shore with DMTD and oscilloscope (ns) DMTD (Dual Mixer Time Difference) Scope

Store & Forward Acquisition model 31 TDC & Routing Matrix Dynamic Memory Acoustic Event Format (128 bytes) TDC event format (6 bytes) Circular Buffer FIFO 0 PMT 0-31 PPC MHz 1 Gb/s Ethernet Port 134 ms Data Time Slice To Computer 1 Time Slice To computer 2 ms (Typical) Slow control (Configuration, Readout on requests) 134 ms Data Detector Clock (ETHERNET) Slow Control Objects FIFO 5 DATA ROUTER Readout Logic Acoustic Data FIFO 6 FIFO 0 Circular Buffer FIFO ms

Time Slice Building Intrinsic Parallelism Performance Parallelism Offshore Nodes Onshore Farm Multi-gigabit Standard Ethernet Switching Clock Distribution/Commands Insertion Time Slice 2 Time Slice 1 Time Slice 0 Start

Network Embedded Embedded Communication middleware (ICE) Communication Communication middleware (ICE) Electronics control & config

Data acquisition setup Reference MHz STANDARD 1000BASE-X ETHERNET Start Command On Shore VIRTEX-6 Board Run Control Target Configuration Data acquisition vxWorks RTOS boot server Host PC Synchronous 1000BASE-X ETHERNET RSOC VIRTEX5FX-70 vxWorks RTOS 1 TDC Channel 1 dedicated TDC ASIC channel MHz MHz Pulse generator

Acquisition results 8 ns 2 ns Standalone measured TCP/IP throughput MHz / MHz /WindRiver Zero Copy buffer /Jumbo frames : 988 Mb/s Current acquisition setup (Source limitation): 60 Mb/s f=10 kHz ns Scope measurement Mean : 100,10667 us RMS : 1,75 ns Power : 7 W

CONCLUSION  Common Readout system functions integrated in a single component (RSOC): Event Time 1 GHz Clock and command distribution Slow-control and data acquisition performed in a RTOS multi-tasking embedded system  RSOC is a node designed to be plugged in a complete Data acquisition System Server/Client topology (ICE) Scalable system