Integrating Timepix(3) Szymon Kulis, Mathieu Benoit, Bas van der Heijden, Frans Schreuder, Henk Boterenbrood, MvB and the Timepix3 designers Xavi Llopart, Tuomas Poikela, Vladimir Gromov, Francesco Zappon Massimiliano de Gaspari and others 10 December 2013

Timepix versus Timepix3 Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom2 TimepixTimepix3 55x55 um 2 pixels 256x256 matrix = 1.4x1.4 cm 2 ToA or ToT measurementToA and ToT simultaneously frame based readoutdata driven readout zero-suppressed frame readout possible max. 120 frames/sec, with 100 tracks/frame = ~10k tracks/s (more is possible) (tens of) millions of tracks/s theoretical max of 80 Mhits/s dead time due to read out of single frame: 8+ ms dead time less not radiation hardradiation hard; several hundred Mrad but not SEU robust FitPix or Relaxd readoutSPIDR or FitPix readout

Timepix integration in common DAQ  Detailed talk given by Mathieu Benoit in WP9.3 meeting on Nov. 21  Lots of info in that talk, will not repeat it here. Please look at  Main bottle-neck: Timepix is either in acquisition mode, or in readout mode  Readout time of 8 (or more) ms per frame -> long dead time  Operation of FitPix and Relaxd is similar Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom3

Our (Nikhef/LHCb) focus is on Timepix3  In view of the LHCb VeloPix developments (ASIC submission aimed for Q3 2014)  VeloPix which has to cope with 600 Mhits/s  But we also have a complete and working Timepix/Relaxd telescope Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom4

SPIDR Speedy PIxel Detector Readout Readout system for Timepix3 and MPX3.1/RX over 10 Gb or 1 Gb Ethernet Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom5

SPIDR Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom6  Firmware properties: Vendor independent and highly configurable 1 Timepix3 at full (80 Mhits/s) speed, or multiple Timepix3 chips at lower speed LFSR lookup tables in FPGA Pixel data over UDP/IP, slow control over TCP/IP  Currently running on a development system (Xilinx VC707) Multiple setups running at Nikhef and CERN  Development of Compact SPIDR ongoing (but at a slower pace) FGPA USB 12V DC/DC 1V2 DC/DC 1V5 DC/DC 2V5 FMC PHY Ext bias +/- 400V Bias supply I2C SFP+ cage 1G/10G Expansion header Trigger/busy

Multiple TPX3 hardware configurations Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom7 Nikhef Compact readout Nikhef Compact readout Cern Chipboard Nikhef Chipboard FMC extender cable 50cm FMC extender cable 50cm FMC 2 VHDCI Xilinx VC707 development board Nikhef Chipboard FMC extender cable 50cm FMC extender cable 50cm Nikhef Chipboard FMC extender cable 50cm FMC extender cable 50cm (Thanks to Jerome Alozy) FitPix Cern Chipboard

Data acquisition  To run at high speed, each SPIDR needs its own DAQ PC A single timepix3 at max. speed produces 5.12 Gbits/s Including the extended time-stamp and overhead this fills ~70% of a 10 GbE link  Data is streamed to disk, without looking at the data Not possible to evaluate each packet at maximum speed Separate monitoring stream which samples (copies) the main data stream  DAQ Format: header of several kBytes (settings etc. ) followed by a stream of pixel packets, 8 bytes each Header format not yet frozen  No trigger/event number or alike  Pixels packets are unique by their time-stamp  Synchronisation and checking of different SPIDR data streams is important Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom8

Data acquisition II Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom9 FTOA 4b SPIDR time 16bToA 14bToT 10bhdr 4bPixAddr 16b FTOA 4b SPIDR time 16b ToA 14b time stamp 30 bits, 25ns resolution + 4 bits, 1.56 ns resolution range: sec  However 26 seconds is too short for a normal run  Timepix3 has an internal 48 bit counter, which is reset with the T0-sync resolution 25 ns, range days  This timer can be read on request, and has unique packet header  -> timer data will end up in the data-stream  Hence to cover a larger time range, the (Leon) processor in the SPIDR FPGA will request readout of this timer every second.

Running with multiple SPIDRs  Required TLU functionality can be very basic  Provide clock, shutter, T0-sync and combine busy signals (OR)  Somewhat more advanced functionality required for high speed monitoring (next slides) Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom10 TLU SPIDR repeater busy clock, T0-sync, shutter

Interface to TLU, sync time-stamp  Synchronising time-stamps Disable shutter Send T0-sync, min. 25 ns Enable shutter Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom11 25 ns (or more) > 1.6 us shutter (ext) tpx3-reset (int, optional) T0-sync (ext) n n-1 n-2 time-stamp no data packets data-out w.o reset data-out with reset > 25 ns

TLU interface, Busy  Data flow is controlled/halted via shutter/busy mechanism  Busy is tied to ‘almost full flag’ of SPIDR ethernet buffer  DAQ PC signals busy/overflow by sending pause-frames to SPIDR -> SPIDR buffers fill up, leading to the SPIDR pulling the busy  In addition DAQ PC can send a ‘halt’ command to Leon processor This will pull also pull the busy line  Will be implemented soon Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom12 shutter (ext-in) busy-0 (out) no data packets data-out1 busy-1 (out) no data packets data-out0

Monitoring of data streams  Monitoring of data streams by copying a fraction of the data  Send data to a dedicated monitoring/slow control PC  Two sampling methods  Software sampling Each DAQ PC takes a snapshot at a given time (CPU timer triggered, or via run control?) However DAQ streams are not synchronised -> need a large snapshot to guarantee overlap between streams  Hardware sampling Dedicated ethernet packets (different destination IP address, and/or port number) are generated by SPIDR Copies data from a range of timestamps to this dedicated monitoring packet Start of monitoring sample could be controlled by TLU Length of monitoring sample defined by SPIDR setting This is foreseen, but not yet implemented Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom13

Summary & Plans  Development of Timepix3 readout is actively ongoing (Nikhef + CERN)  But building a high speed readout is not trivial (so we have to be patient)  We (LHCb) are building a high speed Timepix3 telescope Timepix3 is good exercise for VeloPix which produces 8x more hits/tracks VeloPix has to be submitted Q3 next year  Integrating Timepix3 into an AIDA telescope seems simpler than integrating Timepix  Matching of data from different SPIDR streams using time-stamps Checking of synchronicity is key Separate monitoring data stream Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013Martin van Beuzekom14