1. By: Bas van der Heijden & Martin van Beuzekom
Timepix3 Readout
By: Bas van der Heijden & Martin van Beuzekom
3/20/2013
Nikhef

2. MPX3 and TPX3 readout -> SPIDR
Speedy PIxel Detector Readout
Development on a Xilinx evaluation board
design a custom (small form factor) FPGA board at a later time
Aim to read out 1 Timepix3 at full (80 Mhits/s) speed, or multiple Timepix3 chips at lower speed (less links per TPX3)
Readout bandwidth at least 5 Gbits/s -> use 10 Gigabit ethernet
currently using 1 Gigabit ethernet
10 Gbe VHDL code development started
MPX3.1 and RX readout up and running (low DAQ speed)
Re-use ‘back-end’ code for TPX3 readout
3/20/2013
Nikhef

3. Xilinx ML605 development board
Medipix3 chipboard
Xilinx ML605 development board
3/20/2013
Nikhef

4. By: Bas van der Heijden Nikhef
Timepix3 chipboard
HV for sensor/ grid
test fixture for
diff. probe
No routing
TPX3
lemo00
Power supply
1.5V
Test fixture 11x
150mm
69mm
FMC connector
3/20/2013
By: Bas van der Heijden Nikhef

5. By: Bas van der Heijden Nikhef
Timepix3 chipboard
FMC (high pincount 8gbt’s)
Mux for selecting between gbt and regular IO pins on fpga (PI2PCIE2412ZHE)
ADC & DAC for timepix I2C
Lemo’s (trig/clk/busy/?)
VDD(1.5V) VDDA(1.5V) VDDPLL(1.5) (from one DC/DC? CERN sm01c)
VDDA(3.3 efuse from FMC)
U&I monitor (ina219)
CERN rad-hard powersupply’s?
Flash for storing pixel mask (mp25p32 32Mb)
Simple bias supply for Si sensor? (max668) (12-100V)
Extra pad and connector/pad for HV bias (1000V) (thick sensor/grid+cathode)
Probing fixtures for 8x data_out/data_in/clk_in/enable_in
No routing behind chip (allows removing/thinning of PCB for beamtest)
Chip cooling?
3/20/2013
By: Bas van der Heijden Nikhef

6. Fast data transmission path
64MHz
(recovered clock)
125MHz
156MHz
MAX data rate is 5.12Gbit
lanes)
TPX
RST
TPX
CLK 8x
TimeStamp
14+20bits
2^34*25ns ≈ 7min
Unique
Pixel data to Ethernet packet State Machine
timestamp
20b
FIFO
68b
Depth=4097
8x36Kb EBR
Data
64b
Data
64b
Data
68b
Data
68b
Data
64b TX
BUF-FER
10GB
MAC
10GB
PCS
SerDes
640Mbit
CLK
64MHz
48b + 20b timestamp
Pixel word collector
CLK
10MHz
FIFO 68b (depth=16)
WORD MUX
Data
68b
sop
Port 1
sop
Data 8b
Data
68b
Empty
Control word DEMUX
Empty
Empty
eop
eop
Empty
RdReq
RdReq
RdReq
IP MUX
K-char
WRen
RdReq
XGMII
Remove control header
Send packet on timeout or max size
10G
Base X
Almost
full
Almost full RX
BUF-FER
eop
sop
Control data FIFO
64b
Throttle control
CPU
Port n
Pixel word 68b
Crtl 4b
Address 16b
ToA 14+4b
ToT 10b
Coarse Timestamp20b
3/20/2013
Nikhef

7. Fast data path summary
Add 20 bit timestamp to 48 bit pixel packets from TPX3
data of up to 8 serial links merged into one fifo (round robin)
strip off 4 bit ctrl header in ethernet packet builder
use 4 bit ctrl header to route data (to GBE / CPU)
GBE packet transmitted when either max. length reached or when time-out (configurable)
max. collection time given by MHz -> 7 minutes
3/20/2013
Nikhef

8. control data path 125MHz 156MHz 40/80MHz IP MUX RX Statemachine
Packet decoder
Enable
Data out
Front end
Data
Port 1
ClkIn
MUX
10GB
MAC TX
BUF-FER
10GB
PCS
64b
IP MUX
XGMII
10G
Base X
Throttle control
Veto
64b RX
BUF-FER
T0 sync
Control signals
CPU
Shutter
CPU Port
TPulse
ext_T0 & shutter
3/20/2013
Nikhef

9. control path summary Configuration / control via soft-core CPU (Leon)
Configuration directly from GBE also possible via Rx statemachine
Rx statemachine also for throttling (pause frames)
The Soft-CPU + firmware take care of the HW specific details
Testpulse generator
Shutter control
3/20/2013
Nikhef

10. Throttle structure FPGA PC Pause frame DAQ busy packet
Timepix3
FIFO
TX State-Machine
MAC+PCS
Network card
DAQ
application
10G
Base X
Shutter
FIFO almost full
Throttle control
Pause frame
FrontEnd disable
Other detectors
DAQ busy packet
(continue after …)
FE disable from other detector
3/20/2013
Nikhef

11. Hardware status + plan for next 3 months
2 hardware engineers full time + support for PCB design
MPX3+RX readout working
1 lane to MPX3, 1 Gbe to PC
not yet all modes are implemented/tested
further VHDL development is temporarily stopped; focus on TPX3 specific parts
Aim to have ready before TPX3 returns from fab:
Chipboard
10 GBE (fallback to 1 GBE possible)
VHDL statemachine, interface to TPX3
Pixel packet receiver and packet builder for 10 GBE
Later this year:
features and testing max. data throughput
custom FPGA board
tileable chipboard for MPX3 and TPX3 (2 versions)
3/20/2013
Nikhef

12. Software status and plans
Simultaneous code development for soft-CPU in FPGA and driver library at PC side
Library (“API”) for MPX3+RX functions almost complete, being tested/debugged at the moment
Low level TPX3 calls will be added the coming months
Testprograms (“scripts”) are simply a collection of calls from the library
Calls like: “SetDACs”, “openShutter”, “ReadFrame” etc.
written in C/C++, compile before running (MS Visual Studio Express 2010, free version)
Pixel data written to disk and /or on primitive display
decoder for offline display (Root) will be provided
Additional layer of code will be added to adapt to Pixelman API
High speed DAQ program will follow later this year
3/20/2013
Nikhef

13. Software tools (V)HDL graphical entry tool: Ease from HDLworks
Simulation: Questasim latest version
Synthesis: Xilinx ISE version 14.x
Use of SVN
Software development: MS Visual Studio Express 2010
free version
3/20/2013
Nikhef

14. We would like to have/know:
Final pinout / dimensions of TPX3 for chipboard
Up to date manual (working document?)
Simulation model of TPX3
Other
S-LVDS issue? converter needed?
3/20/2013
Nikhef