1. The trigger-less readout for the Mu3e experiment Dirk Wiedner On behalf of the Mu3e collaboration 31 March 20161Dirk Wiedner

2. Readout Requirements 31 March 2016Dirk Wiedner2 100 MHz muon decays 50 ns readout frames (pixel) O(5000) pixel chips O(7000) scintillating fibers O(7000) timing tiles Online filtering

3. Timing Detectors 31 March 2016Dirk Wiedner3 Scintillating fiber hodoscope Timing tiles On detector zero- suppression O(7000) fibers O(7000) tiles

4. Silicon Pixel Detector 31 March 2016Dirk Wiedner4 Inner double layer Outer double layer Re-curl layers o Both sides (x2) Sensor size o 2x2 cm 2 108 inner sensors 4680 outer sensors

5. HV-MAPS 31 March 2016Dirk Wiedner5 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

6. HV-MAPS 31 March 2016Dirk Wiedner6 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

7. HV-MAPS 31 March 2016Dirk Wiedner7 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm Integrated readout electronics o Zero suppression o 1.25Gbit/s serial LVDS outputs by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

8. HV-MAPS 31 March 2016Dirk Wiedner8 H igh V oltage M onolithic A ctive P ixel S ensors HV-CMOS technology Reversely biased -85V o Charge collection via drift  Fast O(1 ns) o Thinning to 50 μm Integrated readout electronics o Zero suppression o 1.25Gbit/s serial LVDS outputs by Ivan Peric I. Peric, A novel monolithic pixelated particle detector implemented in high- voltage CMOS technology Nucl.Instrum.Meth., 2007, A582, 876

9. Pixel Readout Scheme 31 March 2016Dirk Wiedner9

10. Pixel Readout Scheme 31 March 2016Dirk Wiedner10 Pixel logic o Pixel address (8 bit) o Fine time (8 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Fine time Coarse time Coarse time Column address Column address

11. Pixel Readout Scheme 31 March 2016Dirk Wiedner11 Pixel logic o Pixel address (8 bit) o Fine time (8 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Frame logic Readout buffer Serializer 8 bit Fine time 8 bit Coarse time Coarse time Column address Column address

12. Pixel Readout Scheme 31 March 2016Dirk Wiedner12 Pixel logic o Pixel address (8 bit) o Frame number (4 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Frame logic Readout buffer Serializer Fine time Coarse time Column address 8 bit 32 bit

13. Pixel Readout Scheme 31 March 2016Dirk Wiedner13 Pixel logic o Pixel address (8 bit) o Frame number (4 bit) o 50 ns frames Column logic o Pixel data o Column address o Coarse time Readout buffer Serializer and fast link(s) Pixel address Pixel Logic Column Logic Frame logic Readout buffer Serializer Fine time Coarse time Coarse time Column address Column address 32 bit 4 x serial @ 1.25 Gb/s

14. Data Link Scheme From detector slices to time slices 31 March 2016Dirk Wiedner14

15. Link Overview 31 March 2016Dirk Wiedner15 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm

16. Link Overview 31 March 2016Dirk Wiedner16 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Pixel Detector FPGAs Readout board x12 Readout board x12 PC x48 PC x48

17. Link Overview 31 March 2016Dirk Wiedner17 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor FiberTile Pixel Sensor FiberTile Pixel Sensor FiberTile Pixel Sensor FiberTile Silicon FPGAs x60 Fiber FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 x5000x7000 PC x48 PC x48 O(8Tbit/s)

18. Tile Link Overview 31 March 2016Dirk Wiedner18 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Fiber FPGAs x48 Fiber FPGAs x48 Tile FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 PC x48 x240

19. Link Overview 31 March 2016Dirk Wiedner19 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Fiber FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 PC x48 x240x192 O(4Tbit/s)

20. Link Overview 31 March 2016Dirk Wiedner20 Front end links o Pixel sensor to on-detector FPGA 1.25bit/s LVDS o Timing detector readout Optical links from detector o Front end FPGAs o … to readout boards o 5 Gbit/s Optical links in counting room o Off-detector read out boards o …to PC Farm Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Pixel Sensor Fiber Tile Silicon FPGAs x60 Silicon FPGAs x60 Fiber FPGAs x48 Fiber FPGAs x48 Tile FPGAs x48 Tile FPGAs x48 Readout board x16 Readout board x16 Readout board x8 Readout board x8 Readout board x8 Readout board x8 PC x48 x192 x96 O(4Tbit/s)

21. Front End FPGAs 31 March 2016Dirk Wiedner21 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

22. Front End FPGAs 31 March 2016Dirk Wiedner22 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

23. Front End FPGAs 31 March 2016Dirk Wiedner23 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

24. Front End FPGAs 31 March 2016Dirk Wiedner24 FPGAs on detector o 60 (+96) pieces Receive sensor data o 45 LVDS inputs 5 Gbit/s outputs o 8 optical links o … to counting house Switching data between readout boards farms A-D Front end FPGA 1.25 Gbit/s LVDS in x 45 5 Gbit/s optical Readout board A Readout board A Pixel Sensor Readout board B Readout board B Readout board C Readout board C Readout board D Readout board D

25. Front end FPGA Switching Board 31 March 2016Dirk Wiedner25 FPGA readout boards o 4 per sub-detector 5 Gbit/s optical inputs o 48 inputs 10 Gbit/s optical output o 12 outputs to PCs Switching network o A-D sub-farms o One output per PC Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Sub-farm A Front end FPGA Front end FPGA Front end FPGA PC x12

26. Switching Board 31 March 2016Dirk Wiedner26 FPGA readout boards o 4 per sub-detector 5 Gbit/s optical inputs o 48 inputs 10 Gbit/s optical output o 12 outputs to PCs Switching network o A-D sub-farms o One output per PC Front end FPGA Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Front end FPGA Front end FPGA Front end FPGA PC Sub-farm A x12

27. Switching Board 31 March 2016Dirk Wiedner27 LHCb PCIe40 PCB Marseille First prototypes available Front end FPGA Switching Board 5 Gbit/s Optical x28 PC 10 Gbit/s Optical PC Front end FPGA Front end FPGA Front end FPGA PC Sub-farm A x12

28. GPU-PC 31 March 2016Dirk Wiedner28 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s FPGA PCIe board GPU computer Optical mezzanine connectors

29. GPU-PC 31 March 2016Dirk Wiedner29 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s GPU computer

30. Readout board GPU-PC 31 March 2016Dirk Wiedner30 PC with GPU 10 Gbit/s Fiber input o 8 inputs from sub- detectors Data filtering o Timing Filter on FPGA o Track filter on GPU o Data to tape < 100 MB/s PC 10 Gbit/s Optical x8 PC 10 Gbit/s Optical x8 A Readout board B x8 Readout board

31. Vertex Filter 31 March 2016Dirk Wiedner31 Entire event on GPU Large target o Large spread of muons o Easy vertex separation Reject data without three tracks o … inside area interval on target 1 3 2

32. Vertex Filter 31 March 2016Dirk Wiedner32 Entire event on GPU Large target o Large spread of muons o Easy vertex separation Reject data without three tracks o … inside area interval on target 1 3 2

33. Summary Mu3e has 300M pixels @ 10 8 muons/s 1 Tbit/s data 0-suppressed serial data from active pixel sensors Switched optical network GPU filter farm with optical inputs 31 March 2016Dirk Wiedner33 ++