TPC Readout Electronics: Overview of Developments At Irfu over 2005-2015 Designing Next Generation Systems D. Calvet, Irfu, CEA Saclay, 91191 Gif sur Yvette Cedex, FRANCE

Plan PART I: T2K and subsequent completed developments PART II: On-going developments and future prospects Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

T2K TPC Readout Electronics AFTER chip Technology: AMS CMOS 0,35 µm Die area: 7,8 x 7,4 mm2 72 channels x 511 time-bucket SCA Front End Module 1728-channel readout block 6 Front-End cards (4 AFTER, i.e. 288-channel per FEC) 1 Front-End Mezzanine card Optical link for DAQ; CANbus slow control The AFTER chip is also used in T2K for the readout of the FGD Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

T2K TPC Readout Electronics Back-end electronics 18 DCC’s (124,416 channels) 2.5 kW power supplies for front-end electronics Slow control PC Trigger interface, etc. Data Concentrator Card - DCC Controls 4 FEMs,6912 channels Interfaces to DAQ PC via Gigabit Ethernet Customization of Xilinx ML405 evaluation kit (Triumf/Lpnhe/Irfu) Upgraded in 2014 with Virtex 5 ML507 (Winnipeg & Victoria U.) Overall project schedule Started in January 2005 by discussing AFTER chip specifications Completed design, test, production and full system installation on site of 124.416 channels in fall 2009 → 5 years, ~35 FTE, ~0.6 M€ shared among 4 institutes to build electronics, firmware and software Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

System Exploitation Operation history 1 FEM replaced in January 2011 and 3 FEMs replaced in July 2012 because of slow control problems. Cause: CANbus opto-coupler failure. Interruption of operation from March 2011 to April 2012 caused by great Tohoku earthquake November 2013: 1 FEC replaced due to ADC failure (data lost on 72 channels during several months), 1 FEM replaced due to CANbus opto-coupler failure. Patch applied opto-coupler of all FEMs, also replaced PC for slow control due to failure of some USB ports One LV power supply crate replaced in 2014 All 18 Virtex 4 ML405 DCCs replaced in 2014 by Virtex 5 ML507 (Winnipeg & Victoria University) To date: all channels operational except 144 pads disconnected due to high detector sparking rate → Still have ~40 FECs and ~8 FEMs in spare for future maintenance Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Prototype Electronics for ILC TPC 7-Module Micromegas Demonstrator Front end Electronics – FEMi and FECi Extremely compact design for FECi using chip bounding for the 4 AFTER chips Readout of 1728 channels in a module of 200 cm2 area x 3 cm thickness 19 W (11 mW/channel) 40% less power consumption than T2K for same channel count Volume reduction by 10 compared to T2K module Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Prototype Electronics for ILC TPC 7-Module Micromegas Demonstrator Test-beam setup at DESY… …based-on T2K electronics …based-on FEMi/FECi electronics Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

The GET Readout System Project outline Intelligent Trigger (GANIL) ASIC « AGET » (IRFU) Front-end electronics (CENBG) Configuration, Monitoring, Run control DAQ software (IRFU,GANIL) Back-end electronics (HW: MSU SW: IRFU) Project outline Joint project 2010-2014 between CENBG (Bordeaux), IRFU (Saclay), GANIL (Caen) in France, MSU (United States) and RIKEN (Japan) to develop a generic readout for standard and active target TPC’s Funding received from the French National Research Agency (ANR) in France and NSF in USA Commercially available; > 100.000 channels installed worldwide Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016 | PAGE 8

FRONT-END ASIC: from AFTER1 to AGET2 1. built for the T2K neutrino oscillation experiment 2. Built by the GET collaboration for active target TPCs 12-bit ADC Serial Interface Mode CK In Test CSA;CR;SCAin;DISCRIin(N°1) “Spy” Mode Readout SCA control SLOW CTRL W / R TEST Reset AGET 512 cells SCA FILTER tpeak CSA 1 channel 64 channels ADC Charge range DAC Discri inhibit BUFFER x68 Hit register Σ 64 discri. SCAwrite Trigger pulse Design facts of AGET 0.35 µm AMS CMOS 28 x 28 mm 160-pin LQFP Pin-compatible with AFTER (8 channel less) Main characteristics 72 64* channels, pos/neg set by register, 120 fC to 10 pC (4 ranges), 100 ns-1 µs peaking (16 values) 511 512 time-bin SCA. Sampling: 100 MHz max. Requires external ADC, 12 bit / 25 MHz max Self trigger: 1 discriminator per channel; Analog multiplicity signal. Can be used to make L1 trigger Hit Channel Register: hit pattern can be used to make L2 trigger. Hit pattern can be modified before SCA digitization, e.g. enable channels which are neighbor to hit channels SCA readout modes: all channels, hit channels only, channel set defined on-the-fly Optional internal charge sense amplifier and filter bypass for use with external preamplifiers SCA split mode: 2 banks of 256 time-bins. Can take 2 triggers close apart before digitization Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016 * In red: differences between AFTER and AGET

Minos – A new instrument for in-beam spectroscopy of exotic nuclei Cryostat DALI2 gamma spectrometer Readout Electronics MINOS Tracker and TPC DSSSD Project proposed by A. Obertelli and financed by ERC grant 2011-2015 4K channel TPC with Micromegas amplification + optional DSSSDs Does not require sophisticated self trigger capability. Needed a development cycle shorter than GET → Development of the FEMINOS system, an evolution of T2K electronics to exploit AGET more rapidly and at lower cost compared to GET Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

The Feminos: AN Evolutive system compatible with after and aget T2K FEC: 4 AFTER chips 288-channels FEMINOS Board ×24 max. i.e. 6K channels OR Main features For small to medium scale systems 6K channels Close to the lowest possible dead-time of AFTER and AGET to reach high event rate External trigger or basic self trigger on multiplicity Low cost system in a light infrastructure MINOS FEC: 4 AGET chips (pin-compatible with AFTER) 256-channels Feminos system used in ~10 different projects Production halted. Improved version being designed Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

MINOS TPC In Operation HIMAC, Japan, Oct. 2013 Typical events during a physics run at RIKEN TPC read out by 20 AGET FEC and Feminos HIMAC, Japan, Oct. 2013 Test Beam for the qualification of MINOS TPC TPC read out by 20 AFTER FEC and Feminos (AGET still in development at this time) Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Plan PART I: T2K and subsequent completed developments PART II: On-going developments and future prospects Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

readout ASIC development roadmap 2006 2010-2014 2016 2018+ DREAM 1 ?? ASTRE AFTER and AGET: used in T2K and many other experiments DREAM: Dead-time-less Readout Asic for Micromegas: evolution of AFTER/AGET for Trackers. Simultaneous read and write in SCA. Used for Clas12 upgrade at Jefferson Lab and other projects ASTRE: version of AGET for space applications. Other improvement: shaping time up to 8 µs (e.g. for slow TPCs, e.g. pressured Xe for 0-neutrino bb decay experiment). Chip fabricated under test now. Possible future: an improved AFTER for T2K phase-II? E.g. integrated ADC, smaller package. → Idea under discussion internally, also to be discussed with the T2K collaboration Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Front-END card development roadmap 2006 2010 2013 2016 288-channel AFTER FEC (+FEM) 288-channel AFTER FECi (+FEMi) 288/256-channel AFTER-AGET FEC + Feminos 256-channel AFTER-AGET-ASTRE Read Out Card (ROC) 512-channel DREAM Front-End Unit (FEU) The ROC combines a FEC and Feminos on a single board. Supports AFTER, AGET and ASTRE. 256-channel standalone ROC with Gigabit Ethernet readout for small systems Scalability up to 24 ROC using Feminos TCM and 32 ROC using back-end board TDCM (in design) → ROC layout in progress; prototype production expected by end 2016 Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

back-END card development roadmap 2006+ 2010 2013 2016-2018 ML405/507-based DCC Virtex 4/5 - 4 optical ports ML523-based DCC Virtex 5 - 12 optical ports TCM (clock/trigger only; data through Gbe switch) Spartan 6 - 24 electrical ports TDCM (clock/trigger & data) Zynq – 32 optical ports or other physical layer Our latest development is the Trigger and Data Concentration Module (TDCM) Custom-made carrier board + commercial FPGA ZYNQ module + up to two physical layer mezzanine cards providing a total of 32 link ports for front-end 16-port mock-up currently running on PCIe carrier + FPGA module + 1 physical layer mezzanine card TDCM carrier to be designed in 2017 for installation on 1st TPC of PandaX-III experiment in China Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Towards a TPC readout System for T2K-II Concepts and goals Build on experience gained over last decade of developments Evolution of proven concepts, re-use whenever possible Reduce manpower effort, lower cost, minimize technical risks Hypothesis Use of resistive Micromegas could allow reduction of number of pads without affecting resolution. Current Micromegas: 36 × 48 pads; Hypothesis for new resistive Micromegas: up to 32 × 36 pads 4 TPCs × 2 end-plates × 8 Micromegas modules × up to 1152 pads – 74 K pads max. total No change on specifications (resolution, charge range, drift velocity, trigger rate, etc) A possible architecture Essentially identical to that of current TPC readout: Four 256-288 channel FECs plugged directly (no cable thanks!) at the back of each Micromegas One Front-End Mezzanine card to drive the 4 FECs of each detector module Four back-end boards to connect to the 64 FEMs (one back-end board per TPC) Standard PCs and networking for data acquisition, global event building, run control etc. Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

What to Improve on the FECs? Replace fragile ERNI connector by a more robust one Protection circuit not needed with resistive Micromegas. More compact card. Lower cost: 0.3€ * 74.000 = 22 k€! AGET or ASTRE: no real benefit over AFTER (64 channels vs. 72) DREAM: smaller package but also 64 channels only Stay with AFTER ? Design a new chip ? Remove the ADC. Integrate it in the ASIC, or put it on the FEM side. Board size reduction Lower cost Supply higher power voltage than 5V. Through FEM connector? Reduces power loss and cabling If FEC size reduction is sufficient and Micromegas size is unchanged while channel count is reduced to ~1000, it may be possible to mount FECs parallel to the detector instead of perpendicular 512-channel FECs based on DREAM have been built for Clas12. May be possible and more economical to have two 512-channel FECs per module instead of four 256-channel FECs. Integrate the FEM part on a 512-channel FEC? Only one type of board needed for the front-end. Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

What to Improve on the FEMs? CANbus slow control: was supposed to make the system more robust, but in reality all the problems found on FEMs came from this! Suppress. Do monitoring over the same optical link used for system configuration and data collection Bring 12V-24V instead of 5V DC/DC converters instead of LDO to improve conversion efficiency Channel count Reduction: 4 FECs instead of 6? Or two FECs with 512 channels each? Virtex-II Pro obsolete. Replace by a FPGA of 7th generation, e.g. Artix 7 cheaper, lower power consumption One 4-channel 25 MHz ADC on the FEM rather than one quad-channel 20 MHz ADCs on each FEC. Digitization in 4 phases * 4 AFTER in // instead of all 24 AFTER digitized in // (T2K current scheme) Advantages: 40% global power consumption reduction (mesasured on FECi/FEMi prototypes) and reduces ADC component cost Cons: Digitization time increased from ~2 ms to ~6.5 ms Reduce dead-time by restarting sampling immediately after digitization (Minos scheme) rather than wait until complete data transfer to DAQ (current T2K scheme) Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

What to Improve on thE Back-end SIDE and Services Change the paradigm: Development of full custom back-end board originally planned for T2K TPC/FGD was not successful Backup solution uses commercial evaluation kit with custom add-on card Proposed scheme for T2K-II: a commercial FPGA module + a simple custom made carrier + physical layer on main carrier or mezzanine TDCM will have physical layer part placed on 2 mezzanine cards. Supports up to 32 links, i.e. 55296 front-end channels, 8 fold increase compared to T2K DCC One TDCM per TPC in T2K-II. Four TDCM in total, fits in 1 crate instead of 3 presently. Spartan 6 / Artix 7 / Zynq 7020 supported on Minos TCM Zynq 7030-7045 will be supported on TDCM Use single fiber bi-directional transceivers rather than usual dual fiber transceivers: Halves number of optical fibers Power supplies for the front-end: Present T2K: supply 480 A at 6V to use 3.3V at the end: ~50% of 2.8 kW lost in transport and voltage regulators of front-end Future: expect more than 40% power reduction with sequential digitization scheme, further reduction by 2 if number of channels reduced to ~74K thanks to resistive Micromegas. Power distribution with 12V or 24V instead of 5V: reduces supply current by another factor 2 to 4. In total, cross-section of cables reduced by >10. Higher efficiency, lower cost for cabling, requires less cooling Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

T2K-II TPC readout Electronics work packageS Definition Example Front-end ASIC (design), production, test * Front-End Card design, production, test * Front-End Mezz. design, production, test * Front-End Mezz. firmware * (60 k€ new design) 60 k€ production 75 k€ 30 k€ Front-end ASIC Prod. test-bench * Front-end card Prod. test-bench * Front-end mezz. Prod. test-bench 5 k€ 5 k€ 5 k€ Back-end Board design, production, test Embedded Firmware & Software * DAQ hardware & software Mechanics, Cooling, Power supplies Cabling, … * Detector test & calib. Test-bench 20 k€ 10 k€ 15 k€ Cost estimates based on 74K channels with ~10% spares: 1200 front-end ASICs 300 front-end cards (256-288 channels) or 150 FECs 512-channel 75 front-end Mezzanine cards 6 back-end cards * Work packages with major contribution from Irfu in T2K phase 1 Project structure Work packages of various sizes and budget Irfu could take responsibility for some of them – not necessarily same perimeter as T2K phase 1 Funding and manpower has to be found / identified Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016

Conclusions Past and present Future Successful development of 125.000 channels of TPC and FGD in T2K phase I around the AFTER chip Successor AGET in full production and deployed in many experiments worldwide Portfolio of chips for TPCs: AFTER, AGET, DREAM, ASTRE Up-to-date hardware and firmware available at Irfu to exploit these chips with latest generation FPGA’s Future Discuss the opportunity to design a new ASIC for T2K-II. Still can produce new batch of an existing chip Substantial potential savings compared to current system have been identified Could re-use in T2K-II some existing (or soon existing) blocks: ASIC, back-end board, firmware… We may start at Irfu prototyping studies for T2K-II in 2017 if budget and manpower are found Discuss how the work on T2K-II can be shared among potential collaborators Prepare applications to funding agencies → Project will probably take at least 3 years to complete and requires ~0.3 M€ investment Workshop on Neutrino Near Detectors based on gas TPCs | Geneva, 8-9 November 2016