T2K Time Projection Chambers Front-end Electronics – Experience Return P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse, E. Delagnes, F. Druillole, A. Le Coguie, E. Monmarthe, J-M. Reymond, E. Zonca DSM/IRFU, CEA Saclay, France O. Ballester, IFAE, Barcelona, Spain

Saclay 2-3 December Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

Saclay 2-3 December Tokai to Kamioka (T2K) experiment Main Physics Goal: neutrino oscillation μ disappearance for improved accuracy on  23 e appearance to improve sensitivity to  kT water

Saclay 2-3 December T2K Time-Projection Chambers TPC features Resolution goal: 10% for p < 1 GeV/c Double wall design; inner wall is the field cage Total ~9 m 2 instrumented with 72 Micromegas amplification modules segmented in ~7 mm x ~10 mm pads Custom readout electronics 1 of 3 TPCs shown 1728 pad (36 x 48) Micromegas module 34 cm x 36 cm CalibrationGas systemElectronics; on-line software 1 m 2 m

Saclay 2-3 December TPC Readout Challenges Environment Embedded front-end: limited access, need low power No radiation, low magnetic field (0.2 T), Japanese underground facility Highly segmented detector channels over 9 m 2 of instrumented area → Massive replication of identical modular building blocks Extreme burstiness of data channels; sampling: 12-bit 33 MHz → 50 Tbps → beyond capability of fully digital solution for power budget target Large raw event size but modest average dataflow 50 Tbps during ~15 µs drift time → 90 MBytes Allowable event size for storage: ~250 KBytes → data reduction of ~400 to achieve in real time Spill repetition period: ~3.5 s; Cosmic calibration: 20 Hz max. DAQ average rate: ~250 KBytes × 20 Hz = 5 MByte/s → commercial computers and networking techniques

Saclay 2-3 December Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

Saclay 2-3 December Logical Read-Out Flow Architecture principles Custom front-end ASIC; analog memory (Switch Capacitor Array) ADC + digital buffer mounted close to the detector Multiple optical fibers send data to off-detector concentrator cards Interface to common DAQ via standard gigabit Ethernet network Pre-amp and shapers Samplers and analog memory buffers Analog to digital conversion Digital buffer – zero-suppression Data concentrator Clock/Trigger fanout ~ channels 1728 ASICs On-detector 72 Optical fibers ~4 Tbaud*/s peak *1 baud = 12 bit ~2 ms retention max. 34 Gbaud/s peak 400 Gbit/s peak ~1-10 Gbit/s averaged Shared DAQ ~0.1-1 Gbit/s 432 Front End Cards 18 Data Concentrator Cards 72 Front-End Mezzanines cards Off-detector Global Clock/Trigger Gigabit Ethernet

Saclay 2-3 December Architecture Implementation 3 TPCs 1 m 2,5 m Network TCP/IP PC Linux DCCs Private Ethernet Gigabit Ethernet DAQ control Detector B Detector A Global trigger 1 of 6 TPC end-plates (12-modules) Outside magnetInside magnet 2 of 12 (or 18) Data Concentrator Cards x 6 12 duplex Optical fibres 1 of 72 modules Front End Mezzanine Card (FEM) 288 channel Front End Card (FEC) 1728 pad Micromegas plane Slow control network Optical fiber to/from DCC Low voltage Power supply 1 of 1728 Front-End ASIC “AFTER” 72 channel x 511 time buckets Switched capacitor array

Saclay 2-3 December Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

Saclay 2-3 December ASIC “AFTER” Functional Diagram AFTER 511 cells SCAFILTER 100ns<tpeak<2us CSA 1 channel x72(76) 76 to 1 BUFFER SCA MANAGER SLOW CONTROL Serial Interface W / R Mode CK ADC TEST In Test 120fC<Cf<600fC Power SupplyReference VoltageReference Current Asic Spy Mode CSA;CR;SCAin (N°1) Power On Reset Design features 72 channels x 511 analog memory cells; F write : 1-50 MHz; F read : 20 MHz 4 Charge Ranges (120 fC; 240 fC; 360 fC & 600 fC) Supports positive or negative input signals 16 Peaking Time Values (100 ns to 2 µs) Asic For Tpc Electronic Read-out

Saclay 2-3 December The AFTER chip Design facts 0.35 µm CMOS transistors 7.8 x 7.4 mm die – 160-pin 0.65 mm pitch LQFP single 3.3V supply - 8 mW/channel Performance Integral non-linearity < 1.2% full range; ENC: 350 electrons (no load) Stored charge degradation after 2 ms retention: 0.18 LSB → All specifications met on first silicon! AFTER chip used to readout TPC Micromegas and FGD Silicon PM

Saclay 2-3 December AFTER chip test and production Production of the AFTER chip 5300 chip produced; 4750 OK; Yield: 89% 1800 chips delivered to T2K (TPC + FGD + monitoring chamber) LABVIEW Test Software Protection 1 Protection 2No Protection AFTER Test Card Test bench AFTER test card served as pre-prototype of Front-End Card Xilinx Virtex 2 Pro eval. kit as pre-prototype of FEM board for readout DAQ and Analysis with Ethernet PC and LabView

Saclay 2-3 December Front-End Card (FEC) Features All design concepts validated on AFTER test board 1 FEC reads out 288 channels – 6 layer PCB Consumption: 1 A – 4 W Quad-channel 12-bit ADC AD9229 Passive protection Circuits 4 AFTER chips 3.3V regulators Silicon ID chip V - I –T monitor +4V input Four dual row 80-pin 1.27 mm pitch connectors to detector Pulser Clock fanout PhotoMos relay 80-pin connector to FEM

Saclay 2-3 December Main functions Drive 6 FECs and aggregate data produced (1728 channels, 5.7 Gbps) Buffer one event (raw data), i.e. ~10 Mbit Deliver data to DCC upon request: raw data or zero-suppressed (one programmable threshold per channel) Configuration and slow control, voltage, current, temperature monitoring Front-end Mezzanine (FEM) card ZBT SRAM 2 Gbps optical transceiver CANbus slow control (RJ45) FPGA µC FEC#1 Voltage regulators JTAG Power In Connector FEC#0 BuffersFEC#2 FEC#3FEC#4FEC#5

Saclay 2-3 December Detector Module Read-out Electronics 72-channel ASIC Quad-channel ADC digital Front-end Mezzanine card (FEM) Optical Transceiver FPGA 80-pin connector 288-channel analog Front-End Card (FEC) 1728-pad detector plane Slow-control Network - CANbus Fiber to DCC Low voltage power Materializing the concept 3 year development of ~6 equivalent persons from ASIC specifications to working readout of one detector module

Saclay 2-3 December Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

Saclay 2-3 December Integration of TPC#0 at Triumf Feb. ‘09

Saclay 2-3 December Integration of TPC#0 and #1 at Tokai Photos: Claudio Giganti

Saclay 2-3 December Latest news from T2K nd280 m TPC #2 Tested at Triumf and now in the clean room at Tokai Installation in pit on Dec’ 17th → Fully commissioned TPCs by mid-Jan’10 for closing magnet TPC #0 and TPC #1 Installed in the magnet; DAQ commissioning in progress Awaiting final gas mixture to power central cathode and detector HV → Detector commissioning starts Dec. 7th The nd280 collaboration announced the observation of the first beam neutrino candidate (in the INGRID detector) two weeks ago!

Saclay 2-3 December Plan Presentation of the T2K experiment Architecture of TPC readout electronics Principal components Current status Experience return

Saclay 2-3 December Contribution of Irfu to T2K nd280m Electronics ASIC design, production test, readout electronic cards (design and production supervision) Physics Simulation and experimental data analysis of MicroMegas detectors Detector engineering and integration Design of Micromegas detectors, production coordination, QA Cooling mechanics and structure, cabling, services (HV, LV, fibers…) Firmware and software Embedded firmware and software in the front-end, slow control, DAQ for detector test bench, firmware and software for DCCs, LV control, etc. Project management M. Zito co-leader of TPC project with D. Karlen (Canada), A. Delbart, D. Calvet coordinator of “Micromegas modules” and “Readout Electronics” → Project relies on the diversity of competence at Irfu, teams above critical size and adequate funding

Saclay 2-3 December Project Difficulties Production First batch of wafers of the After chip did not pass QA → 2 months delay to get After chips First batch of FECs failed: humidity stored in After chip package during test requires extended baking procedure → more than 2 months to understand the problem and find workaround Defects on 10% FEM for first batches (broken via below the BGA) no consensus reached between designer, PCB manufacturer, and assembly company on exact cause (coating on vias?). → three month delay: defect analysis, modifications to PCB, new batch Could not solder some connectors on FEMs using lead free process Front-end production took 1 year while 6 months had been planned

Saclay 2-3 December Project Difficulties (con’t) Back-end electronics DCC prototype board did not become functional Unanticipated contribution of Irfu to back-end HW/FW/SW and DAQ. → Back-up solution based on customized commercial evaluation boards deployed by common effort of Triumf/Irfu/Lpnhe. Commercial Xilinx Virtex 4 ML405 Evaluation board Custom Clock board Custom Optical Transceiver board FGD DCC Crate

Saclay 2-3 December What really helped Commercial FPGA evaluation boards Inexpensive, readily available, customizable hardware Can develop and debug firmware/software on proven hardware –Firmware/software development before a dedicated board is available –Simplifies dedicated board debugging because firmware/software already tested and only migration issues need to be solved Common platform for collaborative development among various group → ~45 Xilinx ML405 boards purchased by the TPC-FGD groups! Used at Saclay, Paris, Rennes, Geneva, Barcelona, Hamburg, Tokai, Vancouver $ Part Number: HW-V4-ML405-UNI-G

Saclay 2-3 December Summary What? A channel readout system for the 3 TPCs of the T2K experiment How? A new 72-channel ASICs based on a 511 time-bucket SCA Custom made readout cards at the back of each 72 Micromegas detectors Extensive use of commercial FPGA evaluation boards Principal merit? High density, low power: ~16 mW/channel (2 kW for ch.); ~3 €/ch. Summary and Status 5 year development effort from concept to installation on site → TPC commissioning at J-PARC starting in the next few days