1. UCHOLA Firmware and Tests UCHOLA Design Team Anton Kapliy Mel Shochet Fukun Tang Lauren Tompkins Daping Weng Enrico Fermi Institute University of Chicago ANL, Nov. 22 2011 FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 1

2. Bird’s-eye view: original HOLA FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 2 FIFO Interface to TLK D[32] @ 40 MHz D[32] @ 50 MHz FPGA D[16] @ 100 MHz TX_EN TX_DV Mezzanine data port TLK2501 serializer TLK2501 deserializer FPGA provides a parallel interface to an outside SERDES device (TLK-2501), which feeds serial signal to an optical transmitter. However, TLK-2501: Is getting deprecated and not marketed by TI anymore Rather expensive and hard to find among the distributors Consumes a lot of power Therefore, we chose to move the transceivers inside the FPGA Budget Cyclone IV (~$40) fits the bill Serial connection via optical transceiver (for simplicity, only the forward channel is shown)

3. Bird’s-eye view: dual-output HOLA FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 3 FIFO Interface to TLK D[32] @ 40 MHz D[32] @ 50 MHz FPGA D[16] @ 100 MHz TX_EN TX_DV Mezzanine data port TLK2501 wrapper TLK2501 deserializer >90% of original HOLA code is unchanged. TLK2501 wrapper emulates TLK2501 functionality: “Plugs in” into the original HOLA core code Implements link startup and synchronization TLK receiver on DAQ side is oblivious to the change Firmware created with Altera Quartus 10.1 SP1 (for simplicity, only the forward channel is shown) Old HOLA core Xilinx deserializer Serial connection via optical transceiver ROBINFTK_IM

4. Prototype Card Tests at Chicago: setup (1) FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 4 PC (Gentoo Linux) Performs: Reset Control Verification S32PCI64 “SOLAR” (FEMB) “FILAR” (LDC + ROMB) Dual-output HOLA (LSC) FTK channel DAQ channel Static optical attenuators (850 nm multimode) Two pairs of optical fibers

5. Prototype Card Tests at Chicago: setup (2) FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 5 PCI FILAR Card PCI SOLAR Card Dual HOLA Card Four 7 dB attenuators Configuration & JTAG Ports PWR TST ERR UP0 UP1 XF0 XF1 ACT PWR TST ERR UP0 UP1 XF0 XF1 ACT DAQ UCHOLA Front Panel

6. Prototype Card Tests at Chicago: setup (3) FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 6

7. Prototype Card Tests at Chicago: results FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 7 Power Consumption: 0.62A@3.3V (2 Watts Total) with 2 optical transceivers.0.62A@3.3V Stress Tests: using “SLIDAS” test mode. Pseudo-random patterns are generated inside SOLAR and sent through the DAQ and FTK fibers, which are then read out by the FILAR. SOLAR tries to send at ~150 MB/s PC can readout & verify only at 65 MB/s Flow control nearly always asserted BER= 0.7x10 -16 with -7dB (21 days) BER= 1x10 -12 with -13dB (30 minutes) Additional Tests: Generating data on the PC and passing it to the HOLA through the SOLAR (slow!) SLIDAS test mode with smaller or larger S-Link frame fragments ROD mode: pseudo-random data with ROD-like headers/trailers Setting DAQ return lines on the FILAR and reading them out on the SOLAR 32-bit data port2-Ch Optical Transceivers Top Side View Bottom Side View JTAG PORT FPGA Conf. Port FPGA Drop-in replacement for default HOLA: If FTK fiber is disconnected, the HOLA automatically operates in DAQ-only mode

8. Prototype Card Tests at CERN: setup FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 8 Duo-HOLA ROD ROBIN I ROBIN 2 ROS (PC) DAQ link EDRO board FTK_IM FTK link FTK_IM replicates the data it receives from HOLA and sends it to a 2 nd ROBIN. A PC reads out and validates data from both ROBINs. We control the speed of each readout to selectively exercise XOFF Attenuator XOFF (DAQ) XOFF (FTK) Pattern generator or real data XOFF Pixel and SCT testbenches in SR1

9. Prototype Card Tests at CERN: results FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 9 Discovered that we really need a front panel for the HOLA ◦ Otherwise, there is a danger of mechanical shorts with the BOC card Pixel testbench: ◦ Event rate: 10-20 KHz for >24 hours ◦ FTK can hold the data (XOFF) for an arbitrarily long period of time without causing a timeout  This is probably just a feature of this particular SR1 testbench. ◦ Once FTK XOFF is removed, data flow continues SCT testbench: ◦ Event rate: ~100 Hz (trigger problems prevented us from running faster) ◦ FTK can hold data for up to 10 seconds at 100Hz (shorter at higher rates) ◦ Beyond that, ROS times out, and the corresponding ROD is automatically taken out of data taking. Note that this timeout is configurable. ◦ Need to be careful during system integration! To prevent spurious XOFF during FTK fiber plugging/unplugging, we added a HOLA FTK_XOFF_ENA register that can be programmed through FTK LRL. ◦ When this register is off, the FTK channel is completely passive (no effect on ATLAS) ◦ We plan to have this register off on power-up in the default firmware

10. Production Test List FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 10 PWR TST ERR UP0 UP1 XF0 XF1 ACT PWR TST ERR UP0 UP1 XF0 XF1 ACT DAQ UCHOLA Front Panel 1.Assign a serial number to the card 2.Inspect component soldering 3.Check short/open of 6 DC powers 4.Load Faraday cages and optical transceivers 5.Install front panel 6.Check DC current before FPGA configuration 7.Load FPGA firmware 8.Check DC current after FPGA configuration 9.Mount UCHOLA to SOLAR board 10.Insert all the optical fibers 11.Start test program 12.Fill Elog database with test results 13.Unload UCHOLA and package it for shipping Total production cards: 260 Test time needed for each card (15 minutes): Installation and FPGA configuration: 7 minutes Dynamic Test (return lines + BER 10 -11 ) : 8 minutes Total required test time: 65 Hours

11. Elog Database of Production Tests FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 11 The test results including DC Power and dynamic test status for every card will be recorded in a UCHOLA Elog. Test engineers can login in and write the notes. The Elog will be very useful for tracking the status in the future. Thanks to Mary Heintzhola.uchicago.edu Username: einstein Password: jjff88

12. Installation plans FTK Engineering Meeting at ANL A.Kapliy 11/22/2011 12 First batch of HOLAs will be installed during the winter 2011-2012 shutdown Bjoern prepared a list of HOLAs necessary for the vertical slice. Depending on the # of 45° regions, we’ll need to install 16 to 34 HOLAs All HOLAs will be tested and entered to the eLog by mid-January 2012. Installation planned for late January / February 2012 Jinlong will organize these activities at CERN. I will also fly to CERN for a few weeks. 2-region option: 4-region option: