AMC13XG (XG = Ten Gigabit) E. Hazen, D. Gastler, A. Heister, J. Rohlf, D. Zou Boston University Quad SFP+ Optical cage AVR32 uC MMC T3 connector (JTAG, I2C) Spartan-6 FPGA Kintex-7 FPGA With heatsink DDR3 SDRAM Tongue 2 PCB Clocks T3 connector board removed to show internal detail Tongue 1 PCB GbE, Fabric A 2014-02-04 E. Hazen -- HCAL

2014-02-04 E. Hazen -- HCAL

AMC13XG Status Design Status AMC13XG (10Gb upgrade) prototyped Tested extensively including “full crate” (10 uHTR) Issue with clock discovered; fixed with small design change to T2 board only (details next) Updated link to CDAQ commissioned and tested Currently operating at 5Gb/s but could go at 10Gb Production Status Production run including 30 for HCAL started. Some delays due to (1) funding and (2) parts details later in this talk 2014-02-04 E. Hazen -- HCAL

“Almost Full” Crate Test AMC13XG 4x mCTR2d 6x uHTR 5Gb/s backplane link 100kHz trigger rate Fake uHTR data Ran for 12 hours Saving every 220 evts Results: ~3*1014 bits transferred Readout prescaled by 220, saved events checked (all consistent) CRC, BcN, OrN, EvN checked on every event by AMC13 at 100kHz Small number of corrected errors reported, but may be initialization issue Will repeat test with 12 uHTR when available 2014-02-04 E. Hazen -- HCAL

Serial Link Tests (as presented at TWEPP) These tests establish the reliability of the high-speed serial interconnections. In each plot, the horizontal axis represents the time of sampling of the serial data, with 0 representing the ideal time (middle of bit period). The vertical axis represents the threshold voltage used to characterize a bit as '1' or '0'. Backplane Loop-Back @ 5.0 Gb/s AMC Slot 1 (farthest from AMC13) Fiber Optic Loop-Back @ 10.0Gb/s (30m fiber) -0.5 0 0.5 Time Offset [UI] -0.5 0 0.5 Time Offset [UI] Links run with PRBS pattern for ~ 1016 bits with zero errors seen 2014-02-04 E. Hazen -- HCAL

TTC clock update T2 board design updated to add skew adjust on TTC clocks to backplane (based on results from “full-crate” test). Original T2 design Updated T2 design Additional driver added Feedback taken from each fanout device to match clock/TTC phase Delay in M-LVDS drivers not compensated (several ns) Spartan-6 upgraded to XC6SLX45T for extra clock resources 2014-02-04 E. Hazen -- HCAL

Firmware Major update to AMC13 firmware pending Support 3 simultaneous event builders Support arbitrarily large event fragments This is required to support other users, but we plan to use same firmware for HCAL 2014-02-04 E. Hazen -- HCAL

Event Builder Update (I) 2014-02-04 E. Hazen -- HCAL

Event Builder Update (II) Up to (3) 10Gb/s outputs per crate 1-FED mode Single event builder for 12 AMC slots, 1 link out 2-FED mode Two event builders for 6 slots each, 2 links out 3-FED mode Three event builders for 4 slots each, 3 links out 2014-02-04 E. Hazen -- HCAL

FPGA Resources Kintex-7 XC7K325T-2FFG900 utilization: FF 9% LUT 21% memory LUT 8% BRAM 56% BUFG 31% MMCM 20% PLL 30% Previous estimate 60% this is correct value with all currently-known features implemented. (LUT memory and DDR3 SDRAM are both available if BRAM use exceeds 100%) This includes all the mentioned firmware upgrades plus 3 DAQ link outputs. The Spartan-6 FPGA utilization was below 20% on all resources, and has been upgraded to a larger part for the clock fix, so it is not an issue. 2014-02-04 E. Hazen -- HCAL

Funding etc Orders for 80+ AMC13 received (roughly 50:50 US:Euro) Funding sorted out (draft SOW in hand) for production of ~ 50 modules plus required engineering Remainder of modules to be produced through CERN (possibly with US supplier) 2014-02-04 E. Hazen -- HCAL

Production Status New T2 design will be prototyped (qty 2 assembled) and tested before committing to bulk assembly (all PCBs received) Purchasing begun for a large run of AMC13XG (65 pcs, some for G-2). Currently several long lead time items, so we are looking at first boards received end April :( We have cast a wide net to try to track down the parts in stock through overseas distribution Difficult parts: Kintex-7 FPGAs currently due 3/28 DDR3 Memory currently due 3/1 Various samtec currently due 2/14 Harting AMC conn. currently due 2/14 Xilinx are the most problematic. 2014-02-04 E. Hazen -- HCAL

Software Re-write of AMC13 software beginning Change from flat address table to hierarchy Remove extra class layers so that base class for AMC13 calls uHAL directly Re-write command line tools Assist in development of generic GUI which can support AMC13 using new address table scheme Re-write AMC13 XDAQ support to be more generic (usable by other subdetectors) 2014-02-04 E. Hazen -- HCAL

Summary Design complete Firmware updates underway Funding secured Production delayed due to parts issues Not (yet) a problem for HCAL? 2014-02-04 E. Hazen -- HCAL

Backup 2014-02-04 E. Hazen -- HCAL

2014-02-04 E. Hazen -- HCAL

Optional AMC13 T3 Board Cu input for clock, control signals Used for TCDS (TTC/TTS system upgrade) Courtesy Magnus Hansen 2014-02-04 E. Hazen -- HCAL

2014-02-04 E. Hazen -- HCAL Preprint:JINST_101P_1113 [TWEPP-13] Authors :E. Hazen, A. Heister, C. Hill, J. Rohlf, S.X. Wu, D. Zou Title :The AMC13XG: A New Generation Clock/Timing/DAQ Module for CMS MicroTCA JINST_101P_1113 has been published as: 2013_JINST_8_C12036 Available at : http://dx.doi.org/10.1088/1748-0221/8/12/C12036 2014-02-04 E. Hazen -- HCAL

New Data Format (proposed) 2014-02-04 E. Hazen -- HCAL