1. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 1 CMS Calorimeter Trigger SLHC Regional Calorimeter Trigger System Design and Prototypes Tom Gorski University of Wisconsin July 20, 2009

2. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 2 GCT Muon Aux Card Update Pushbutton Reset (for Microblaze) S-Link Connectors TTS TTCrx RS-232 GTP Links

3. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 3 GCT Muon Aux Card Update Currently developing test firmware for the Aux Card Firmware to be completed in Fall, 2009 Test firmware is a hybrid microblaze/HDL implementation User interface: C program via RS-232 to PC terminal Dedicated HDL blocks to test GTP, TTC, TTS & S-Link Interfaces

4. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 4 SLHC Cal. Trig. Demonstrator SLHC Cal. Trig. Demonstrator Prototype Design First Step: determine requirements to provide proof of principle of major elements of SLHC Cal. Trig. System Evaluate SLHC Cal. Trig. Conceptual Design Needs performance to incorporate algorithms Shown by M. Bachtis, S. Dasu, K. Flood (UW) in this workshop Compatible with firmware developed for these algos. K. Compton, M. Schulte, et al., (UW) Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2009. Must work with ECAL 1, HCAL 2 upgrade designs & eventually in combination with upgraded tracking trigger primitives Ph. Busson 1 (LLR), J. Mans 2 (UMinn) Integrated with SLHC Optical SLB J.C. da Silva, LIP. Clear evolutionary path from present LHC RCT Backwards-compatible functionality maintained.

5. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 5 RCT Calorimeter Trigger Evolution RCT GCT: Sources GCT: Main GT/GMT GCT/ uTCA ETCC: TPGs HTR: TPGs Cu FO RCT GCT: Sources GT/GMT GCT/ uTCA uTCA- HTR: TPG oSLB RCT/ uTCA GCT/ uTCA GT/GMT ETCC: TPGs uTCA-HTR: TPGs oSLB Step 1 (2009) Step 2: ↓ OR ↓ RCT/ uTCA ETCC: TPGs uTCA-HTR: TPGs oSLB Step 3Step 4 oSLB GCT: Sources GT/GMT GCT/ uTCA RMC SLB ETCC: TPGs SLB Matrix & Aux Cards  oSLB HTR: TPGs oSLB

6. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 6 RCT Receiver Card Review: Current HCAL/ECAL to RCT Link HCAL HTR or ECAL TCC Vitesse V7216 Tx RCT RCVR Mezz Vitesse V7216 Rx RCT Phase ASIC 4X 1.2 Gbps Copper Links (19 bits data + 5 bits Hamming per link per crossing) Intersection of 4,032 links at common destination (RCT) Link Xmt & Rcv Clks are 3X the LHC Clock (~120 MHz)—no long term drift issues Elastic buffers in V7216 Rx chips to manage short term clk jitter RCT Phase ASIC provides channel bonding function, hamming code check, and buffering for 4X RCT processing pipeline (~160 MHz) in about 3 crossings 120 MHz Link Clock skew tolerances: ±6ns @ Tx, ±1ns @ Rx Scheme is stable, reliable, and has low latency 4.8 Gbps aggregate

7. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 7 Clock Frequencies and Data Rates Calorimeter data produced by a 40.08 MHz synchronous process Carried by high-speed serial links between processing stages FPGAs support different clock domains for the high speed serial links and the programmable fabric, but at the cost of increased latency at the interface Governing parameter is the Link Parallel I/O Clock Frequency Key Decision: Run FPGA processing fabric and/or serial links synchronous to the LHC clock or on a decoupled timebase? Decoupling frees designers to chose frequencies that maximize link data rates Price of decoupling: Increased latency on link/fabric interface Increased complexity in pipeline control and link protocol (empty cycles) Advantages of coupling: Perfect match in bandwidth between calorimeter, links, and processing pipeline Lowest latency on interfaces Simplified pipeline control and link protocol Existing Latency Constraints + Xilinx Rocket I/O Tile design exerts a strong influence towards the coupled approach

8. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 8 TPG Decompression Functions RCT Currently uses full LUTs for each H/E tower pair (2 17 × 18 bits) Implements arbitrary conversion function plus H/E cut for electrons LUTs require significant board space, not practical for upgrade— need to go to functions instead Functions can place significant demands on special FPGA resources (e.g., DSP slices, block RAM) Demand depends on function type: Mantissa/Exponent least costly—simple shift Piecewise linear uses Add/Multiply per HCAL or ECAL tower Logarithmic/Exponential about 4-8× more costly than piecewise linear Will need to settle on general scheme early on in the process in collaboration with ECAL and HCAL groups Pursued design approach will have parameters stored in FPGA RAM, and allow changes to them without requiring FPGA resynthesis

9. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 9 oSLB RCT side (LIP Development) FPGA Optical Rcvr Optical Xmtr Incoming TPG (4.8 Gbps from oSLB TPG side) Repeated TPG to SLHC RCT 120.24 MHz Clock 120 MHz Synchronous Parallel Data to Phase ASIC Latency for Current 7216-based Link: 3.4 bunch crossings for cable (85ns) 0.8 bunch crossings for 7216 Tx (19ns) 2.5 bunch crossings for 7216 Rx (62ns) TOTAL: 6.6 bunch crossings (166ns) Based on what we know so far about Rocket I/O, the budget may need to be increased by 1 or 2 crossings (from J. C. De Silva)

10. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 10 SLHC RCT Block Diagram (56η × 12φ slice) TTC/DAQ Card Processing Card (Backplane) Processing Card Processing Card Inter-crate Φ-sharing Links Inter-crate Corner- sharing Links HCAL/ECAL TPGs from oSLB Cards Input Card Trigger Partial- Products η-sharing Links Output Links to GCT Clock and Control TTC/DAQ Connections Trigger Data to DAQ

11. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 11 SLHC RCT Crate (1 of 6) Input Card TTC/DAQ Card Processing Card Input Card Processing Card Input Card Processing Card Input Card Output Links to GCT HCAL/ECAL TPGs from oSLB Cards Φ-sharing Links to Input Cards In other Crates Corner-sharing Links to Input Cards in other Crates Crate Output to DAQ Clock/Control from TTC (uTCA form factor) Hub Controller

12. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 12 Input Card 2 Input Card Coverage/Sharing Input Card 3 56 towers in η 72 towers in φ Input Card 2 Input Card 1 Input Card 0 Input Card 4 Input Card 5 Input Card 6 12 towers/crate Input Card 3 Input Card 2 Input Card 1 Input Card 0 Input Card 4 Input Card 5 Input Card 6 Crate N Input Card 3 Input Card 2 Input Card 1 Input Card 0 Input Card 4 Input Card 5 Input Card 6 Crate N+1 Crate N-1 8 towers/card

13. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 13 SLHC RCT Input Card (12η × 8φ) Link Switch Microcontroller (Mezzanine) FPGA HF TPG (1) Ethernet Clock Circuitry φ-sharing (4) Frontpanel (Serial Links) Backplane Fast Buffer SRAM ECAL TPG (12) Corner-sharing (4) η-sharing (6) To Proc. Card (~12) To DAQ Card (1-2) Clock and Ctrl HCAL TPG (12)

14. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 14 (~20η × 12φ) SLHC RCT Processing Card (~20η × 12φ) Link Switch Microcontroller (Mezzanine) FPGA Ethernet Clock Circuitry Fast Buffer SRAM To DAQ Card (1-2) Output to GCT Frontpanel (Serial Links) Backplane Clock and Ctrl From Input Cards (~30)

15. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 15 SLHC RCT “TTC”/DAQ Card (Evolution of UW GCT Aux Card) DAQ Xmt Interface Microcontroller (Mezzanine) FPGA Ethernet TTC Interface Clock and Control To Input and Proc. Cards To DAQ FrontpanelBackplane DAQ Data from Input and Proc. Cards (10-20) Clock/Control From TTC To TTS

16. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 16 SLHC RCT Key Points: Double-width AMC-style (uTCA) modules (148.8mm height, 181.5mm deep) 3 Card Types: Input Card, Processing Card, TTC/DAQ Card (evolution of UW GCT aux card, possibly common card) Input Card receives HCAL/ECAL TPGs, performs inter-region data sharing needed by algorithms (7 cards/crate) Processing card receives partial products from Input Cards, completes regional processing, delivers output to GCT (~3 cards/crate) TTC/DAQ interfaces crate to the TTC and DAQ systems (1/crate) Single Crate Encompasses Full 56-tower η Width Card microcontroller implemented as a Mezzanine Can perform uTCA arbitration if needed High-performance I/O interface to card for Trigger Supervision functions Single hardware/firmware implementation Backplane contains combination of passive and switched interconnections Passive good choice for η-sharing Switches for some routing between Input and Processing cards TPG Inputs are fundamentally compatible with the envisioned upgrade path Affects link and pipeline clock frequency choices Goal is to have a single firmware image for each card type, and to configure individual cards via RAM

17. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 17 SLHC Cal. Trig Prototype (R&D Platform for Conceptual Design) Link Switch Microcontroller (Mezzanine) FPGA (e.g., Xilinx XC5VTX240T) Ethernet Link Clock Conditioning Circuitry Fast Buffer SRAM GTX Links (6) GTX Links (12) Clock and Ctrl SNAP 12 Transceiver (6X/6R) GTX Links (6)

18. Tom Gorski, U. Wisconsin, July 20, 2009 SHLC RCT - 18 SLHC Cal. Trig Prototype R&D Hardware Platform Prove layout/PCB fabrication concepts and principal technologies for Input and Processing cards Circuit Prototypes: Microcontroller Mezzanine Concept Link/Pipeline Clock Conditioning External SRAM/FPGA Interface Double-size AMC module (149mm x 182mm) Suitable for trigger algorithm development Capable of supporting hardware/firmware/system R&D for multiple years