A proposal for a full upgrade of the L1CTT/DFE for Run2b Meenakshi Narain Boston University Outline: u Why do we need a full upgrade? u The new proposed design..

Groups  Simulation and Algorithm development: u Kansas: s Graham Wilson, Carsten Hensel u Manchester: s (Liang Han), Terry Wyatt u Notre Dame: s Mike Hildredth  Hardware: u Boston University: s Meenakshi Narain, Eric Hazen, Ulrich Heintz, Shouxiang Wu u FNAL: s Marvin Johnson, Stefan Gruenendahl, Jamieson Olsen

L1CTT Upgrade Motivation  In Run IIa and RunIIb, the L1 Track Trigger Provides: u CFT tracks for L1Muon Seeds u High p T isolated track trigger capability u CFT tracks +CPS clusters  embryonic electrons u Found tracks for STT  b-tagging  We believe the Run IIa L1CTT will fail at the occupancies we expect in Run IIb, resulting in u Many more fake high-p T tracks for all systems u The loss of rejection from combining subsystems

Physics Motivations  Must trigger on high-p T leptons with high efficiency, low fake rate:  HW  bb e or bb , HZ  bbll u Cal-Track match is important/necessary here  Must provide clean track samples for b-tags with the STT  HZ  bb, Z  bb  Could (Must?) provide high-p T tracks for hadronic tau triggers  H  (gets you 35% equivalent lumi boost) u Cal-Track match is important/necessary here  (Correlations between Systems is Necessary)

Run IIb Luminosity Projections ~1.6e32 ~2.8e32 Accelerator draft plan: Peak luminosities Peak Luminosity (x10 30 cm -2 sec -1 ) We are here ~5.6e31

Run IIa at high luminosity  Even at modest occupancies, the high-p T single track trigger would fire on ~2% of events (>100kHZ) Isajet MC events Nominal 5E32 (1 high pT track) (1 high+1 medium pT)

L1CTT Granularity  Tracking trigger rates sensitive to occupancy  Upgrade stategy: u Narrow tracker roads by using individual fiber hits (singlets) rather than pairing adjacent fibers (doublets) u Cal-track matching Run IIb  Use full fiber resolution to restrict roads Run IIa

L1CTT Algorithm Results  With baseline version of new L1CTT: Scheme/ p T range Tracking Efficiency (%) Rate of Fake Tracks (% events) Resources ABCDEFGH (RunIIa) (p T >10 GeV)  k X 8 abcdefgh (p T >10 GeV)   k X 16 abcdefgh (5 GeV<p T <10 GeV)   k X 12 abcdEFGH (3 GeV<p T <5 GeV)   k X 12 abcdEFGH (1.5 GeV<p T <3 GeV)   k X 12

L1CTT architecture  A multistage system u Analog Front End (AFE): s Signals from the tracker u Mixer s Sort signals in trigger sector wedges u Digital Front End (DFEA): s Track Trigger logic u Octant (CTOC) s Combine track information from several DFE boards u CTTT s Construct track trigger terms u Trigger Manager s Construct 32 AND/OR terms used by the L1 Trigger Framework in forming the trigger decision

L1CTT architecture  The approved upgrade: u Replace Digital Front End daughter boards (DFEA): s Need to fit 3-5 times more equations s Rebuild these cards using larger FPGAs s Use XC2V6000 Chip with 6M sytem gates

Equation Resource study As a function of eff /number of equations: Even the worst case equation scenario all singlets in 4 pT bins – fits in two XC2V6000

DFEA daughter board  First prototype with XC2V4000 FPGAs in hand and tested

Lessons from Run2a commissioning  Current “approved plan” u upgrade of the DFEA daughter boards. u Provide a 5x increase in available logic resources.  With the experiences over the last year while commissioning the present Run2a L1CTT system a few very important issues have surfaced: u Lack of extensive testability of the input and output information from the DFEA u Excessive Firmware download times s currently about four hours (with 2 parallel jobs) –may need restarts (5 automatic, 2-3 manual) s Limited by the slow 1553 interface s Scales up to about 12 hours/job for RunIIb equation file sizes u Clock and SCL signal distribution (dependent on upstream info) u A shortage of spare backplanes (there are none)

Desired Improvements  Reduce FPGA configuration download times: u From many hours/ half a day to minutes u Concerns about reliablility of the download of firmware 3-5x the existing size. Permits and improves reliability. u realistic times for debugging and updates  Improve Testability: u Provide diagnostic access to the processor FPGAs on the DFEA DB. u Improved ability to inject test data and capture input data for in-situ testing. u Enhance post-manufacturing testing (boundary scan or other type)  Provide a means to inject SCL-provided control signals u ( RF Clock, First Crossing, etc) to an entire crate for bench-test and commissioning. This will permit stand-alone operation of the DFEA system for debugging and commissioning  Move input cable plant from front of the crates to the rear.  Provide LED status indicators on the front panel.  Improve reliability over the existing daughterboard connectors

The Proposal  In order to add testability to the system, one needs: u A new motherboard (daughterboard – to a less extent) design. u Improved diagnostics: input and output buffers, and L3 capability.  To alleviate colossal download times we need: u A new DFE crate controller with a faster connection to the D0 online computers. u A new DFE backplane design Utilize this chance to put the cables to the back. u No transition boards.  Detailed specifications of the boards are being developed – follow links through the Run2 trigger web page

DFE Backplane  Standard VME size, 21 slot, 6U monolithic backplane with custom connectors u Slot 1 is used for the DFE crate controller, DFE modules are located in slots u In slots 2-21, the backplane will support up to ten input cables and four output cables passing through the backplane.  The backplane carries a simple read/write bus for communication with the DFE modules.  Also carries clock and control bits from the SCL receiver  The DFE boards can also share data with their neighboring boards  A high-voltage (48V) power distribution scheme is under consideration. u Eliminates potentially hazardous high-current low voltage supplies u allow more power to be distributed over the backplane with smaller conductors, eliminates heavy cables, remote sensing oscillations u However the concern is here noise in the calorimeter – tests will be done before deciding on this scheme.

DFE Crate Controller  The controller must download firmware to the track-finding FPGAs on the logic boards via the backplane read/write bus. u In addition, it provides external access to the bus for configuration, monitoring and debugging.  The existing DFE controller: u flash memory for local FPGA configuration storage, and a 1553 control bus interface for external access.  The new controller will prove a direct interface to the read/write bus via a high speed (gigabit class) fiber optic link (also simpler). u The flash memory and microcontroller will be eliminated. u The firmware for the track-finding FPGAs will be downloaded directly over the fiber optic link at very high speed, allowing fast firmware changes for updates and debugging. u In addition, test patterns may be downloaded to the DFEA boards and captured output data read out over the link.  It is currently planned to use a commercial gigabit-ethernet fiber interface module to simplify the controller logic such that it may be implemented in a single FPGA.

A suggested solution based on CMS design: VMEbus Controller with Gigabit Ethernet – A custom board designed and developed at OSU – Based on XILINX Virtex-II Pro – Optical transceiver (for Gbit Ethernet) – Communicates with stand-alone PC via Ethernet – Inexpensive: ~ $600 each, tests begin soon.

DFE Mother Boards  Redesign for increased testability u Direct read/write access to daughter board FPGAs u Status LEDs on front panel u Replace or eliminate unreliable spring contact connectors u Eliminate rear transition modules

DFEA daughter board  First prototype in hand and tested  Will need to make a few changes due to the new design of the MB etc u Improved system level diagnostics u By improving ability to send test patterns u Capture input/output buffers and enhanced Level3 support u Improve reliability over the existing spring-contact PCB surface daughterboard connectors

Incremental Equipment Cost Cost k$ Backplane 14 Crate Controller 7.5 Optical download and control link 5K Test equipment + splitters + crate 17 Motherboards 61 MB Prototypes + test stand 25 Total 129 +Labor Costs

New Schedule  Aim for: u installation in Summer of 2005 u A slice test during Fall 2004 shutdown.

Installation & Commissioning  The proposed scheme is driven by the desire to enhance testability and reliability of firmware downloads. u Eases Installation and commissioning  Plan to run on a partial crate of the new system on the platform in parallel with the existing DFEA using LVDS splitters u Key to successful commissioning of the new system  Advantages: u Can assemble the whole crate outside of the collision hall u Extensive testing of the entire chain possible before putting in collision hall s Enhanced testing capability s Use Run2a Data derived test vectors to verify u Take out the old crates (2 of them) and replace them with new ones u Use the existing Run2a tools used for commissioning s Low level changes will be transparent at user level

Summary  Proposal to upgrade the entire L1CTT DFE hardware u Based on lessons learnt from Run2a L1CTT commissioning u Provide more in-situ testing capability s Add more I/O buffers and L3 readouts u Reduce the firmware download times significantly s Use high speed fiber optic links u Installation a bit simpler s extensive tests can be performed prior to installation u Commissioning times will be reduced s Faster downloads s Uses existing infrastructure

DFE Motherboards  The Digital Front End (DFE) Motherboard is a general purpose, high bandwidth platform for supporting reconfigurable logic such as FPGAs. u It is intended for applications where a DSP or other microprocessor is too slow.  The DFE motherboard is a 6U x 320mm Eurocard with fully custom hard metric backplane connectors.  Eight point-to-point links bring data onto the DFE motherboard at an aggregate data rate of about 12 Gbps. u Each physical link consists of five twisted pairs (Low Voltage Differential Signals) u These are terminated with hard metric female connectors on the DFE backplane and passed through to the DFE motherboard. u After entering the DFE motherboard, the ten links are sent to receivers, which convert the serial data back to a 28 bit wide bus running at 53 MHz. u These busses, in turn, are buffered and routed to the two daughtercards.  The outputs from the daughterboards are fed back to the motherboard, u converted to serial data by channel link transmitters and SLDB (serial link daughter boards) mounted on the motherboard. u The serial data are passed through the backplane to the output cables.  A “local bus” bridged to the backplane bus will allow direct read/write access to the individual FPGAs on the DFEAs for debug and monitoring.  Additionally, a dedicated bus for cross-communication between the two DFEA daughterboards will be provided.

Core Trigger Menu Simulations Total L1 bandwidth budget= ~3 kHz Total rate: ~15kHz ~30kHz ~3.2kHz Additional headroom available from topological cuts available in upgraded L1cal Higher muon p T threshold with upgraded CTT 1E32 2E32 2E32