Upgrade to the Read-Out Driver for ATLAS Silicon Detectors Atlas Wisconsin/LBNL Group John Joseph March 21 st 2007 ATLAS Pixel B-Layer Upgrade Workshop

2 Silicon Read Out Driver (SiROD)

3 256 MB SDRAM VME Slave FPGA Configuration Controller ROD BUSY Histogram 128Mb Flash CNFG & BOOT Memory ROD CONTROLLER FPGA XC4VFX20-10 Power PC Core FORMATTER FPGA XC3S x EFB /ROUTER FPGA XC4VLX60-10 DSP FARM TI 320C MHz 384MB SDRAM 4x DIAGNOSTIC FIFOs VME Bus Back Of Crate (BOC) CNFG DATA to All FPGAs RESET ROD BUSY D32 A32/D32 ROD Control/VME Path D16 ADDR D8 D GB SDRAM ADDR ARB MOVE HPI INTERFACE TO VME CONTROLLER BOOT SLAVE DSP FROM FLASH

4 ROD CONTROLLER FPGA XC4VFX20 FORMATTER FPGA EFB/ROUTER FPGA XC4VLX60 DSP FARM TI 320C MHz 384MB SDRAM 4x Xoff BOC FORMATTER FPGA INPUT FIFO 32K Deep S-Link Event Fragments ROD BUSY TTC EVENT ID READOUT TIM MDSP SP BOC FE CMD (48) XC3S4000 SiROD DATA PATH Xoff DATA STATUS

5 COMPONENT COMPARISONS: FORMATTER RevF SiROD Spartan2E Series: XC2S600-6FG456 Cost: $100 x 8 = $800 Available resources: 6912 Slices, 288K RAM Bits per FPGA Used resourses (1) : 2850 Slices (41%), 268K RAM Bits (93%) Used resourses (8) : 22.8K Slices (41%), 2140K RAM Bits (93%) Module FIFOs: 32 – 2048 x 32 Dual Port RAM Blocks Next Generation SiROD Spartan3 Series: XC3S4000-8FG900 Cost: $220 x 2 = $440 Available resources: 54K Slices, 3456K RAM Bits per FPGA Required resources (2) : 23K Slices, 1072K RAM Bits Module FIFOs: x 32 Dual Port RAM Blocks

6 COMPONENT COMPARISONS: EFB/ROUTER RevF SiROD Spartan2E Series: XC2S400-6FG676/XC2S400-6FG456 Cypress K x18 15ns FIFOs Cost: $115 + $85 + $210 = $410 Available resources: 9600 Slices, 1856K RAM + FIFO Bits Used resourses : 6080 Slices (63%), 1824K RAM + FIFO Bits Trap FIFOs: x 32 Dual Port RAM Blocks Next Generation SiROD Virtex4 Series: XC4LX60-10FG1152 Cost: $558 Available resources: 26K Slices, 2880K RAM Bits Required resources: 6080 Slices, 1824K RAM Bits Trap FIFOs: x 32 Dual Port RAM Blocks

7 COMPONENT COMPARISONS: CONTROLLER RevF SiROD Spartan2E Series: XC2S600-6FG676 / TI320C DSP Cost: $130 + $100 = $230 Available resources: 6912 Slices, 288K RAM + FIFO Bits Used resourses : 5130 Slices (74%), 188K RAM Bits (65%) DSP Processor Next Generation SiROD Virtex4 Series: XC4FX20-10FG Cost: $228 Available resources: 8544 Slices, 1224K RAM Bits, Power PC Core Required resources: 5130 Slices, 188K RAM Bits, uProcessor Conversion from TI DSP to Xilinx Power PC core: Risk Evaluation Required

8 Silicon RevE(F) SiROD Master DSP Controller FPGA Formatter FPGAs Router FPGA Event Fragment Builder FPGA Slave DSPs 9U VME VME Interface Clock Distribution

9 ROD CONTROLLER FPGA XC4VFX20 128MB SDRAM EFB/ROUTER FPGA XC4VLX60 DSP FARM TI 320C MHz 384MB SDRAM VME J1 J2 INPUT FIFO 32K Deep FORMATTER FPGA XC3S4000 SiROD New Layout BOC J3 FORMATTER FPGA XC3S4000 VME CONTROLLER FPGA XC3S400 DSP FARM TI 320C MHz 384MB SDRAM 128Mb FLASH JTAG CONNECTOR ON FRONT PANEL TO PROGRAM VME CONTROLLER FPGA ROM

10 WHERE DO WE GO FROM HERE? Preliminary Design Study Recommendations: Purchase 2 Xilinx Virtex4 Evaluation Boards One for the LX series One for the FX series Combine the EFB/EventMemory/Router Designs to simulate and implement the proposed changes for verification Simulate and implement the Formatter as proposed for verification Simulate and implement the Controller to learn how to use the Power PC core and to plan a transition from the TI 6201 to the Xilinx PC core. Make the new design transparent to the DAQ system. Simulate the Control path to understand the limitations of VME BW

11 Additional Slides

12 RODBus Interface Control & Status Registers (32) RODBus Data (16) RODBus Address RODBus Control Link Input MUX & Half Clock Counters IN0 ModeBits FIFO 512x24 Link FIFO 2048 x 32 Link Decoder: 40/80/160MHz Read Out Controller DATA OUT (40) TOKEN FMT MB (12) Link FIFO 2048 x 32 Link Decoder: 40/80MHz Link FIFO 2048 x 32 Link Decoder: 40MHz Link FIFO 2048 x 32 Link Decoder: 40MHz IN1 IN2 IN3 IN4 IN5 IN6 IN7 IN8 IN9 IN10 IN11 FMT Type FMT ID HT LIMIT ROD BUSY Trailer Detect TRAILER (12) PIXEL FORMATTER FPGA (8x)

13 EVENT FRAGMENT BUILDER FPGA EventMemA FIFO 16Kx48 Engine 0 Engine 1 EventMemC To/From Router To/From Formatter To/From Controller Output FIFO Data (43) Header and Trailer Data (32) Halt Output Header/Trailer Generator Error Summary Word Format & Count L1/BC ID Check Error Check INC/DEC L1ID L1 & BC IDs Event Data & Trigger Type Dynamic Mask Link Num. (6 bits) Time-out Error L1ID Error BCID Error Data (32 bits) Formatter Num. (2 bits) Link Num. (4 bits) Data (32 bits) Data Valid Time Out INC/DEC L1ID Purpose: Collect Formatter output, check L1 and BC IDs, count errors, generate Event Header & Trailer FIFO Controller Xoff EventMemB FIFO 16Kx48

14 ROD Router (FPGA) & Slave DSPs SDSP 3 SDSP 2 SDSP 1 SDSP 0 Read, format and direct the event data 2 Traps Event Type: ATLAS, ROD or TIM 2 Traps Event Type: ATLAS, ROD or TIM 2 Traps Event Type: ATLAS, ROD or TIM 2 Traps Event Type: ATLAS, ROD or TIM DMA Transfer Engine bit FIFO DMA Transfer Engine S-LINK To/From ROS To/From Event Fragment Builder XOn / XOff S-Link S-Link Data Dsp Halt Output Halt Output Error Format Data Data Valid Output FIFO (43b) Event Header and Trailer Data (32b) Texas Instruments 6713 floating point DSPs running at 220 MHz for monitoring and calibration histogramming Purpose: Route formatted data to Level-2 and/or Slave DSPs (histogram) bit FIFO