1. PCI BASED READ-OUT RECEIVER CARD IN THE ALICE DAQ SYSTEM
F. Carena1, W. Carena1, P. Csató2, E. Dénes1, T. Kiss2, J. C. Marin1, R. Divià1, K. Schossmaier1, C. Soós1, J. Sulyán2, A. Vascotto1, P. Vande Vyvre1
1CERN/EP-AID (Geneva)
2 KFKI-RMKI (Budapest)
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

2. ALICE DAQ SYSTEM Detector Detector Detector Detector DDL DDL DDL DDL
Front-End
Electronics
(FEE)
Front-End Electronics (FEE)
Source Interface Unit (SIU)
Read-Out
Receiver
Card
(RORC)
Destination Interface Unit (DIU)
PCI-based Read-out Receiver Card (PRORC)
Detector
Detector
Detector
Detector
DDL
DDL
DDL
DDL
event fragments
Local Data Collector
DDL
DDL
LDC
DDL
LDC
DDL
PRORC
PRORC
PRORC
PRORC
Multimode optical cable
Detector Data Link (DDL)
sub-events
DAQ Network
Event builder
Event builder
full events
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

3. PCI-based Readout Receiver Card - LECC 2002, Colmar
FEATURES
Interface between the DIU and PCI local bus
32 bit/33 MHz PCI version, max. throughput 132 MB/s
PCI master capability, data push architecture
Autonomous operation with little software assistance
Supports multi-paged memory management
Direct data transfer to the PC memory
No local memory on the board
Small elasticity buffers between different clock domains
Built-in test capability
Internal pattern generator can produce formatted data
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

4. PCI-based Readout Receiver Card - LECC 2002, Colmar
HARDWARE
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

5. PCI to local bus FIFO (8 x 32 bit) Local bus to PCI FIFO (8 x 32 bit)
PCI INTERFACE
Incoming Mailbox
PCI to local bus FIFO (8 x 32 bit)
PCI bus interface (32 bit/33 MHz)
Local bus interface
Local bus
PCI bus
Local bus to PCI FIFO (8 x 32 bit)
Outgoing Mailbox
AMCC S5935
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

6. Memory manager and command interpreter
FIRMWARE
PRORC firmware
Transmitter FIFO
Read DMA controller
Pattern Generator
Memory manager and command interpreter
DDL
AMCC S5935
AMCC logic interface
DDL interface
Receiver FIFO
Write DMA controller
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

7. FIRMWARE: DDL INTERFACE
Handles the full-duplex bus between the PRORC and the DIU
Provides clock domain separation using dual port FIFO memories
Transmit data and command to the DIU
Receive data and status from the DIU
Inject data into the DDL or the receive FIFO using the pattern generator
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

8. FIRMWARE: INTERNAL CONTROL
Interprets commands passed through the mailboxes
Controls other firmware block according to the required operation
Manages the DMA control registers in the AMCC
Handles the Free FIFO
Includes the read and write DMA engines
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

9. FIRMWARE: AMCC INTERFACE
Controls the half-duplex bus between the AMCC and the FPGA
Performs arbitration for different local bus accesses
Manages FIFO read and write operations
Manages mailbox read and write operations
Handles the hardware interrupt generated by the AMCC
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

10. OPERATION: THE FREE FIFO
PRORC
PC memory bank
Firmware
Free FIFO
page address
page address
page address
PC CPU
readout
Allocation of free pages
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

11. OPERATION: DMA TRANSFER
DDL
PRORC
PC memory bank
Firmware
PC CPU
No involvement
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

12. OPERATION: THE READY FIFO
DDL
PRORC
PC memory bank
Firmware
Ready FIFO
address
page status
address
page status
address
page status
PC CPU
readout
Delivery of filled pages
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

13. PCI-based Readout Receiver Card - LECC 2002, Colmar
THE TEST BED
Front-End Emulator Interface Card (FEIC): it emulates a detector readout
It generates data blocks of random size up to 1 MB
It drives the DDL SIU
It is sensitive to back-pressure
Source side
Two FEIC with SIU
Destination side
Two PRORC+DIU on a PCI bus and Gigabit Ethernet on the other
A PC 2 x Pentium III 1 GHz with two PCI bus (32bit/33MHz and 64bit/66MHz)
Linux operating system
ALICE data acquisition software (DATE)
FEIC
SIU
FEIC
SIU
DDL
PRORC
DIU
PRORC
DIU
LDC
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

14. PCI-based Readout Receiver Card - LECC 2002, Colmar
TEST I: SINGLE PRORC
Single PRORC w/o Event Builder
DDL saturated for block size above 5 kB:
101 MB/s
Event rate saturated for block size below 5 kB:
events/s
PRORC handling overhead in LDC: 28 µs
LDC
The firmware/software system withstands the DDL rate
The nominal system specification has been met
PRORC
DIU
FEIC
SIU
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

15. TEST II: TWO PRORC CARDS
Two PRORCs w/o Event Builder
Same saturation pattern, but the PCI bus is the limiting factor:
127 MB/s
Software overhead scales:
events/s
PRORC handling overhead in LDC: 54 µs
LDC
PRORC
DIU
The firmware/software system fully exploits the PC architecture
FEIC
SIU
FEIC
SIU
PRORC
DIU
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

16. TESTS III: PRORC + EVENT BUILDER
Single PRORC with Event Builder
The overall performance is determined by the speed of the Gigabit Ethernet card:
70 MB/s
Software overhead does not change:
events/s
PRORC handling overhead in LDC: 27 µs
Event Builder
The system behaves as expected and the performance fulfills the needs
LDC
GbE
PRORC
DIU
FEIC
SIU
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

17. PCI-based Readout Receiver Card - LECC 2002, Colmar
APPLICATIONS
Inner Tracking System (ITS), Silicon Drift (SD) detector
Dedicated ASIC performing DSP functions (CARLOS)
Interface to the DDL (CARLOS-rx)
Test patterns generated by a pattern generator have been collected using DATE
Time Projection Chamber (TPC)
Prototype Readout Control Unit (RCU) is tested with the PRORC
TPC sector is going to be tested at CERN
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

18. PCI-based Readout Receiver Card - LECC 2002, Colmar
SUMMARY
Hardware, firmware and software have been developed
The card is ready for production
PRORC provides efficient readout of the DDL
100 MB/s nominal speed has been achieved
PRORC and DDL have been successfully integrated in DATE
Long-term tests show stable performance
Tested with the DDL using Front-end Emulator Cards
Tested in standalone mode using the Embedded Data Generator
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar

19. PCI-based Readout Receiver Card - LECC 2002, Colmar
9-13 September, 2002
PCI-based Readout Receiver Card - LECC 2002, Colmar