1. PXD DAQ (PC option) Status Report
T.Higuchi (KEK)
Jan.26,2011
TRG/DAQ Workshop

2. Sketch of RocketIO  PCIe Card
Buffer full indicating signal
LVDS/RJ45
Buffer
Xilinx FPGA
XC5VFX70T-2?
Optical link
>6.25Gbps
Clocking Crystal
(312MHz)
AURORA on RocketIO
x8 PCIe (Gen1)
x4 PCIe (Gen2)

3. Status of the Option #3
For the preparation of the backup option, we put an order of a “pre-study” of the RocketIO  PCIe card performance to a company.
We pay mainly for the R&D of the firmware.
Purpose of the “pre-study”
Verify that the data transfer speed from the SFP+ through the PCIe server via an FPGA exceeds 6.25Gbps (before the 8b10b encoding) using a prototype RocketIO  PCIe card.
Measure the CPU load of the PCIe server at when it receives the data at 6.25Gbps.

4. Hardware Platform of the Pre-Study
EK-V6-ML605-G-J
Virtex-6 FPGA evaluation kit provided by Xilinx.
FPGA = XC6VLX240T.
Capability of PCIe Gen2 (x4).
x2 FMC connectors interfacing to optional daughter card.

5. Hardware Platform of the Pre-Study
TD-BD-FMC-OPT4BOARD
FMC daughter card with x4 SFP+.
AVAGO’s optical transceivers.
Released on Aug.11,2010 from Japanese company.
PCI-express host server

6. Schematic Drawing of the Pre-Study. . . . . . . . . . . . . . . .
TD-BD-FMC-OPT4BOARD . . . . . . . . . . . . . . . .
Loopback optical link
FMC
FMC
LX240T
PCIe
ML605
PCIe server

7. 1st Meeting with the Company
On Jan.18th, KEK
Status
Waiting for ML605 and the optical link card delivered.
For the time being, following items carried out:
Device driver development,
Firmware R&D using SP605 (Spartan6 eveluation kit)
Platform PC
The PCIe host PC should be prepared by KEK.
x58 chipset.
Purchased at ~$3,000 in the market.
For the time being, company’s PC is used for the R&D.
OS = CentOS 5.5 (kernel version = ).

8. 1st Meeting with the Company
Device drivers
[Q] How large physical memory can be allocated as a single chunk?
This would determine DMA performance.
[A] Only a few kB.
If one needs large memory, say 16MB, try followings:
static unsigned char buf[16 * 1024 * 1024];
Or build a special kernel that does not use last 16MB part of the physical memory.