R&D for First Level Farm Hardware Processors Joachim Gläß Computer Engineering, University of Mannheim Contents –Overview of Processing Architecture –Requirements for Hardware Processors –R&D for Hardware Processors –Conclusion March 10, 2005 CBM Collaboration Meeting

Overview of Processing Architecture Processing resources Hardware processors –L1/FPGA Software processors –L1/CPU Active Buffers Sub-farm network –Pnet Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Overview of Processing Architecture Processing resources Sub-farm size –1000 x 10 Gbit/s detector output links –storing and sorting according to epochs/events –several epochs per sub- farm (no duplicate MAPS data) one scenario –64 sub-farms –16 epochs stored and processed per subfarm –4 AB units (á 4 ABs) –8 L1/FPGA units (á 4 processors) –32 L1/CPU Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Overview of Processing Architecture Processing resources Active Buffer (AB) –interface between detector link, Bnet and Pnet store data from detector link and output to Bnet receive sorted data according to epochs from Bnet event building according to timestamps within one epoch output event data to processing units –histogramming of timestamps, data transfer protocols, control of large memory => FPGA Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Overview of Processing Architecture Processing resources Hardware processors (L1/FPGA) –process 10 Gbit/s input data stream in real-time –most reasonable technology: FPGA => L1/FPGA –processing in parallel hardware + flexibility of programming –forefront of chip development (regular internal structure) > 100 k logic gates, > 500 MHz for complex algorithms integrated multi gigabit transmitter (MGT): up to 20 x 10 Gbit/s serial I/O Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Overview of Processing Architecture Processing resources Software processors (L1/CPU) –general purpose processors PCs system-on-a-chip processors (better GFlops/€ or GFlops/Watt ?) –IBM Blue Gene, STI Cell processor, Strech S5000 Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Overview of Processing Architecture Processing resources Sub-farm network (Pnet) –connections only inside one sub-farm (short distances ~1 m) –freely programmable although for fixed algorithm only fixed point-to-point links NO all-to-all network, like Bnet –usage of built-in MGTs of FPGAs COTS switches PCIe plausible candidate Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Requirements for Hardware Processors Algorithms in FPGA functional:-> simulation timing:-> simulation, (R&D prototype) External memories (data buffers, large memories, …) –e.g. LUTs for Hough transform: several MByte (ca. 20 addresses) => ZBT SRAM functional:-> simulation timing:-> simulation, R&D prototype (clock feedback, termination, …) –e.g. Active Buffer: several tenth of MBytes (1000 x 10µs x 1GByte/s) => DDR SDRAM functional:-> simulation timing:-> simulation, R&D prototype (clock feedback, termination, …) Interconnection network –e.g. Pnet, MGTs up to 10 Gbit/s –test physical layer of multi gigabit communication chip-to-chip:-> simulation of impedances, R&D prototype board-to-board: –optical:-> chip-to-chip –backplane:-> simulation of impedances, connectors, R&D prototype Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Architecture of R&D Prototype communication via backplane –4 boards, all-to-all –different length of traces –up to 10 Gbit/s serial –=> FR4 Rogers DDR ZBT FPGA connector SFPSFP XC2VPX20 Flash RS232 Ethernet PPC zeroXT 10GB SMT Ethernet Flash DDR Linux µC FPGA with MGTs –up to 10 Gbit/s serial –=> XC2VPX20 (8 x MGT) –=> XC2VPX70 (20 x MGT) externals –2 x ZBT SRAM –2 x DDR SDRAM –for PPC: Flash, Ethernet, … initialization and control –standalone board/system –microcontroller running Linux Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering

Conclusion R&D prototype to learn: –physical layer of communication 2.5 Gbit/s up to 10 Gbit/s chip-to-chip board-to-board (-> connectors, backplane) PCB layout, impedances PCB material (FR4, Rogers, …) –next step: communication protocols –more resources needed => XC2VPX70?, Virtex4? (availability?) –external memories fast controllers for ZBT and DDR RAM PCB layout, termination, … Joachim Gläß, Univ. Mannheim, Institute of Computer Engineering