Presentation is loading. Please wait.

Presentation is loading. Please wait.

16th IEEE NPSS Real Time Conference 2009 University of Heidelberg

Published byBarnard Wilson Modified over 6 years ago

Similar presentations

Presentation on theme: "16th IEEE NPSS Real Time Conference 2009 University of Heidelberg"— Presentation transcript:

1 A Unified Interconnection Network with Precise Time Synchronization for the CBM DAQ-System
16th IEEE NPSS Real Time Conference 2009 University of Heidelberg Computer Architecture Group Frank Lemke, Niels Burkhardt, David Slogsnat, Ulrich Bruening

2 16th IEEE NPSS Real Time Conference 2009
Outline Motivation FAIR & CBM CBM Interconnection Network and Protocol Deterministic Latency Features Conclusion & Outlook 16th IEEE NPSS Real Time Conference 2009

3 16th IEEE NPSS Real Time Conference 2009
Motivation CBM detector of FAIR at the GSI in Darmstadt has special demands on the DAQ System Synchronization mechanisms Limited space for hardware Radiation tolerance Self triggered frontend electronics Flexible enough for all required hierarchical structures Leads to the development of a new CBM network protocol 16th IEEE NPSS Real Time Conference 2009

4 FAIR at GSI Darmstadt Germany
Facility for Antiproton and Ion Research (FAIR) will extend the existing GSI accelerator and synchrotron (until 2016) with Construction of two high-energy superconducting synchrotrons Five storage rings Numerous new detector serving five fields of physics Science Vol 318, 2 November 2007 16th IEEE NPSS Real Time Conference 2009

5 16th IEEE NPSS Real Time Conference 2009
CBM Compressed Baryonic Matter (CBM) Detectors components are: Silicon Tracking System (STS) Micro-Vertex Detector (MVD) Ring Imaging Cherenkov detector (RICH) Transition Radiation Detectors (TRD) Muon Chamber/absorber system (MUCH) Resistive Plate Chambers (RPC) Electromagnetic Calorimeter (ECAL) Projectile Spectator Detector (PSD) “Design and implementation of a hierarchical DAQ network,” Deutsche Physikalische Gesellschaft e.V. Fruehjahrstagung, March 10, 2008 16th IEEE NPSS Real Time Conference 2009

6 CBM Network Requirements
High data bandwidth with error detection Reliable fault tolerant solution for control messages Synchronization of all detector end nodes Space efficiency Cost limitations Usage of only one fiber link seems to be an efficient solution 16th IEEE NPSS Real Time Conference 2009

7 Hierarchical Network Concept
Flexible hierarchical structure Readout Controller (ROC) for connecting to different kinds of detectors Data Combiner Board (DCB) for bundling and processing the data Active Buffer Board (ABB) for processing and cluster connect Data flow ABB DCB ROC … FEE Slow Control Administration Packets 16th IEEE NPSS Real Time Conference 2009

8 16th IEEE NPSS Real Time Conference 2009
CBM Network - Features Forward error correction of 1-Bit for all special characters Protocol is extendable to continue sending over networks Open-ended hierarchical structure for concentrator network Automatic initialization Optimized data bandwidth about 90 % ( about 73 % with 8b/10b) Hardware retransmission for Control Packets System wide clock recovery with low jitter Deterministic link latency feature for well defined Deterministic Latency Messages 16th IEEE NPSS Real Time Conference 2009

9 16th IEEE NPSS Real Time Conference 2009
Jitter Cleaner Default recovered peak-to-peak clock jitter between ps is not good enough for FPGAs (MGT) solution build a jitter cleaner Jitter cleaner facts: Connected through mezzanine SI570 CDCLVD110 LMK03000 (as jitter cleaner) Cleaned clock jitter: Peak-to-peak below required 40ps RMS jitter below 10ps 16th IEEE NPSS Real Time Conference 2009

10 Jitter Cleaner Measurement Setup
16th IEEE NPSS Real Time Conference 2009

11 16th IEEE NPSS Real Time Conference 2009
Jitter Measurement 16th IEEE NPSS Real Time Conference 2009

12 CBM Network - Traffic Classes
Data Transport Messages (DTM) Data Messages with CRC Only with error detection Detector Control Messages (DCM) Control Messages with CRC Retransmission on error Deterministic Latency Messages (DLM) 1-Bit Error correction Additional Administration Packets IDLE, INIT, ACK, NACK … Arbiter Control Retransmission CRC Data CRC Link Link_init DLM ACK/NACK 16th IEEE NPSS Real Time Conference 2009

13 16th IEEE NPSS Real Time Conference 2009
CBM Protocol - Packets DTM: DATA CRC SOP EOP 16+2 Bit 8 – 64 Byte DCM: Control CRC SOSC EOP 16+2 Bit 8 – 64 Byte DLM: SYNC DLM … 16+2 Bit Administrative packets: SYNC INIT IDLE … 16+2 Bit ACK 16th IEEE NPSS Real Time Conference 2009

14 Deterministic Latency Message
Insertion into every part of a packet: CRC EOP 16+2 Bit DATA SOP 8 – 64 Byte DLM … DLM 16+2 Bit DATA CRC SOP 8 – 64 Byte EOP DATA SOP 16+2 Bit 8 – 64 Byte CRC EOP DLM 16th IEEE NPSS Real Time Conference 2009

15 CBM Network - Specifics
A special feature is providing DLM Enables time synchronization with epoch markers Various special user defined event signalling with deterministic latency Specific implementation through Priority request insertion Specific FPGA configuration Deterministic and structural separated HDL coding 16th IEEE NPSS Real Time Conference 2009

16 16th IEEE NPSS Real Time Conference 2009
CBM Network - Facts The CBM network modules are implemented in Verilog HDL Optimized link interface for easy adaption in any part of the network FPGA independent code, no vendor specific modules in the logic implementation used Serializer with the possibility for deterministic insertion from Xilinx used; others have to be tested Clock distribution with low jitter Easily portable to an ASIC 16th IEEE NPSS Real Time Conference 2009

17 Status of the CBM Network
Concept of the network protocol is complete All parts of the network are simulated All concepts are proven in hardware Lab tests have shown the usability Hardware parts for a Demonstrator are developed, build and successfully tested by the different groups at several locations Leads to first Demonstrator which is currently being build 16th IEEE NPSS Real Time Conference 2009

18 16th IEEE NPSS Real Time Conference 2009
Conclusion & Outlook Prove of concept done, all components tested Beam time in August/September with Demonstrator-I will be build-up Our experiences led us to this next generation network, protocol and system boards Readout Controller Data Combiner Board Active Buffer Board 16th IEEE NPSS Real Time Conference 2009

19 16th IEEE NPSS Real Time Conference 2009
Thank you for your attention! Questions? 16th IEEE NPSS Real Time Conference 2009

20 16th IEEE NPSS Real Time Conference 2009
Back-up Slides 16th IEEE NPSS Real Time Conference 2009

21 Data Combiner Boards - Facts
- HTX Connector with 16bit LVDS bidirectional interface - FPGA Virtex-4 FX60/100 speed grade -10,-11 or -12 - 6 SFPs - 128MByte of DDR2 DRAM (optional 512MByte) - 512 Mb of user FLASH, 16bit interface to FPGA - 125MHz low jitter clock oscillator - Power supply with only 12V and 3.3V from HTX connector - Power consumption of 6 to 24W DCB is completely tested DCB is successfully in use by many partners 16th IEEE NPSS Real Time Conference 2009

22 16th IEEE NPSS Real Time Conference 2009
Interface Definition 16th IEEE NPSS Real Time Conference 2009

Download ppt "16th IEEE NPSS Real Time Conference 2009 University of Heidelberg"

Similar presentations

Sumitha Ajith Saicharan Bandarupalli Mahesh Borgaonkar.

Sumitha Ajith Saicharan Bandarupalli Mahesh Borgaonkar.
FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005.

FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005.
An ATCA and FPGA-Based Data Processing Unit for PANDA Experiment H.XU, Z.-A. LIU,Q.WANG, D.JIN, Inst. High Energy Physics, Beijing, W. Kühn, J. Lang, S.

An ATCA and FPGA-Based Data Processing Unit for PANDA Experiment H.XU, Z.-A. LIU,Q.WANG, D.JIN, Inst. High Energy Physics, Beijing, W. Kühn, J. Lang, S.
Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden 2013 HK 10.7 University of Heidelberg Computer Architecture.

Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden 2013 HK 10.7 University of Heidelberg Computer Architecture.
SODA: Synchronization Of Data Acquisition I.Konorov  Requirements  Architecture  System components  Performance  Conclusions and outlook PANDA FE-DAQ.

SODA: Synchronization Of Data Acquisition I.Konorov  Requirements  Architecture  System components  Performance  Conclusions and outlook PANDA FE-DAQ.
TRBnet for the CBM MVD-Prototype Borislav Milanović In cooperation with: J. Michel, M. Deveaux, S. Seddiki, M. Traxler, S. Youcef, C. Schrader, I. Fröhlich,

TRBnet for the CBM MVD-Prototype Borislav Milanović In cooperation with: J. Michel, M. Deveaux, S. Seddiki, M. Traxler, S. Youcef, C. Schrader, I. Fröhlich,
Tuesday September Cambridge1 GDCC “next replacement of the LDA” Franck GASTALDI.

Tuesday September Cambridge1 GDCC “next replacement of the LDA” Franck GASTALDI.
Future DAQ Directions David Bailey for the CALICE DAQ Group.

Future DAQ Directions David Bailey for the CALICE DAQ Group.
DEVICES AND COMMUNICATION BUSES FOR DEVICES NETWORK

DEVICES AND COMMUNICATION BUSES FOR DEVICES NETWORK
U N C L A S S I F I E D FVTX Detector Readout Concept S. Butsyk For LANL P-25 group.

U N C L A S S I F I E D FVTX Detector Readout Concept S. Butsyk For LANL P-25 group.
Understanding Data Acquisition System for N- XYTER.

Understanding Data Acquisition System for N- XYTER.
Network-on-Chip Energy-Efficient Design Techniques for Interconnects Suhail Basit.

Network-on-Chip Energy-Efficient Design Techniques for Interconnects Suhail Basit.
A PCI Card for Readout in High Energy Physics Experiments Michele Floris 1,2, Gianluca Usai 1,2, Davide Marras 2, André David 3 2003 IEEE Nuclear Science.

A PCI Card for Readout in High Energy Physics Experiments Michele Floris 1,2, Gianluca Usai 1,2, Davide Marras 2, André David IEEE Nuclear Science.
Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University.

Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University.
Data Acquisition Backbone Core J. Adamczewski-Musch, N. Kurz, S. Linev GSI, Experiment Electronics, Data processing group.

Data Acquisition Backbone Core J. Adamczewski-Musch, N. Kurz, S. Linev GSI, Experiment Electronics, Data processing group.
An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.

An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.
Design Criteria and Proposal for a CBM Trigger/DAQ Hardware Prototype Joachim Gläß Computer Engineering, University of Mannheim Contents –Requirements.

Design Criteria and Proposal for a CBM Trigger/DAQ Hardware Prototype Joachim Gläß Computer Engineering, University of Mannheim Contents –Requirements.
Summary DAQ Walter F.J. Müller, GSI, Darmstadt 12 th CBM Collaboration Meeting 17 October 2008.

Summary DAQ Walter F.J. Müller, GSI, Darmstadt 12 th CBM Collaboration Meeting 17 October 2008.
LHCb front-end electronics and its interface to the DAQ.

LHCb front-end electronics and its interface to the DAQ.
1 07/10/07 Forward Vertex Detector Technical Design – Electronics DAQ Readout electronics split into two parts – Near the detector (ROC) – Compresses and.

1 07/10/07 Forward Vertex Detector Technical Design – Electronics DAQ Readout electronics split into two parts – Near the detector (ROC) – Compresses and.

Similar presentations

Ads by Google