Presentation is loading. Please wait.

Presentation is loading. Please wait.

Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University.

Published byErik Ferguson Modified over 8 years ago

Similar presentations

Presentation on theme: "Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University."— Presentation transcript:

1 Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University of Heidelberg Frank Lemke, Sebastian Manz and Wenxue Gao 19.03.2010

2 Frank Lemke DPG Frühjahrstagung 2010 2 Outline Data Acquisition Topology DAQ Features DAQ Components -Readout Controller -Data Combiner Board -Active Buffer Board Conclusion & Outlook

3 Frank Lemke DPG Frühjahrstagung 2010 Data Acquisition Topology ABB DCB ROC … FEB DCB ROC … FEB Computing Cluster (FLES) … … Detector System Optical connection Non-Optical connection FEB = Frontend Board ROC = Readout Controller DCB = Data Combiner Board ABB = Active Buffer Board

4 Frank Lemke DPG Frühjahrstagung 2010 CBM Optical Link Protocol CBM protocol -Three traffic classes -Delivers specific adapted performance & features Generic link protocol -Minimized interface -Modularized -Efficient interface synchronization -Transparent UserNetwork

5 Frank Lemke DPG Frühjahrstagung 2010 ROC Clock & Synchronization Net ROC Frontend to Detector System ………………… DCB Synchronization and clock distribution over optical link Backend to Computing Cluster (FLES) …… JC JC = Jitter cleaning and clock recovery ROC JC Clock & Sync. Gen. Tree structured connection

6 Frank Lemke DPG Frühjahrstagung 2010 Jitter Cleaner Module Requirements -Internal deterministic implementation -Recovered clock usage -Peak-to-peak below 40ps Results -Extender connected with mezzanine -National LMK03000 Family -MMCX to transmit clock -Bit precise synchronisation V 1.0 V 1.1

7 Frank Lemke DPG Frühjahrstagung 2010 Readout Controller Board Functions -Interfacing the front-end electronics -Timestamp expansion -Data transport via optics (or Ethernet)‏ Board development -SysCore 1: Virtex 4 FX20 -SysCore 2: better suited connectors -SysCore 3 (planned): mother- and daughter- board, Spartan6? V 1.0 V 2.0 (exemplary)‏

8 Frank Lemke DPG Frühjahrstagung 2010 ROC – Functional Diagram Virtex-4 FX20 Ethernet, MGTs for communication DDR SDRAM Actel Flash-FPGA for configuration SD-Card and Flash Many User-I/O pins (picture from http://cbm-wiki.gsi.de)‏

9 Frank Lemke DPG Frühjahrstagung 2010 ROC – Modular Firmware Design ? ? Readout Logic Transport Logic nXYTER ROC nXYTER ROC FEET ROC Ethernet MGTs Bus FIFO ABB Driver DABC GPIO © N.Abel

10 Frank Lemke DPG Frühjahrstagung 2010 Data Combiner Board Functions -Data switching and concentration multiple ROCs => ABB -6 SFPs -250MHz low jitter programmable clock oscillator -Interface for jitter cleaner mezzanine Board development -DCB V1.0: Virtex-4 FX100 -DCB V1.3: Improvements on MGT reference clock insertion V 1.0 V 1.3

11 Frank Lemke DPG Frühjahrstagung 2010 Receive clk Transmit clk Functional Diagram DCB V1.3 X V6-fx100 JC ABB SFP DLM LP Special deterministic latency message handling Erroneous detector control message retransmission Efficient switching mechanism Well defined clocking scheme ROCs

12 Frank Lemke DPG Frühjahrstagung 2010 Hardware Developments Protocol development Protocol test and verification Clock distribution with jitter cleaning Synchronization via deterministic latency messages Crossbar and switching logic

13 Frank Lemke DPG Frühjahrstagung 2010 Active Buffer Board Functions -Buffering the incoming hit packets -Hits reorganization -Transfer events to the host node -PCI Express 4-lane -DMA upstream: DAQ data packets from buffer to the host node Board development -ABB-1: Virtex 4 FX60 -ABB-2 (AVNET): Virtex 5 LX110T -ABB-3 (ML605): Virtex 6 LX240T ABB-1 ABB-3

14 Frank Lemke DPG Frühjahrstagung 2010 ABB Diagram and Measurement Control messages invoke the data flow from ROC via DCB to ABB Hit packets are first buffered in the FIFO Host software starts DMA to transfer the buffered hits to the host memory Measured bandwidth of 224 MB/s for one optical link

15 Frank Lemke DPG Frühjahrstagung 2010 Software Developments Driver under Linux 2.6 Performance emphasized Well-layered architecture ABBDaemon masks driver details to the users -Enables access from multiple users -Uses UNIX IPC mechanism Provided test programs -RAM memory -One-hole memory / FIFO -CTL messages

16 Frank Lemke DPG Frühjahrstagung 2010 Test Setup

17 Frank Lemke DPG Frühjahrstagung 2010 Conclusion & Outlook Prototype systems working and in use Virtex6 adaption and development for all boards ROC -Modular hardware (mother/daughterboard)‏ DCB -Protocol HDL optimizations ABB -Event Building - Flexible strategies for hit dispatching -Unified test program for more efficient debugging

18 Frank Lemke DPG Frühjahrstagung 2010 18 Questions ? Thank you for your attention !

Download ppt "Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University."

Similar presentations

StreamBlade SOE TM Initial StreamBlade TM Stream Offload Engine (SOE) Single Board Computer SOE-4-PCI Rev 1.2.

StreamBlade SOE TM Initial StreamBlade TM Stream Offload Engine (SOE) Single Board Computer SOE-4-PCI Rev 1.2.
Protocols and software for exploiting Myrinet clusters Congduc Pham and the main contributors P. Geoffray, L. Prylli, B. Tourancheau, R. Westrelin.

Protocols and software for exploiting Myrinet clusters Congduc Pham and the main contributors P. Geoffray, L. Prylli, B. Tourancheau, R. Westrelin.
FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005.

FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005.
An ATCA and FPGA-Based Data Processing Unit for PANDA Experiment H.XU, Z.-A. LIU,Q.WANG, D.JIN, Inst. High Energy Physics, Beijing, W. Kühn, J. Lang, S.

An ATCA and FPGA-Based Data Processing Unit for PANDA Experiment H.XU, Z.-A. LIU,Q.WANG, D.JIN, Inst. High Energy Physics, Beijing, W. Kühn, J. Lang, S.
DABCDABC D ata A cquisition B ackbone C ore RT2010 J.Adamczewski-Musch, H.G.Essel, S.Linev 1 Data Acquisition Backbone Core Framework.

DABCDABC D ata A cquisition B ackbone C ore RT2010 J.Adamczewski-Musch, H.G.Essel, S.Linev 1 Data Acquisition Backbone Core Framework.
1 Design of the Front End Readout Board for TORCH Detector 10, June 2010.

1 Design of the Front End Readout Board for TORCH Detector 10, June 2010.
General Purpose FIFO on Virtex-6 FPGA ML605 board midterm presentation

General Purpose FIFO on Virtex-6 FPGA ML605 board midterm presentation
Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf

Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf
General Purpose FIFO on Virtex-6 FPGA ML605 board Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf 1 Semester: spring 2012.

General Purpose FIFO on Virtex-6 FPGA ML605 board Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf 1 Semester: spring 2012.
24/03/2011P.-A. Loizeau – Univ. Heidelberg The CBM Time of Flight wall electronic readout chain Pierre-Alain Loizeau PI – Uni Heidelberg DPG 2011, Münster.

24/03/2011P.-A. Loizeau – Univ. Heidelberg The CBM Time of Flight wall electronic readout chain Pierre-Alain Loizeau PI – Uni Heidelberg DPG 2011, Münster.
Data Acquisition Backbone Core DABC J. Adamczewski, H.G. Essel, N. Kurz, S. Linev GSI, Darmstadt The new Facility for Antiproton and Ion Research at GSI.

Data Acquisition Backbone Core DABC J. Adamczewski, H.G. Essel, N. Kurz, S. Linev GSI, Darmstadt The new Facility for Antiproton and Ion Research at GSI.
HEP UCL Cambridge University Imperial College London University of Manchester Royal Holloway, University of London University College London Matthew Warren,

HEP UCL Cambridge University Imperial College London University of Manchester Royal Holloway, University of London University College London Matthew Warren,
Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden 2013 HK 10.7 University of Heidelberg Computer Architecture.

Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden 2013 HK 10.7 University of Heidelberg Computer Architecture.
The MPC Parallel Computer Hardware, Low-level Protocols and Performances University P. & M. Curie (PARIS) LIP6 laboratory Olivier Glück.

The MPC Parallel Computer Hardware, Low-level Protocols and Performances University P. & M. Curie (PARIS) LIP6 laboratory Olivier Glück.
SODA: Synchronization Of Data Acquisition I.Konorov  Requirements  Architecture  System components  Performance  Conclusions and outlook PANDA FE-DAQ.

SODA: Synchronization Of Data Acquisition I.Konorov  Requirements  Architecture  System components  Performance  Conclusions and outlook PANDA FE-DAQ.
TRIGGER-LESS AND RECONFIGURABLE DATA ACQUISITION SYSTEM FOR POSITRON EMISSION TOMOGRAPHY Grzegorz Korcyl 2013.

TRIGGER-LESS AND RECONFIGURABLE DATA ACQUISITION SYSTEM FOR POSITRON EMISSION TOMOGRAPHY Grzegorz Korcyl 2013.
ETRAX CRIS architecture and Xilinx FPGA Peter Zumbruch Experiment control systems group GSI (KS/EE)

ETRAX CRIS architecture and Xilinx FPGA Peter Zumbruch Experiment control systems group GSI (KS/EE)
RiceNIC: A Reconfigurable and Programmable Gigabit Network Interface Card Jeff Shafer, Dr. Scott Rixner Rice Computer Architecture:

RiceNIC: A Reconfigurable and Programmable Gigabit Network Interface Card Jeff Shafer, Dr. Scott Rixner Rice Computer Architecture:
Future DAQ Directions David Bailey for the CALICE DAQ Group.

Future DAQ Directions David Bailey for the CALICE DAQ Group.
Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Similar presentations

Ads by Google