Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University.

Slides:

Advertisements

Similar presentations

StreamBlade SOE TM Initial StreamBlade TM Stream Offload Engine (SOE) Single Board Computer SOE-4-PCI Rev 1.2.

Advertisements

Protocols and software for exploiting Myrinet clusters Congduc Pham and the main contributors P. Geoffray, L. Prylli, B. Tourancheau, R. Westrelin.

FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005.

An ATCA and FPGA-Based Data Processing Unit for PANDA Experiment H.XU, Z.-A. LIU,Q.WANG, D.JIN, Inst. High Energy Physics, Beijing, W. Kühn, J. Lang, S.

DABCDABC D ata A cquisition B ackbone C ore RT2010 J.Adamczewski-Musch, H.G.Essel, S.Linev 1 Data Acquisition Backbone Core Framework.

1 Design of the Front End Readout Board for TORCH Detector 10, June 2010.

General Purpose FIFO on Virtex-6 FPGA ML605 board midterm presentation

Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf

General Purpose FIFO on Virtex-6 FPGA ML605 board Students: Oleg Korenev Eugene Reznik Supervisor: Rolf Hilgendorf 1 Semester: spring 2012.

24/03/2011P.-A. Loizeau – Univ. Heidelberg The CBM Time of Flight wall electronic readout chain Pierre-Alain Loizeau PI – Uni Heidelberg DPG 2011, Münster.

Data Acquisition Backbone Core DABC J. Adamczewski, H.G. Essel, N. Kurz, S. Linev GSI, Darmstadt The new Facility for Antiproton and Ion Research at GSI.

HEP UCL Cambridge University Imperial College London University of Manchester Royal Holloway, University of London University College London Matthew Warren,

Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden 2013 HK 10.7 University of Heidelberg Computer Architecture.

The MPC Parallel Computer Hardware, Low-level Protocols and Performances University P. & M. Curie (PARIS) LIP6 laboratory Olivier Glück.

SODA: Synchronization Of Data Acquisition I.Konorov  Requirements  Architecture  System components  Performance  Conclusions and outlook PANDA FE-DAQ.

TRIGGER-LESS AND RECONFIGURABLE DATA ACQUISITION SYSTEM FOR POSITRON EMISSION TOMOGRAPHY Grzegorz Korcyl 2013.

ETRAX CRIS architecture and Xilinx FPGA Peter Zumbruch Experiment control systems group GSI (KS/EE)

RiceNIC: A Reconfigurable and Programmable Gigabit Network Interface Card Jeff Shafer, Dr. Scott Rixner Rice Computer Architecture:

Future DAQ Directions David Bailey for the CALICE DAQ Group.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

GBT Interface Card for a Linux Computer Carson Teale 1.

U N C L A S S I F I E D FVTX Detector Readout Concept S. Butsyk For LANL P-25 group.

NEDA collaboration meeting at IFIC Valencia, 3rd-5th November 2010 M. Tripon EXOGAM2 project Digital instrumentation of the EXOGAM detector EXOGAM2 - Overview.

Understanding Data Acquisition System for N- XYTER.

Design and Performance of a PCI Interface with four 2 Gbit/s Serial Optical Links Stefan Haas, Markus Joos CERN Wieslaw Iwanski Henryk Niewodnicznski Institute.

David Bailey University of Manchester. Overview Aim to develop a generic system –Maximise use of off-the-shelf commercial components Reduce as far as.

R&D for First Level Farm Hardware Processors Joachim Gläß Computer Engineering, University of Mannheim Contents –Overview of Processing Architecture –Requirements.

D ata A cquisition B ackbone C ore DABCDABC , Huelva J.Adamczewski, H.G.Essel, N.Kurz, S.Linev 1 Work.

Management of the LHCb DAQ Network Guoming Liu * †, Niko Neufeld * * CERN, Switzerland † University of Ferrara, Italy.

Data Acquisition Backbone Core J. Adamczewski-Musch, N. Kurz, S. Linev GSI, Experiment Electronics, Data processing group.

An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.

Design Criteria and Proposal for a CBM Trigger/DAQ Hardware Prototype Joachim Gläß Computer Engineering, University of Mannheim Contents –Requirements.

Latest ideas in DAQ development for LHC B. Gorini - CERN 1.

Summary DAQ Walter F.J. Müller, GSI, Darmstadt 12 th CBM Collaboration Meeting 17 October 2008.

1 07/10/07 Forward Vertex Detector Technical Design – Electronics DAQ Readout electronics split into two parts – Near the detector (ROC) – Compresses and.

DABCDABC D ata A cquisition B ackbone C ore J.Adamczewski, H.G.Essel, N.Kurz, S.Linev 1 Work supported by EU RP6 project.

18/03/2010 DPG Bonn – March Session: HK 48 A demonstrator for the CBM Time of Flight wall electronic readout chain Pierre-Alain Loizeau PI – Uni.

Status DAQ Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 14 th CBM Collaboration Meeting Friday, 9 October 2009.

Active Buffer Status in CBM DAQ W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz 06 Oct th CBM Collaboration Meeting Split.

Modeling PANDA TDAQ system Jacek Otwinowski Krzysztof Korcyl Radoslaw Trebacz Jagiellonian University - Krakow.

Management of the LHCb Online Network Based on SCADA System Guoming Liu * †, Niko Neufeld † * University of Ferrara, Italy † CERN, Geneva, Switzerland.

DABCDABC ROC-based DAQ: latest developments and perspectives Jörn Adamczewski-Musch, Hans G. Essel, Sergey Linev GSI, Experiment Electronics: Data Processing.

STS Readout chain – Revisited Readout-ASIC to ROC Interconnect Walter F.J. Müller, GSI, Darmstadt CBM Collaboration Meeting 13 th April 2010.

1 SysCore for N-XYTER Status Report Talk by Dirk Gottschalk Kirchhoff Institut für Physik Universität Heidelberg.

XLV INTERNATIONAL WINTER MEETING ON NUCLEAR PHYSICS Tiago Pérez II Physikalisches Institut For the PANDA collaboration FPGA Compute node for the PANDA.

Guirao - Frascati 2002Read-out of high-speed S-LINK data via a buffered PCI card 1 Read-out of High Speed S-LINK Data Via a Buffered PCI Card A. Guirao.

DDRIII BASED GENERAL PURPOSE FIFO ON VIRTEX-6 FPGA ML605 BOARD PART B PRESENTATION STUDENTS: OLEG KORENEV EUGENE REZNIK SUPERVISOR: ROLF HILGENDORF 1 Semester:

BPM stripline acquisition in CLEX Sébastien Vilalte.

Trigger Hardware Development Modular Trigger Processing Architecture Matt Stettler, Magnus Hansen CERN Costas Foudas, Greg Iles, John Jones Imperial College.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Research Unit for Integrated Sensor Systems and Oregano Systems Cern Timing Workshop 2008 Patrick Loschmidt, Georg Gaderer, and Nikolaus Kerö.

S.Anvar, V.Gautard, H.Le Provost, F.Louis, K.Menager, Y.Moudden, B.Vallage, E.Zonca, on behalf of the KM3NeT consortium 1 IRFU/SEDI-CEA Saclay F

17/02/06H-RORCKIP HeidelbergTorsten Alt The new H-RORC H-RORC.

Rutherford Appleton Laboratory September 1999Fifth Workshop on Electronics for LHC Presented by S. Quinton.

KM3NeT Offshore Readout System On Chip A highly integrated system using FPGA COTS S. Anvar, H. Le Provost, F. Louis, B.Vallage – CEA Saclay IRFU – Amsterdam/NIKHEF,

DHH Status Igor Konorov TUM, Physics Department, E18 PXD DAQ workshop Münzenberg –June 9-10, 2011.

PC-based L0TP Status Report “on behalf of the Ferrara L0TP Group” Ilaria Neri University of Ferrara and INFN - Italy Ferrara, September 02, 2014.

The Evaluation Tool for the LHCb Event Builder Network Upgrade Guoming Liu, Niko Neufeld CERN, Switzerland 18 th Real-Time Conference June 13, 2012.

Firmware and Software for the PPM DU S. Anvar, H. Le Provost, Y.Moudden, F. Louis, E.Zonca – CEA Saclay IRFU – Amsterdam/NIKHEF, 2011 March 30.

DHH at DESY Test Beam 2016 Igor Konorov TUM Physics Department E18 19-th DEPFET workshop May Kloster Seeon Overview: DHH system overview DHE/DHC.

CHEP 2010, October 2010, Taipei, Taiwan 1 18 th International Conference on Computing in High Energy and Nuclear Physics This research project has.

M. Bellato INFN Padova and U. Marconi INFN Bologna

16th IEEE NPSS Real Time Conference 2009 University of Heidelberg

“FPGA shore station demonstrator for KM3NeT”

CALICE DAQ Developments

TELL1 A common data acquisition board for LHCb

CoBo - Different Boundaries & Different Options of

STAR-CBM Joint Workshop Heidelberg, Physikalisches Institut

TELL1 A common data acquisition board for LHCb

Cluster Computers.

Presentation transcript:

Frank Lemke DPG Frühjahrstagung 2010 Time synchronization and measurements of a hierarchical DAQ network DPG Conference Bonn 2010 Session: HK 70.3 University of Heidelberg Frank Lemke, Sebastian Manz and Wenxue Gao

Frank Lemke DPG Frühjahrstagung Outline Data Acquisition Topology DAQ Features DAQ Components -Readout Controller -Data Combiner Board -Active Buffer Board Conclusion & Outlook

Frank Lemke DPG Frühjahrstagung 2010 Data Acquisition Topology ABB DCB ROC … FEB DCB ROC … FEB Computing Cluster (FLES) … … Detector System Optical connection Non-Optical connection FEB = Frontend Board ROC = Readout Controller DCB = Data Combiner Board ABB = Active Buffer Board

Frank Lemke DPG Frühjahrstagung 2010 CBM Optical Link Protocol CBM protocol -Three traffic classes -Delivers specific adapted performance & features Generic link protocol -Minimized interface -Modularized -Efficient interface synchronization -Transparent UserNetwork

Frank Lemke DPG Frühjahrstagung 2010 ROC Clock & Synchronization Net ROC Frontend to Detector System ………………… DCB Synchronization and clock distribution over optical link Backend to Computing Cluster (FLES) …… JC JC = Jitter cleaning and clock recovery ROC JC Clock & Sync. Gen. Tree structured connection

Frank Lemke DPG Frühjahrstagung 2010 Jitter Cleaner Module Requirements -Internal deterministic implementation -Recovered clock usage -Peak-to-peak below 40ps Results -Extender connected with mezzanine -National LMK03000 Family -MMCX to transmit clock -Bit precise synchronisation V 1.0 V 1.1

Frank Lemke DPG Frühjahrstagung 2010 Readout Controller Board Functions -Interfacing the front-end electronics -Timestamp expansion -Data transport via optics (or Ethernet)‏ Board development -SysCore 1: Virtex 4 FX20 -SysCore 2: better suited connectors -SysCore 3 (planned): mother- and daughterboard, Spartan6? V 1.0 V 2.0 (exemplary)‏

Frank Lemke DPG Frühjahrstagung 2010 ROC – Functional Diagram Virtex-4 FX20 Ethernet, MGTs for communication DDR SDRAM Actel Flash-FPGA for configuration SD-Card and Flash Many User-I/O pins (picture from

Frank Lemke DPG Frühjahrstagung 2010 ROC – Modular Firmware Design ? ? Readout Logic Transport Logic nXYTER ROC nXYTER ROC FEET ROC Ethernet MGTs Bus FIFO ABB Driver DABC GPIO © N.Abel

Frank Lemke DPG Frühjahrstagung 2010 Data Combiner Board Functions -Data switching and concentration multiple ROCs => ABB -6 SFPs -250MHz low jitter programmable clock oscillator -Interface for jitter cleaner mezzanine Board development -DCB V1.0: Virtex-4 FX100 -DCB V1.3: Improvements on MGT reference clock insertion V 1.0 V 1.3

Frank Lemke DPG Frühjahrstagung 2010 Receive clk Transmit clk Functional Diagram DCB V1.3 X V6-fx100 JC ABB SFP DLM LP Special deterministic latency message handling Erroneous detector control message retransmission Efficient switching mechanism Well defined clocking scheme ROCs

Frank Lemke DPG Frühjahrstagung 2010 Hardware Developments Protocol development Protocol test and verification Clock distribution with jitter cleaning Synchronization via deterministic latency messages Crossbar and switching logic

Frank Lemke DPG Frühjahrstagung 2010 Active Buffer Board Functions -Buffering the incoming hit packets -Hits reorganization -Transfer events to the host node -PCI Express 4-lane -DMA upstream: DAQ data packets from buffer to the host node Board development -ABB-1: Virtex 4 FX60 -ABB-2 (AVNET): Virtex 5 LX110T -ABB-3 (ML605): Virtex 6 LX240T ABB-1 ABB-3

Frank Lemke DPG Frühjahrstagung 2010 ABB Diagram and Measurement Control messages invoke the data flow from ROC via DCB to ABB Hit packets are first buffered in the FIFO Host software starts DMA to transfer the buffered hits to the host memory Measured bandwidth of 224 MB/s for one optical link

Frank Lemke DPG Frühjahrstagung 2010 Software Developments Driver under Linux 2.6 Performance emphasized Well-layered architecture ABBDaemon masks driver details to the users -Enables access from multiple users -Uses UNIX IPC mechanism Provided test programs -RAM memory -One-hole memory / FIFO -CTL messages

Frank Lemke DPG Frühjahrstagung 2010 Test Setup

Frank Lemke DPG Frühjahrstagung 2010 Conclusion & Outlook Prototype systems working and in use Virtex6 adaption and development for all boards ROC -Modular hardware (mother/daughterboard)‏ DCB -Protocol HDL optimizations ABB -Event Building - Flexible strategies for hit dispatching -Unified test program for more efficient debugging

Frank Lemke DPG Frühjahrstagung Questions ? Thank you for your attention !