Presentation is loading. Please wait.

Presentation is loading. Please wait.

17 January 2012Dietrich Beck, EE(BEL) Towards the FAIR Timing System using White Rabbits Towards FAIR White Rabbit Timing System.

Similar presentations


Presentation on theme: "17 January 2012Dietrich Beck, EE(BEL) Towards the FAIR Timing System using White Rabbits Towards FAIR White Rabbit Timing System."— Presentation transcript:

1 17 January 2012Dietrich Beck, EE(BEL) Towards the FAIR Timing System using White Rabbits Towards FAIR White Rabbit Timing System

2 17 January 2012Dietrich Beck, EE(BEL) Acknowledgements GSI Timing Team: Marcus Zweig, Stefan Rauch, Mathias Kreider, Cesar Prados, Wesley Terpstra, Ralph Bär, Dietrich Beck former members: Tibor Fleck, Sergio Mauro CERN Timing Team: Tomasz Włostowski, Javier Serrano, Maciej Lipinski, Evangelia Gousiou, Erik van der Bij, Jean-Claude Bau, Pablo Alvarez GSI/EE: Jan Hoffmann, Nikolaus Kurz, Holger Brand, …

3 17 January 2012Dietrich Beck, EE(BEL) Towards FAIR…

4 17 January 2012Dietrich Beck, EE(BEL) 23 November 2011

5 17 January 2012Dietrich Beck, EE(BEL) 8 December 2011

6 17 January 2012Dietrich Beck, EE(BEL) Bild von heute 12 January 2012

7 17 January 2012Dietrich Beck, EE(BEL) 12 January 2012

8 17 January 2012Dietrich Beck, EE(BEL) General Machine Timing System @ FAIR parallel execution of beam production chains cycles: 20ms to hours trigger and sync. equipment actions 1 µs precision in 99% of all cases few ns precision for kickers (ps for rf-systems: BuTiS) many rings > 2000 devices connected to timing system large distances robustness: lose at most one message per year

9 17 January 2012Dietrich Beck, EE(BEL) Idea: Timing System Based on Time Not: 1km distance:  5  s propagation time 1  s precision: requires compensation for cable length  Timing Master Equipment 1 do this Equipment 2 do this

10 17 January 2012Dietrich Beck, EE(BEL) Idea: Timing System Based on Time Instead: equipment pre-programmed for autonomous action at a given time pre-programming via timing-events required: clock synchronization  ns required  distribution of information and execution of action are decoupled !!! timing-events must be sent “early enough”timing-events must be sent “early enough” upper bound latency for transmission (e.g. 100  s): real-time!upper bound latency for transmission (e.g. 100  s): real-time! lossless transmission: robustness!lossless transmission: robustness! Timing Master Equipment 1 do this @ 15:03 Equipment 2 do this @ 15:03 do that @ 15:01 "Oh dear! Oh dear! I shall be too late!" ???

11 17 January 2012Dietrich Beck, EE(BEL) White Rabbit Cooking Recipe I: Clocks network: Gigabit-Ethernet PTP (Precision Time Protocol) IEEE1588-2008 with value t 1.. t 4 one can calculate one-way link delay syntonize clock rate by tracking the value calculate clock offset

12 17 January 2012Dietrich Beck, EE(BEL) White Rabbit Cooking Recipe I: Clocks network: Gigabit-Ethernet PTP (Precision Time Protocol) IEEE1588-2008 –free-running oscillators on network nodes –need to re-sync often: lot’s of traffic, bad for determinism –only 1-100  s synchronization of clocks SyncE (Synchronous Ethernet) –receiver’s clock recovered from 125MHz carrier –8ns precision precise phase measurement and adjustment clock synchronization with sub-ns precision and low-ps jitter

13 17 January 2012Dietrich Beck, EE(BEL) White Rabbit Cooking Recipe II: Network System Timing Master multi-layered WR Switches WR Nodes GPS Clock propagation: Determinism –“timing events shall arrive early enough” –upper bound latency from timing master to nodes –no handshake: use UDP or raw Ethernet (no TCP) –switches implement cut-through for high-priority messages Robustness –bit errors: Hamming code –packet loss: Reed-Solomon code Redundancy –redundant links between switches –link aggregation (in combination with Reed-Solomon)

14 17 January 2012Dietrich Beck, EE(BEL) transceiver FPGA VCO 20MHz VCO 125MHz WR clock!!! WR Cooking Recipe III: Implementation… path asymmetry (two cables) impacts phase compensation  White Rabbit PTP –one bidirectional single-mode optical fiber –different wave lengths for TX/RX: speed differs slightly (dispersion) –link-delay-model relative delay coefficient α depends only on the type of fiber (off-line) fixed delays are measured on-line White Rabbit node host bus interface

15 17 January 2012Dietrich Beck, EE(BEL) Inside the FPGA: within FPGA: Wishbone “System-on-Chip” bus for interconnection of IP cores FPGA

16 17 January 2012Dietrich Beck, EE(BEL) Transmission of Timing-Events: Etherbone idea: Connect distant Wishbone buses via a network protocol layer. Ethernet + Wishbone = Etherbone direct memory access to devices via White Rabbit network used for timing-events

17 17 January 2012Dietrich Beck, EE(BEL) The White Rabbit Project development in the frame of CERN’s renovation and the FAIR projects collaboration: CERN, GSI, …, industry partners Open HardWare Repository: www.ohwr.orgwww.ohwr.org –Open Source: Software and HDL –“CERN Open Hardware License”: Hardware commercial production and support “The White Rabbit (WR) project is a multi-laboratory and multi-company effort to bring together the best of the data transfer and timing worlds …” (Proc. ICALEPCS 2009)

18 17 January 2012Dietrich Beck, EE(BEL) Timing Systems @ FAIR General Machine Timing System based on White Rabbit fairly cheap: € 600,- SPEC board sub-ns synchronization distribution of (clock) time-stamps timing-events BEL group one global timing master clock derived from BuTiS Talk T. Fleck, FAIR Technikforum Bunch phase Timing System (BuTiS) invest € 15k per receiver station ps precision (100ps/km accuracy) distribution of clocks!!! NO time-stamps at FAIR NO timing-events HF group one global BuTiS center Talk P. Moritz, FAIR Technikforum Remark: If you are interested time-stamps, you need White Rabbit. If you are interested in the most accurate clocks, you need BuTiS. If you are interested in both, you need both.

19 17 January 2012Dietrich Beck, EE(BEL) 1.settings management (via normal network) a.preloads set-values to the FECs (set-value can also be a ramp…) b.provides schedule to timing-master 2.according to schedule, timing-master broadcasts timing-events to FECs (timing-events: time of execution and reference to a set-value) 3.timing-receiver of FEC: timely triggering of FEC, FEC applies set-value The FAIR Control System (slightly simplified ) 1b: Data supply: schedules, alternatives 1a: Data supply: set-values 3. timing receiver 2: broadcast timing-events

20 17 January 2012Dietrich Beck, EE(BEL) The FAIR Control System (slightly simplified ) Data supply: schedules, alternatives Data supply: set values timing receiver Analogy to performing a classical piece of music: composer: settings management conductor: timing master musicians: FECs move of conductor's baton: timing-events - one timing-event may trigger multiple FECs

21 17 January 2012Dietrich Beck, EE(BEL) Timing Master Control System (application layer) LSA (settings management) API Equipment 1Equipment N FESA WR Equipment 2Equipment 3 Beam Interlock SystemPost Mortem System API BuTiS another link Equipment 4 a link Equipment.. Common Systems Alarm, Archive, Logging, Oscilloscope API BuTiS Disclaimer: This is my personal view and may be incorrect and not up-to-date

22 17 January 2012Dietrich Beck, EE(BEL) BuTiS and WR

23 17 January 2012Dietrich Beck, EE(BEL) What the Timing System is About campus wide distribution system for –timing-events –time-stamps –other machine or beam related telegrams not limited to accelerator controls available to “everyone” typically, data is propagated top-down other traffic only in exceptional cases –kicker control –bunch-to-bucket transfer –beam interlock notification

24 17 January 2012Dietrich Beck, EE(BEL) What the Timing System is NOT About general purpose communication transmission of bulk data personnel protection system things not related to machine timing Exceptions only in well justified cases FECs not connected to FESA Info-Service part of machine protection system

25 17 January 2012Dietrich Beck, EE(BEL) Next Steps… @GSI: Set-up a first network 2012 usage in a field environment investigation of stability and long-term effects connection between new and existing timing system first tests with existing machines at GSI

26 17 January 2012Dietrich Beck, EE(BEL) …Next Steps @GSI/CERN: WR starter kit (timing system demo) spring 2012 @Saclay: Low energy part of p-linac: first WR based timing system provided by GSI (early 2013) @CERN: Installation of WR based timing system at AD ring (2013) Installation of WR link between CERN and Gran Sasso (2012)

27 17 January 2012Dietrich Beck, EE(BEL) Status Slovenia: In-Kind contribution timing receivers. Receivers and FPGA building blocks for FAIR timing system specified. Germany/GSI: In-Kind contribution General Machine Timing System: specification until March 2012. White Rabbit PTP link between two nodes established sub-ns synchronization with a jitter of  hundred ps propagation of WR PTP over three layers of switches FPGA building blocks for simple timing system exists integration work in progress planning of WR network cabling in progress First WR nodes developed by EE

28 17 January 2012Dietrich Beck, EE(BEL) Further reading www.ohwr.org https://www- acc.gsi.de/wiki/Timing/TimingSystemDocumentation (search for “FAIR timing system”)https://www- acc.gsi.de/wiki/Timing/TimingSystemDocumentation

29 17 January 2012Dietrich Beck, EE(BEL) Moersbacher grund view towards SIS 100, 23 November 2011


Download ppt "17 January 2012Dietrich Beck, EE(BEL) Towards the FAIR Timing System using White Rabbits Towards FAIR White Rabbit Timing System."

Similar presentations


Ads by Google