Instrument Control System Seminar, 20 th -24 th October 2014 Time Synchronisation via Ethernet An introduction to IEEE 1588 Andreas Jost

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Time Synchronisation VLT Grandmaster

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 VLT Time Synchronisation Network NTP PTP ESO Time Bus NAP UT1 NTP PTP ESO Time bus Paranal Control building UT AT Instruments Telescopes/VLTI NAP UT2 NAP UT3 NAP UT4 NTP PTP ESO Time Bus

Instrument Control System Seminar, 20 th -24 th October 2014 IEEE1588 Time Sync

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 PTP Message Exchange PTPPrecision Time Protocol (Application Layer) UDPUser Datagram Protocol (Transport Layer) IPInternet Protocol (Network Layer) MACMedia Access Control PhyPhysical Layer

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Determination of Phase Change Rate (Drift) – one step

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Determination of Phase Change Rate (Drift) – two step

Instrument Control System Seminar, 20 th -24 th October 2014 Time Synchronisation

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Determination of Delay and Offset

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Time delay round trip Master  Slave Line delay round trip = 2x T ld

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 IEEE1588: Time Synchonisation First, one node (IEEE 1588 master clock) transmits a "Sync" telegram, which contains the estimated transmission time. The exact transmission time is captured by a clock and transmitted in a second "Follow Up" message. Based on the first and second telegram and by means of its own clock, the receiver can now calculate the time difference between its clock and the master clock. To achieve the best possible results, the IEEE 1588 time stamps should be generated in hardware or as close as possible to the hardware. The telegram propagation time is determined cyclically in a second transmission process between the slave and the master ("delay" telegrams). The slave can then correct its clock and adapt it to the current bus propagation time.

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Residence Time residence time transition delay Residence time - depends on traffic load - can be asymetric - cannot be compensated accurately by sync message Slave Grandmaster Clock Overall delay Sync message multicast

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Boundary Clocks in Network

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Transparent Clock

Instrument Control System Seminar, 20 th -24 th October 2014 TC accumulates residence time in correction field

Instrument Control System Seminar, 20 th -24 th October 2014 Transparent Clock Types The IEEE standard knows two types of transparent clocks, namely: End-to-End (E2E) & Peer-to-Peer (P2P) End-to-End TCs only measure the time taken for a PTP event message (those who get time stamped) to transit the bridge and provide this information to the receiving clocks in the correction field. No propagation delay of the link connected to the port is corrected by the E2E TC. E2E TCs use the delay request / delay response mechanism for the delay measurement whereby the Residence time of the delay request /delay response messages are corrected in the same way stated above. Peer-to-Peer TCs use the peer delay mechanism for the delay measurement. In addition to providing PTP event transit time information the P2P TC also provides corrections for the propagation delay of the link connected to the port receiving the PTP event message (correction field).

Instrument Control System Seminar, 20 th -24 th October 2014 Transparent Clock – End-to-End Delay Measurement Sync Stream e2e Delay Measurement

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 End to End Delay TC Ordinary Switch MC Permits the use of ordinary switches, routers etc, „residence time“ of ordinary network equipment is considered as transition time delay (  results in larger time error) Total time delay

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Transparent Clock – Peer-to-Peer Delay Measurement

Instrument Control System Seminar, 20 th -24 th October 2014 Transparent clock: Peer-to-Peer The peer delay mechanism measures the port to port propagation delay time between two directly connected ports sharing the same communication technology. The peer delay mechanism is independent of the state of a port (master or slave). It operates separately in both directions of the link. Clock A and Clock B perform the calculation of the peer-delay In case Clock B is two-step

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 P2P TC require that all network devices support IEEE1588 Ordinary switches would ignore the Pdelay message,  time sync would therefore fail due to lack of response P2P takes traffic load of Master clock, as the Pdelay request is exchanged only between neighbouring network hardware it allows to improve the time synchronisation by more frequent exchange of Pdelay messaging without increasing the overall network traffic. It supports well ring networks TC Peer-to-Peer

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 All links are periodically measured, so delay between the master and slave are already known when the network path changes. Note that peer-delay messages are exchanged even on ports blocked to prevent loops, such as by the Rapid Spanning Tree Protocol. There is no chance of Sync and Delay_Request messages taking different paths, since there are no Delay_Request messages. There is no need to worry about the master clocks ability to respond to Delay_Request messages when there are a lot of slaves, it only has to send the Sync and Follow_Up. Peer-to-Peer

Instrument Control System Seminar, 20 th -24 th October 2014 Cascading Boundary Clock Ring Network Master BC 6x Boundary Clock  6x Jitter of clock accumluated

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Jitter – BC vs TC Reference: Hirschmann Transparent Clock Boundary Clock Constant Jitter

Instrument Control System Seminar, 20 th -24 th October 2014 Use of Boundary Clock TC PEER-to-PEER TC End-to-End

Instrument Control System Seminar, 20 th -24 th October 2014 Boundary Clock – use at network points to change communication technology Boundary clocks are required wherever there is a change of the communication technology or other network elements block the propagation of the PTP messages. Furthermore it is recommended that a Boundary clock be used wherever there is a network component that inserts significant delay fluctuation. Boundary clocks have typically more than 2 ports, with one port serving as a PTP slave port to an upstream master clock, and the other ports serving as PTP clock masters to downstream PTP clocks. So with Boundary clocks you get time distribution trees.

Instrument Control System Seminar, 20 th -24 th October 2014 Boundary Clock vs Transparent Clocks The Boundary clocks (BC) defined in both versions of the IEEE1588 Standard. Standard v1 evidenced two problems of BC when used in (highly) cascaded networks. Namely, there is nonlinear decreasing synchronization accuracy and rising resynchronization time after network reconfiguration. To eliminate these effects the concept of transparent clocks (TC) has been introduced in the IEEE 1588 standard version 2. Transparent clocks were added in IEEE to correct the “residence time” of the network device like an Ethernet Switch. The residence time is accumulated in a field (correction field) of the SYNC (one-step) or FollowUP (two-step) message. Since transparent clocks are stateless they have no impact on there configuration time of e.g. ring topology networks

Instrument Control System Seminar, 20 th -24 th October 2014 Topology and „Best Master Clock“

Instrument Control System Seminar, 20 th -24 th October 2014 Limits of time synchronisation

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Test time Synchronisation 1 pps

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 IEEE 1588 accuracy at Slave Clock Reference: Siemens ESO TIME Bus < 10 us absolute to UTC ( relative time synchronisation between two slaves: < 1us, jitter < +/-200ns) rms, max jitter peaks can be up to several us depending on network setup

Instrument Control System Seminar, 20 th -24 th October 2014 PTP – common message header Peer-to-Peer End-to-End

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Calculate UTC from PTP timestamp PTP Announce message TAI = posix algortithm (PTP timestamp) UTC = posix algorithm (PTP timestamp – currentUtcOffset)  ISO 8601:2004

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Relationships between timescales

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 A IEEE 1588 network configures and segments itself automatically. For this, each node uses the "best master clock" algorithm (BMC) in order to determine the best clock in the segment. Every PTP clock stores its features within a specified dataset. These features are transmitted to other nodes within its "Sync" telegrams. Based on this, other nodes are able to synchronize their datasets with the features of the actual master and can adjust their clocks accordingly. Due to the cyclic running of the BMC, nodes can also be connected or removed during propagation time (hot plugging).. Configuration of a IEEE1588 Network

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 No distinction is made in the IEEE 1588 protocol between a software clock and a hardware clock. However, to be able to work with synchronism in the nanosecond range, hardware support is required. Generally, the synchronization errors – caused by software jitter – cannot be eliminated. With a pure software solution (e.g. Windows systems), the error may actually be in the micro- or milli-second range IEEE1588 Network – Hardware/Software

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Grandmaster Clock M900 M600 NI PXI GPS

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Slave PTP270PEX Bridge to EtherCAT Software implementation IEEE1588 v2 slave

Instrument Control System Seminar, 20 th -24 th October 2014 Transparent/Boundary clock CISCO Nexus

Instrument Control Systems Seminar 2014, 20 th -24 th October 2014 Application areas for IEEE1588