1. Synchronization Distributed System

2. Why synchronization? Two sharpshooters in a multiplayer online game kill the same target. Which one gets the points? To determine that process A send message m1 before or after m2 to process B Each machine in the distributed system assigned a local time using its own physical clock

3. Problem! A common ‘Resource’ Node A Node B request A request B A Distributed System Q. WHICH REQUEST WAS MADE FIRST? 3

4. Solution A Global Clock ?? Global Synchronization? Problem! A common ‘Resource’ Node A Node B request A request B A Distributed System Q. WHICH REQUEST WAS MADE FIRST? 4

5. Individual Clocks? Are individual clocks accurate, precise? One clock might run faster/slower? Problem! Q. WHICH REQUEST WAS MADE FIRST? A common ‘Resource’ Node A Node B request A request B A Distributed System Solution 10:00 AM 10:02 AM 5

6. Fundamental limits The notion of physical time is problematic in distributed systems: limitations in our ability to timestamp events at different nodes sufficiently accurately to know the order in which any pair of events occurred, or whether they occurred simultaneously.

7. Lack of Global Time in DS It is impossible to guarantee that physical clocks run at the same frequency Lack of global time, can cause problems Example: UNIX make –Edit output.c at a client –output.o is at a server (compile at server) –Client machine clock can be lagging behind the server machine clock

8.  A DS consists of a number of processes.  Each process has a state (values of variables).  Each process takes actions to change its state, or to communicate (send, receive).  An event is the occurrence of an action.  Events within a process can be ordered by the time of occurrence.  In DS we also need to know the time order of events on different processors & between different processes. Processes and Events

9. Clock Synchronization When each machine has its own clock, an event that occurred after another event may nevertheless be assigned an earlier time.

10. Physical Clocks (1) Computation of the mean solar day.

11. Physical Clocks How do we measure real time? –17th century - Mechanical clocks based on astronomical measurements Solar Day - Transit of the sun Solar Seconds - Solar Day/(3600*24) –Problem (1940) - Rotation of the earth varies (gets slower) –Mean solar second - average over many days

12. Physical Clocks (2) TAI seconds are of constant length, unlike solar seconds. Leap seconds are introduced when necessary to keep in phase with the sun.

13. Clock Synchronization Algorithms The relation between clock time and UTC when clocks tick at different rates.

14. Clock Drift: different rates of counting time –physical variations of underlying oscillators –variance with temperature –Even extremely small differences accumulate over a large number of oscillations leading to observable difference in the counters –drift rate: difference in reading bet. a clock and a nominal “perfect clock” per unit of time measured by the reference clock 10 -6 seconds/sec for quartz crystals 10 -7 - 10 -8 seconds/sec for high precision quartz crystals

15. Physical Clocks 1/86400 of a solar day International Atomic Time (TAI) = avg of cesium 133 clocks: (# ticks since midnight Jan 1, 1958) / 9192631 Universal Coordinated Time (UTC) = TAI  leap seconds (to adjust with solar time) Radio broadcast for UTC receivers WWV SW station in Colorado: - broadcast accurate to  1 msec - receive accurate to  10 msec Why can't we put UTC or GPS receivers on all our computers?

16. Accuracy of Computer Clocks Modern timer chips have a relative error of 1/100,000 - 0.86 seconds a day To maintain synchronized clocks –Can use UTC source (time server) to obtain current notion of time –Use solutions without UTC.

17. In DS each processor has its own timer. Timers drift from true real time at different rates. Maximum Drift Rate (MDR) of a clock depends on clock characterization and the environment. Max difference between two clocks with similar MDR is 2 * MDR Physical Clocks & Synchronization Max-Synch-Interval = (Max-Acceptable-Diff) / (MDR * 2)

18. Skew between computer clocks in a distributed system

19. Global Positioning System Computing a position in a two-dimensional space

20. Cristian’s Algorithm – External Synch External source S Denote clock value at process X by C(X) Periodically, a process P: 1.send message to S, requesting time 2.Receive message from S, containing time C(S) 3.Adjust C at P, should we set C(P) = C(S)? Reply takes time When P adjusts C(P) to C(S), C(S) > C(P)

21. Cristian's Algorithm Getting the current time from a time server.

22. Network Time Protocol (NTP) Most widely used physical clock synchronization protocol on the Internet (http://www.ntp.org)http://www.ntp.org –Currently used: NTP V3 and V4 10-20 million NTP servers and clients in the Internet Claimed Accuracy (Varies) –milliseconds on WANs, submilliseconds on LANs, submicroseconds using a precision timesource –Nanosecond NTP in progress

23. NTP Design Hierarchical tree of time servers. –The primary server at the root synchronizes with the UTC. –The next level contains secondary servers, which act as a backup to the primary server. –At the lowest level is the synchronization subnet which has the clients.

24. Network Time Protocol Figure 6-6. Getting the current time from a time server.

25. NTPs Offset Delay Estimation Method Source cannot accurately estimate local time on target –varying message delays NTP performs several trials and chooses trial with minimum delay –Let a = T1−T3 and b = T2−T4. –If differential delay is small, the clock offset Ɵ and roundtrip delay δ of B relative to A at time T4 are approximately given by Ɵ = (a + b)/2, δ = a − b T 4 T 3 T 2 T 1 Server B Server A Time mm' Time

26. Time servers: Christian’s algorithm m r m t p Time server,S T round := total round-trip time t := time value in message m t estimate := (t + T round /2) Receiver of UTC signals Time by S’s clock when reply msg arrives [t+min, t+T round -min] Accuracy:(T round /2 - min)

27. Limitations of Cristian’s algorithm Variability in estimate of T round –can be reduced by repeated requests to S & taking the minimum value of T round Single point of failure –group of synchronized time servers multicast request & use only 1st reply obtained Malicious interference –Protection by authentication techniques

28. The Berkeley algorithm Gusella & Zatti (1989) –Co-ordinator (master) periodically polls slaves estimates each slave’s local clock averages the values obtained (incl. its own clock value) ignores any occasional readings –Slaves are notified of the adjustment required This amount can be positive or negative Sending the updated current time would introduce further uncertainty, due to message transmit delay –Elimination of faulty clocks averaging over clocks that do not differ from one another more than a specified amount –Election of new master, in case of failure no guarantee for election to complete in bounded time

29. The Berkeley Algorithm a)The time daemon asks all the other machines for their clock values b)The machines answer c)The time daemon tells everyone how to adjust their clock

30. Uses an elected master to synchronize The elected master pools or broadcasts to all machines for their time, adjusts times received for RTT & latency, averages times, and tells each machine how to adjust. In some systems multiple time servers are used. Avg. time is more accurate, but still drifts Berkeley Algorithm

31. Lamport’s notion of logical time For many purposes, it is sufficient that all machines agree on the same time –… Emphasis on internal consistency If two processes do not interact, lack of synchronization will not be observable –… and thus will not cause problems Ordering of events is needed to avoid ambiguities

32. Logical Clocks For many DS algorithms, associating an event to an absolute real time is not essential, we only need to know an unambiguous order of events Lamport's timestamps Vector timestamps

33. Logical Clocks  We are interested in relative time between events (their order), define relation, Happens-Before (  ) :  On the same process: a  b, if time(a) < time(b)  If p1 sends m to p2: send(m)  receive(m)  If a  b and b  c then a  c

34. Logical Clocks Lamport Algorithm uses this relationship to provide a partial ordering of events: – All processes use a counter (clock) with initial value of zero – The counter is incremented by and assigned to each event, as its timestamp. – A send (message) event carries its timestamp – For a receive (message) event the counter is updated by Max(receiver-counter, message- timestamp) + 1

35. Lamport timestamps for the events ab cd bc df af

36. Lamport Timestamps 3 processes, each with its own clock. The clocks run at different rates. Lamport's algorithm corrects the clocks.

37. Vector Clocks (1) Concurrent message transmission using logical clocks.

38. Vector Clocks Mattern, 1989 & Fidge, 1991: –clock := vector of N numbers (one per process) –V i [i] := V i [i] + 1, before P i timestamps an event –Clock vector is piggybacked with messages –When P i receives : V i [j] := max{ t[j], V i [j] }, for j=1, …, N –V i [j], j i: #events that have occurred at P j and has a (potential) effect on P i –V i [i]: #events that P i has timestamped e e’ V(e) < V(e’)

39. Vector Logical Clocks Vector Logical time guarantees this: – All hosts use a vector of counters (logical clocks), i th element is the clock value for host i, initially all zero. – Each host i, increments the i th element of its vector upon an event, and assigns the vector to the event. – A send(message) event carries its vector timestamp (counter vector) For a receive(message) event, Max(V receiver [j], V messag [j]), if j is not self V receiver [j] + 1otherwise V receiver [j] =

40. Vector Timestamps

41. Example: Vector Logical Time p 1 p 2 p 3 p 4 0,0,0,0 Vector logical clock Message (vector timestamp) Physical Time 0,0,0,0 (1,0,0,0) 1,0,0,0 1,1,0,0 2,0,0,0 2,0,1,0 (2,0,0,0) 2,0,2,0 2,0,2,1 (2,0,2,0) 1,2,0,0 2,2,3,0 (1,2,0,0) 4,0,2,2 4,2,4,2 (4,0,2,2) 2,0,2,2 3,0,2,2 (2,0,2,2) 2,0,2,3 4,2,5,3 (2,0,2,3) n,m,p,q

42. Enforcing Causal Communication Figure 6-13. Enforcing causal communication.

43. QUESTIONS ? 43