1. CS 3471 CS 347: Parallel and Distributed Data Management Notes13: Time and Clocks

2. CS 3472 Reference: Leslie Lamport, Time, clocks, and the ordering of events in a distributed system, Communications of the ACM, Volume 21, Issue 7 (July 1978), pages: 558 - 565. Available at: http://portal.acm.org/citation.cfm?id=359563

3. CS 3473 Time and clocks Time and clocks are fundamental concepts in distributed systems e.g.:timeouts identifying calls, transactions,… setting priorities versions of data...

4. CS 3474 What is time? How can clocks be implemented? Questions

5. CS 3475 Ordering of events More basic than time: event ordering event ahappened event b 9 am before 10 am Do not need physical time to order events event ahappened before event b can affect

6. CS 3476 Ordering of events “  ” is a partial ordering [ total ordering: for any two events a,b (a  b) either a  b or b  a partial ordering: a,b can be concurrent ]

7. CS 3477 The model ProcessProcessProcess P Q R P4P4 P3P3 P2P2 P1P1 R4R4 R3R3 R2R2 R1R1 Q7Q7 Q6Q6 Q3Q3 Q2Q2 Q5Q5 Q1Q1 Q4Q4 TIMETIME

8. CS 3478 Events within a process are totally ordered Sending or receiving a message is an event No assumptions about message transmission times E.g.: P 1  P 2 P 1  Q 2 P 3, Q 3 concurrent

9. CS 3479 Definition The relation  on the set of events of a system is the smallest relation satisfying the following 3 conditions: 1) if a,b events in same process, and a comes before b, then a  b 2) if a is sending a message and b is receipt of same message by a different process, then a  b 3) if a  b and b  c, than a  c assume a  a if a  b and b  a, than a,b are concurrent

10. CS 34710 Logical Clocks ProcessProcess Process P QR P4P4 P3P3 P2P2 P1P1 R4R4 R3R3 R2R2 R1R1 Q7Q7 Q6Q6 Q3Q3 Q2Q2 Q5Q5 Q1Q1 Q4Q4 TIMETIME C R =9 C R =8 C R =7 C R =6 C R =5

11. Overview Logical Clocks Physical Clocks –basic properties –synchronization scheme –synchronization with clock server –probabilistic synchronization CS 34711

12. CS 34712 Logical Clock C i =logical clock (counter) at process i C[b] = reading of C j when event b occurs at process j Clock condition for any events a,b IF a  b THEN C[a] < C[b] No relationship to physical time

13. CS 34713 Clock Condition If a and b are events in process i and a comes before b, then C i [a] < C i [b] If a is the sending of a message by process i and b is the receipt of that message by process j, then C i [a] < C j [b] C1C1 C2C2 If a  b THEN C[a] < C[b] can be satisfied if the following conditions hold:

14. CS 34714 ProcessProcess Process P QR P4P4 P3P3 P2P2 P1P1 R4R4 R3R3 R2R2 R1R1 Q7Q7 Q6Q6 Q3Q3 Q2Q2 Q5Q5 Q1Q1 Q4Q4 TIMETIME C R =7 C R =5 C Q =10 7>5 by C1 10>7 by C2

15. CS 34715 To implement C1, C2: IR1 Each process i increments C i between any two successive events IR2 - Let a be event “process i sends message to j” - Message contains timestamp T m = C i [a] - When message arrives at j (1) IF T m > C j THEN C j  T m (2) event “arrival of message” takes place

16. CS 34716 Note: C i [a] < C j [b]  a  b Note: a,b concurrent  C i [a] = C j [b] Note: C i [a] = C j [b]  a,b concurrent

17. CS 34717 Ordering Events Totally (breaking ties) Example: A server wants to execute requests in order a is event that originated one request (at client) b is event that originates second request (at other client) If C[a] < C[b], service a first (it could be that a  b) If C[a] = C[b], pick one (a,b concurrent) e.g., pick one with lower [ node#, process id ]

18. CS 34718 Total Ordering Let “<” be a total ordering of the processes Define total ordering of events “  ” a  b (a event in process i; b in j) if and only if (1) C i [a] < C j [b] or (2) C i [a] = C j [b] and i < j “  ” not unique

19. CS 34719 Anomalous behavior C=100C=50 Reserve a TelephoneReserve a seat call seat server Client AClient B 1 2 3

20. CS 34720 Anomalous behavior C=100C=50 Reserve a TelephoneReserve a seat call seat server Client AClient B 1 2 3 C= 9am

21. CS 34721 Solutions (1) Include “telephone call” in “system” (2) Use perfect physical clocks (3) Use real physical clocks - may be “slightly off” - does not eliminate anomaly, but reduces its likelihood - useful for fault detection

22. CS 34722 Physical Clocks C i (t) = clock reading at process i at physical time t time=t C i (t)

23. CS 34723 time=t C i (t) C i + (x) C i - (x) C i - (x)=lim C i (x - |  |) C i + (x)=lim C i (x + |  |) 00 x Assume Ci(t) is a continuous, differentiable function except when clock is reset (message arrived) 00

24. CS 34724 How do we enforce clock condition? If a  b then C(a) < C(b)

25. CS 34725 Each process i increments C i between any two successive events For each process i, if i does not receive a message at time t, then C i is differentiable at t and dCi(t) dt IR1 IR1’ Logical clocks > 0 process i a b TIMETIME

26. CS 34726 Let a be event “process i sends message m to j” m contains timestamp T m = C i [a] when m arrives at j (1) IF T m > C j THEN C j  T m (2) event “arrival of m” takes place IR2 Logical clocks

27. CS 34727 Let a be event “process i sends message m to j” at physical time t m contains timestamp T m = C i (t) Let  m be minimum transmission delay for m m arrives at process j at physical time t’ t’ > t +  m IF Tm +  m > C j - (t’) THEN C j +  T m +  m After clock adjust, event “arrival of m” takes place IR2’ optional

28. CS 34728 i j t t’t’ m mm IF Tm +  m > C j - (t’) THEN C j +  T m +  m TIMETIME

29. CS 34729 Additional properties Clock drift There exists a constant k <<1 such that for all processes i dCi(t) dt - 1 < k PC1

30. CS 34730 For typical crystal controlled clocks, k < 10 -6 time CiCi xx+1 y y+1 k k y+1+k y+1-k dCi(t) dt - 1 < k

31. CS 34731 Clock synchronization for all i,j:| Ci(t) - Cj(t) | <  PC2

32. CS 34732 can be satisfied if a message is sent on every link at least every  seconds PC2 diameter = d d=4 (not 3)

33. CS 34733  let  m be the transmission time of message m  m = minimum delay + unpredictable delay  m =  m + Z m Assume constant  such that for all m: Z m   Assume that for all m:  m =  t 1 t2t2 maximum transit time:  +  shortest transit time: 

34. CS 34734 Theorem PC2 holds for all t > t o +  d with   d(2k  +  ) (assuming  +  <<  ).

35. CS 34735 Argument for d=1 slow fast sf Worst case scenario: t 1 : m leaves f for s with T m = C f (t 1 ) t 2 : m arrives at s: C s (t 2 ) = C f (t 1 ) +  C f (t 2 ) - C s (t 2 ) is as large as possible,  t 2 +  : no communication for  sec; f is as fast as possible, s as slow as possible. Just before communication at t 2 +  clocks are as far apart as possible, 

36. CS 34736 t 2 +  t2t2 f s kk kk    time clock reading So,  = 2  +  Now just need largest possible 

37. CS 34737 C f (t 1 ) t 1 C s (t 2 ) t 2 f s ++ C f (t 2 ) = C f (t 1 ) + (1+k)(  +  ) C s (t 2 ) = C f (t 1 ) +   = (1+k)(  +  ) -  max  = k(  +  ) + 

38. CS 34738  =2k  +  max  = k(  +  ) +  Thus:  = 2  +  + k(  +  ) Assuming that k<<1 and (  +  ) <<  we get:   2  + 

39. CS 34739 Summary Physical clocks: clock condition a  b  C(a) < C(b) drift < k | Ci(t) - Cj(t) | <  can be implemented as discrete

40. CS 34740 Uses for Physical Clocks: (1) To order events (2) Timeouts Example: “Reply to my request by time t 1 ” At time t 2 = (  +  +  )( 1+  ) + t 1 we can timeout my clock may drift

41. CS 34741 my node other t 1 on my clock  max trans  +  t 1 at remote clock t 2 = (  +  +  )( 1+  ) + t 1

42. CS 34742 Do physical clocks rule out anomalous behavior? No anomalous behavior if C s (t 2 )>C f (t 1 )  min f fast clock s slow clock message out of system C f (t 1 ) C s (t 2 ) C s (t 1 )=C f (t 1 ) - 

43. CS 34743 Worst possible scenario: Smallest possible transmission time  min C s, C f as far apart as possible C s (t 1 ) = C f (t 1 ) -  Smallest possible C s (t 2 ) = C f (t 1 ) -  + S.P.increment = C f (t 1 ) -  +  min (1-  ) No anomalous behavior if C s (t 2 )>C f (t 1 )

44. CS 34744 No A.B. if C s (t 2 ) > C f (t 1 ) I.e., if C f (t 1 ) -  +  min (1-  ) > C f (t 1 )  1 - k  min >

45. CS 34745 Using a clock service With IR2’, a fast clock speeds up all clocks Use instead a single, reliable clock service (eg. WWV), assume it is perfect “real time” (k=0) IR2’ is now: C j + (t ’ )  T m +  min whenever timing message arrives If max. transmission time from central clock is <  +  then  k    (1 + k  (Why? )

46. CS 34746 Using a clock service - analysis 0  error at synchronization time, node j   node j server Cj=+Cj=+ Cj=+Cj=+ ++++ ++  time slowest clock message fastest clock message

47. CS 34747 Using a clock service - analysis - cont.   elapsed time  server time 0  Node that got fastest message Node that got slowest message time  possible difference slow clock fast clock

48. CS 34748 Why can period be   ? ++  t t +  server j   fast slow

49. CS 34749   elapsed time  server time 0  Node that got fastest message Node that got slowest time    k(   kk worst case for clock reset

50. CS 34750  server time 0  Node that got fastest message Node that got slowest time  k(   kk worst case for clock reset |C s (t) - C j (t)|    |C k (t) - C j (t)|  2  

51. CS 34751 Setting clocks back C j (t) time timing msg arrives cannot use other solutions?

52. CS 34752 How can we reset a clock after a failure?

53. CS 34753 Probabilistic clock synchronization |C j (t) - C k (t)|  2k   k  CsCs k = drift  =update period  =max. transmission variability

54. CS 34754 Probabilistic clock synchronization |C j (t) - C k (t)|  2k   k  CsCs k = drift  =update period  =max. transmission variability   % of messages transmission time

55. CS 34755 Idea #1 When site j wants to synchronize, it requests time from server T 2  v C j  T+  +v/2 server time j assume k =0

56. CS 34756 Idea #1 When site j wants to synchronize, it requests time from server T 2  v C j  T+  +v/2 server time j  T+  +v T assume k =0

57. CS 34757 Idea #1 When site j wants to synchronize, it requests time from server T 2  v C j  T+  +v/2 server time j  T+  +v T T+  T assume k =0

58. CS 34758 Idea #1 When site j wants to synchronize, it requests time from server T 2  v C j  T+  +v/2 server time j  T+  +v T T+  T |C s (t) - C j (t)|  v/2 assume k =0

59. CS 34759 If we are lucky, v will be small and we get tight synchronization If we are unlucky, v is large and we lose!! |C s (t) - C j (t)|  v/2

60. CS 34760 Idea #2 (a) pick desired desired bound, D (b) repeat request until v/2  D (c) if more than q requests, we die!

61. CS 34761 area=p 22 2  D p.d.f of round trip delay Prob. 1 request fails = Pr[roundtrip time > 2   D]= p Prob. q requests fail = p q Prob. We can synchronize = 1 - p q  Choose q so that it is “very likely” that algorithm works...

62. CS 34762 If k = 0, we are done –step 1: all nodes synchronize –step 2: we stay synchronized forever If k > 0, synchronize every  seconds