1. 1 Causal Delivery Advanced Networks PhD. Saúl Pomares Hernández

2. 7-2 Clocks, events and process states r A distributed system is defined as a collection P of N processes p i, i = 1,2,… N r Each process p i has a state s i consisting of its variables (which it transforms as it executes) r Processes communicate only by messages (via a network) r Actions of processes: m Send, Receive, change own state r Event: the occurrence of a single action that a process carries out as it executes e.g. Send, Receive, change state r Events at a single process pi, can be placed in a total ordering denoted by the relation  i between the events. i.e. e  i e’ if and only if e occurs before e’ at p i r A history of process p i: is a series of events ordered by  i history(p i )= h i =

3. 7-3 Logical time and logical clocks Happened- before (Lamport 1978) r Instead of synchronizing clocks, event ordering can be used 1.If two events occurred at the same process p i (i = 1, 2, … N) then they occurred in the order observed by p i, that is     2.when a message, m is sent between two processes, send(m) happened before receive(m) 3.The happened before relation is transitive a  b (at p 1 ) c  d (at p 2 )b  c because of m 1 also d  f because of m 2 Not all events are related by  consider a and e (different processes and no chain of messages to relate them) they are not related by  ; they are said to be concurrent; write as a || e

4. 7-4 Lamport’s logical clocks r A logical clock is a monotonically increasing software counter. It need not relate to a physical clock. r Each process pi has a logical clock, Li which can be used to apply logical timestamps to events m LC1: Li is incremented by 1 before each event at process pi m LC2: ( a) when process pi sends message m, it piggybacks t = Li (b) when pj receives (m,t) it sets Lj := max(Lj, t) and applies LC1 before timestamping the event receive (m) e  e’ implies L(e)<L(e’) The converse is not true, that is L(e)<L(e') does not imply e  e’

5. 7-5 Vector clocks r Vector clock V i at process p i is an array of N integers r VC1:initially V i [j] = 0 for i, j = 1, 2, …N r VC2:before p i timestamps an event it sets V i [i] := V i [i] +1 r VC3:p i piggybacks t = V i on every message it sends r VC4:when p i receives (m,t) it sets V i [j] := max(V i [j], t[j]) j = 1, 2, …N ( then before next event adds 1 to own element using VC2) Note that e  e’ implies V(e)<V(e’). The converse is also true

6. 7-6 Causal Order deliver [BIR91] Causal Ordering: If send(m)  send(m’), then  k  g delivery k (m)  delivery k (m’) T i m e (0,0,0) m2m2 A B C (1,0,0) (1,1,0) (1,0,0) m1m1 The message m 2 arrives but can not be delivered Only after the delivery of m 1, The message m 2 also can be delivered. Initial vector Delivery condition if (VT(m’ )[i] = VT(p j )[i] +1 and VT(m’ )[k]  VT(p j )[k]  (k  i, k=1…n) then delivery(m)

7. 7-7 Causal Order deliver r The general algorithm of vector time for causal delivery is as follows: r Initially, VT(pi)[j] = 0  j=1…n. r For each event send(m) at pi, r VT(pi)[i] = VT(pi)[i] + 1. r Each multicast message by process pi is timestamped with the updated value of VT(pi). r For each event deliveredj(m’), pj modifies its vector time in the following manner: r VT(pj)[k]=VT(m’) For each reception receive(m’) à pj, i  j, m’=(i,VT(m’),message) r To enforce a causal delivery of m’ r i. Delivery condition r if not (VT(m’)[i] = VT(pj)[i] +1 and VT(m’)[k]  VT(pj)[k]  (k  i, k=1…n) r then r wait r else r ii delivery(m)

8. 7-8 Problem m1m1 m2m2 m3m3 m4m4 m5m5 t

9. 7-9 Causal Order deliver, Multigroup Case If send i (m,g)  send j (m’,g’), then  k  g  g’ delivery k (m)  delivery k (m’) p 1 p 2 p 3 g 1 g 3 g 2 t m2m2 m3m3 m1m1 Delivery of event m 3 must be delayed. p 2  g 1  g 2 (1,0,x) (x,0,0) (0,x,0) (1,0,x) (x,0,0) (1,x,0) (1,0,x) (x,0,1) (1,x,0) g 1 ={p 1, p 2 } g 2 ={p 2, p 3 } g 3 ={p 1, p 3 } m1m1 m2m2 m3m3 Delivery condition - VTa(m)[i] = VTa(pj)[i] +1 -  k : (pk  ga Λ k  i): VTa(m)[k] ≤VT(pj)[k] and -  g : (g  Gj): VTg(m) ≤VTg(pj) g3g3 g1g1 g2g2 p 1 p 3 p 2

10. 7-10 Exercise p 1 p 2 p 3 ch 1 ch 3 ch 2 e5e5 e4e4 e2e2 e3e3 e1e1 Delivery of event e 5 must be delayed. p 2  ch 1  ch 2 ch 1 ={?} ch 2 ={?} ch 3 ={?} ?...... ? ch 3 ch 1 ch 2 p 1 p 3 p 2 e1e1 e5e5

11. 7-11 The Basic Principles (cont.) The causal relation, denoted by  : 1.  x, a    y, b  if x=y  a < b 2.  x, a    y, b  if  x, a  is the sending of an event and  y, b  is the delivery of that event. 3.  x, a    y, b  if  z, c  | (  x, a    z, c    z, c    y, b  )

12. 7-12 Immediate Dependency Relation Immediate Dependency Relation  : e  e’  [ (e  e’)   e”  E,  (e  e”  e’)] The problem with causal ordering : The amount of control information emitted for large values of n = |G| is prohibitively high.

13. 7-13 Immediate Dependency Relation (cont.) Causal Intra-Channel Ordering: If send(e)  send(e’), then  k  c delivery k (e)  delivery k (e’) Proposition 1: If  e,e’  E send(e)  send(e’), then  k  c delivery k (e)  delivery k (e’)

14. 7-14 Serial Events Immediate Dependency Relation  : e  e’  [ (e  e’)   e”  E,  (e  e”  e’)] e 4 e 2 e 3 e ppp 4 p 5 t 1 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 e 2 e 2 e 4 e 4 e 3 e 3 p 1234 p 5 t e1e1 e2e2 e3e3 e4e4  e3  e4 e3  e4 IDR Graph Immediate Dependency

15. 7-15 Concurrent Events IDR Graph Immediate Dependency Relation  : e  e’  [ (e  e’)   e”  E,  (e  e”  e’)] e 5 e 2 e 3 e 2 e 3 e 5 S 1 S 1 S 1 S 6 S 1 S 8 e 1 S 1 S 2 S 1 S 3 S 1 S 4 S 1 S 7 S 1 S 5 e 4 p 1 p 2 p 3 p 4 p 5 e 6 t e5e5 e1e1 e2e2 e3e3 e4e4  e6e6 ( e 2 || ( e 3  e 4 ) )  e 6 Immediate Dependency

16. 7-16 Concurrent Relation || : e || e’   (e  e’  e’  e) Immediate Dependency Relation  : e  e’  [ (e  e’)   e”  E,  (e  e”  e’)]. Observation : (e’  e  e”  e)  e’ || e” e5e5 e1e1 e2e2 e3e3 e4e4  e6e6

17. 7-17 For Multi-Channel Case Immediate Inter-Channel Dependency Relation  : (e,c)  (e’,c’)  [((e,c)  (e’, c’))  (e”, c’’)  E, ((e,c)  (e”, c’’)  (e’, c’)  c’’  c  c’’  c’)] Observation: If only one channel exists in the system, then  = 

18. 7-18 Causal Inter-Channel Ordering: If send(e,c)  send(e’,c’), then  k  c  c’ delivery k (e)  delivery k (e’) Proposition 2: If  e,e’  E send(e,c)  send(e’,c’), then  k  c  c’ delivery k (e)  delivery k (e’)

19. 7-19 Inter-channel Dependency Immediate Inter-Channel Dependency Relation  : (e,c)  (e’,c’)  [((e,c)  (e’, c’))  (e”, c’’)  E, ((e,c)  (e”, c’’)  (e’, c’)  c’’  c  c’’  c’)] Proposition 2: If  e,e’  E send(e,c)  send(e’,c’), then  k  c  c’ delivery k (e)  delivery k (e’) ((e 1,ch 1 )  (e 2,ch 3 ))  (e 3,ch 2 ) Events with IICDR to e 3 p 1 p 2 p 3 ch 1 ch 3 ch 2 t e2e2 e3e3 e1e1 Delivery of event e 3 must be delayed. p 2  ch 1  ch 2

20. 7-20 The Multi-Channel Causal Algorithm

21. 7-21 The Multi-Channel Causal Algorithm

22. 7-22 The Multi-Channel Causal Algorithm

23. 7-23 The Multi-Channel Causal Algorithm

24. 7-24 The Multi-Channel Causal Algorithm

25. 7-25 Implementation The MCP General Structure Channel Session Consistent Session Consistent Session Participant Causal Ordering Channel * 1 * * * 1 1 * * * * * 1 1 * 1 JSDT

26. 7-26 Membership Membership service The rest of Participants Participant p k req_join(ch, p k ) serv_join(ch,p k,n p) init_join(p k, p i,VT(pi)[i]) Wait for serv_join  Wait for np-1 messages init_join.  Actualization of its VT join(ch, p k ) Memory reservation for a new participant p k Only after the reception of join, we consider p k like a member of channel ch The only no causal message Join Procedure

27. 7-27 Membership Membership service The rest of Participants Participant p k Leave Procedure req_leave(ch, p k ) Wait for serv_leave leave(ch, p k ) leaving notification of p k serv_leave(ch,p k ) Only after the reception of leave, on efface toute information concernant p k

28. 7-28 Implementation (cont.) The MCP Architecture Network Cooperative Distributed Engineering System g channels 1 2... Multi-Channel Coordination Protocol Java Shared Data Toolkit Light Reliable Multicast Protocol