1. 1 Chapter 11: Coordination and Agreement From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 3, © Addison-Wesley 2001 Presentation based on slides by Coulouris et al; modified by Jens B Jorgensen, University of Aarhus

2. 2 Failure model zAssumptions: Independent processes, reliable channels, no Byzantine errors. zFailure detector: yMay be unreliable (yield “Unsupected” or “Suspected”). yMay be reliable (yield “Unsuspected” or “Failed”).

3. 3 Distributed mutual exclusion zSituation: A number of processes want to access some shared resource. zProblem: Prevent interference, maintain consistency; critical section. zExamples: yShared files. yCar park monitoring.

4. 4 Distributed mutual exclusion – basics zApplication-level protocol: yenter() yresourceAccess() yexit() zCorrectness criteria: ySafety yLiveness y->-fairness zPerformance measures: yBandwith. yClient delay. yThroughput. ySynchronization delay.

5. 5 Distributed mutual exclusion – central server algorithm

6. 6 Distributed mutual exclusion – ring-based algorithm

7. 7 Distributed mutual exclusion – time-stamp based algorithm On initialization state := RELEASED; To enter the section state := WANTED; Multicast request to all processes;request processing deferred here T := request’s timestamp; Wait until (number of replies received = (N – 1)); state := HELD; On receipt of a request at p j (i ≠ j) if (state = HELD or (state = WANTED and (T, p j ) < (T i, p i ))) then queue request from p i without replying; else reply immediately to p i ; end if To exit the critical section state := RELEASED; reply to any queued requests;

8. 8 Distributed mutual exclusion – time-stamp based algorithm example p 3 34 Reply 34 41 34 p 1 p 2 Reply

9. 9 Elections zSituation: A unique process to play a particular role among a set of processes must be chosen. zProblem: All processes must agree on the choice. zExamples: yCentral server algorithm for mutual exclusion. yCoordinator process in Berkeley algorithm for internal clock synchronization.

10. 10 Elections – basics zProtocol: yA process may call an election. yAny process is either participant or non-participant in an election. yThe elected process should be chosen as the one with largest identifier. zCorrectness criteria: ySafety. yLiveness. zPerformance measures: yBandwidth. yTurn-around time.

11. 11 Elections – ring-based algorithm zN processes arranged in a ring; a coordinator must be elected; no failures occur. zInitially, each process is non-participant. zSome process pi sends an election message, elctn(i). zWhen a process pr receives a message elctn(i): yIf r<i then forward elctn(i); participant(pr) := true; endif; yIf (r>i and not(participant(pr))) then xforward (elctn(r)); participant(pr) := true; endif; yIf (r>i and participant(pr)) then skip (* do not forward *); endif; yIf r=i then participant(pr) := false; send(elctd(r)); endif; zWhen a process pr receives an elected message, elctd(c): yparticipant(pr) := false; elected(r) := c; y if r=c then skip (* do not forward *) else forward (elctd(c)); endif;

12. 12 Elections – ring-based algorithm example Note: The election was started by process 17. The highest process identifier encountered so far is 24. Participant processes are shown darkened

13. 13 Multicast communication zThe aim is for each of a group of processes to receive copies of the messages sent to the group. zA process issues only one multicast operation to send a message to a group of processes instead of issuing multiple send operations. zMulticast operations may provide: yDelivery guarantees. yEfficiency. yConvenience to programmers.

14. 14 Multicast communication – basics zFailure model: Reliable channels, processes may crash. zProcesses are member of groups (open or closed). zOperations: ymulticast(g,m). ydeliver(m).

15. 15 Multicast communication – basic multicast zB-multicast based on a reliable one-to-one send operation: yTo B-multicast(g,m), for each p in g, send(p,m). yOn receive(m) at p, B-deliver(m) at p. zMay be implemented using threads to perform the send operations concurrently. zMay suffer from the ack-implosion problem.

16. 16 Multicast communication – reliable multicast zIntegrity: A correct process p delivers a message m at most once. Furthermore p in group(m) and m was supplied to a multicast operation by sender(m). zValidity: If a correct process multicasts message m, then it will eventually deliver m (assumption: closed groups). zAgreement: If a correct process delivers message m, then all other correct processes in group(m) will eventually deliver(m).

17. 17 Multicast communication – reliable multicast algorithm

18. 18 Multicast communication – orderings; bulletin board example Bulletin board: os.interesting Item FromSubject 23A.HanlonMach 24G.JosephMicrokernels 25A.HanlonRe: Microkernels 26T.L’HeureuxRPC performance 27M.WalkerRe: Mach end

19. 19 Multicast communication – ordering relations zFIFO ordering: If a correct process issues multicast(g,m) and then multicast(g,m’), then every correct process that delivers m’ will deliver m before m’. zCausal ordering: If multicast(g,m) -> multicast(g,m’), then any correct process that delivers m’ will deliver m before m’. zTotal ordering: If a correct process delivers m before it delivers m’, then any other correct process that delivers m’ will deliver m before m’. zHybrid ordering relations exist.

20. 20 Multicast communication – ordering examples Notice the consistent ordering of totally ordered messages T 1 and T 2, the FIFO-related messages F 1 and F 2 and the causally related messages C 1 and C 3 – and the otherwise arbitrary delivery ordering of messages.

21. 21 Multicast communication – bulletin board example revisited Bulletin board: os.interesting Item FromSubject 23A.HanlonMach 24G.JosephMicrokernels 25A.HanlonRe: Microkernels 26T.L’HeureuxRPC performance 27M.WalkerRe: Mach end

22. 22 Summary zFailures and failure detection. zDistributed mutual exclusion: yCentral-server algorithm. yRing-based algorithm. yTime-stamp based algorithm. zElections: yRing-based algorithm. zMulticast communication yBasic multicast. yReliable multicast. yOrdering semantics.