1. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Chapter 6 Synchronization (2) DISTRIBUTED SYSTEMS (dDist) 2014

2. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Plan Clock synchronization in distributed systems –Physical clocks –Logical clocks Ordered multicasting Mutual exclusion Election

3. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion We often want to ensure that only one process accesses a critical region Requirements: ensure –Mutually exclusive access –Deadlock freedom –Starvation freedom –Fairness Types of algorithms –Permission-based –Token-based

4. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Centralized Algorithm (1/2) Simulate how mutual exclusion is done in a one-processor system –Let one process act as a coordinator –Send request to coordinator to ask for permission

5. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Centralized Algorithm (2/2) Mutually exclusive access? –Coordinator only lets one process in Deadlock freedom? –If no process is in critical region, any other that wants to can be granted Starvation freedom? –No process can end up waiting forever for resource Fairness? –Access granted in order of receipt at coordinator Only 3 messages per resource use –But coordinator is single point of failure and potential bottleneck

6. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Decentralized Algorithm (1/2) Sigma m,n : protocol based on DHTs –m and n are parameters and m > n/2 –Logically replicate resource to n coordinators Name ith replica ”r_name-i” and let node with id lookup(H(r_name-i)) be responsible for it We assume they are all different –To enter into critical region Get grant from m coordinators If cannot get enough grants, give up and retry later –Problem: failure/recovery of coordinator May handle up to f failures where – Probability of violating mutual exclusion is small

7. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Decentralized Algorithm (2/2) Mutual exclusion –Node forgets previous grants every 10000 seconds –Probability of breaking exclusivity in one attempt

8. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Distributed Algorithm (1/3) Totally ordered multicast of requests for resources to timestamp and order requests 1.Receiver did not send request: Send OK back 2.Receiver sent request, is accessing resource: Wait and queue request 3.Receiver sent request, is not accessing resource Receiver has higher timestamp on its request: Send OK back Receiver has lower timestamp on its request: Wait and queue request When requester has OK from all processes it accesses resource –When done, send OK to all processes in queue

9. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Distributed Algorithm (2/3)

10. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Distributed Algorithm (3/3) Guarantees –Mutual exclusion? –Deadlock freedom? –Starvation freedom? –Fairness? PROBLEM: –n-points-of-failure (instead of 1) Could try to detect whether node has failed –n bottlenecks (instead of 1) Could try to just get OK from majority

11. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Token Ring Algorithm (1/2) Assume processes can be connected in a (logical) ring –Token circulates on the ring –Take the token if need to access critical region –If token not needed, send it to next neighbor

12. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Mutual Exclusion: A Token Ring Algorithm (2/2) Guarantees –Only one token at one process in a given time Mutual exclusion? Deadlock freedom? Starvation freedom? Fairness? But what if token is lost? –Need to detect that it is lost and reintroduce it –Must be done without introducing more than one –An interesting problem in itself!

13. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Election (1/2) There is often a need for selecting a process with some special role –E.g., choose coordinator in a centralized protocol –Where to reintroduce a lost token If processes have unique id –Try to find process with highest id, make this leader –Algorithms differ in how they locate process with highest id If processes do not have unique id (or similar) –Give them unique ids or we can’t distinguish them

14. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Election (2/2) Since Election protocols are often used to get robustness, like electing a new coordinator, they should themselves be robust Robust := –They cannot assume that all processes are alive –They cannot assume that only one election is running at a time Why?

15. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Bully Algorithm (1/3) Assume that every process knows the ids of every other process –Static group, but some members may have failed P notices that there is no coordinator – P wants to hold an election –P sends an ELECTION message to all processes with higher identifiers –If no one responds, P wins the election and becomes coordinator (everyone higher is dead) –If one of the higher-ups answers, it takes over – P’s job is done (someone higher is alive)

16. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Bully Algorithm (2/3) Figure 6-20. The bully election algorithm. (a) Process 4 holds an election. (b) Processes 5 and 6 respond, telling 4 to stop. (c) Now 5 and 6 each hold an election.

17. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 The Bully Algorithm (3/3) Figure 6-20. The bully election algorithm. (d) Process 6 tells 5 to stop. (e) Process 6 wins and tells everyone.

18. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 A Ring Algorithm Build view of processes by circulating message –Two steps: ELECTION and COORDINATOR –ELECTION message constructs view of alive processes during circulation –Then circulate the view, and all pick highest candidate as coordinator

19. Elections in Wireless Environments (1/4) Limited range of wireless –Each node only connected to a few other nodes May want to elect node with particular properties –High battery-level –Near the “middle” of the network –… Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

20. Elections in Wireless Environments (2/4) Figure 6-22. Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)–(e) The build-tree phase

21. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Elections in Wireless Environments (3/4) Figure 6-22. Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)–(e) The build-tree phase

22. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Elections in Wireless Environments (4/4) Figure 6-22. (e) The build-tree phase. (f) Reporting of best node to source.

23. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Elections in Large-Scale Systems (1/2) Requirements for superpeer selection: 1.Normal nodes should have low-latency access to superpeers. 2.Superpeers should be evenly distributed across the overlay network. 3.There should be a predefined portion of superpeers relative to the total number of nodes in the overlay network. 4.Each superpeer should not need to serve more than a fixed number of normal nodes.

24. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Elections in Large-Scale Systems (2/2) Simple example of a protocol Assume we want L leaders in m-bit Chord DHT –Use the most significant bits –The leader for p is lookup(p & 11…11000) Each super-peer is then responsible for an expected nodes k

25. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Summary Mutual exclusion –Centralized –Decentralized –Distributed –Token-based Election –Bully –Ring –Decentral and large-scale