1. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Chapter 7 Consistency And Replication (2) DISTRIBUTED SYSTEMS (dDist) 2014

2. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Plan Motivation Consistency models –Data-centric consistency –Client-centric consistency Replica management Consistency protocols

3. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Consistency Protocols Describe algorithm for achieving a given consistency model –Data-centric –Client-Centric

4. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Continuous Consistency Bounding deviations so as to avoid –Numerical errors E.g., that deviation of replicas’ stock quote is more than 1.00$ –Staleness errors E.g., that last stock quote is more than 1 hour old –Order errors E.g., that more than 3 updates to stock quote are outstanding

5. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Numerical Deviation (1/5) Assume (to get a toy example to study) –N servers, S 1 … S N –Single data item x –A write, W(x), has a weight, weight(W), that is the amount that x changes E.g., x = 1; x := x * 3   weight is 2 –weight(w) > 0 We have a limit on numerical deviation for the value of x, v i, at server S i W(x) x1 x2 x3 origin(W) W(x) L1 L2 L3 Log of writes performed E.g., multicast or epidemic protocol

6. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Numerical Deviation (2/5) Define I.e., sum of writes executed by S i originating from S j –Note, TW[i,i] are all the values submitted by S i

7. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Numerical Deviation (3/5) Define Goal –Ensure Consistent value of x Current value of x at S i

8. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Numerical Deviation (4/5) Assume –When forwarding the write, S i also forwards TW[i,k] –S k maintains this as a view, TW k [i,k], of what S i has seen from k –We have 0 ≤ TW k [i,k] ≤ TW[i,k] If then S k forwards writes from its log, L k –S k waits to commit new writes until these are commited at S i Guarantees that deviation is bounded as wanted…

9. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Numerical Deviation (5/5) TW k [i,k] ≤ TW[i,k] for all i TW k [i,k] ≤ TW[i,k] for all i S i ’s view of itself is up-to-date TW[k,k]-TW k [i,k] ≤ ∂ i /(N-1) for all k TW[k,k]-TW k [i,k] ≤ ∂ i /(N-1) for all k

10. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Staleness Deviations (1/2) Goal –Ensure that there are no W(x) older than TO i that S i has not seen If we know drift rate of clocks and communication delay, we can synchronize within a bound

11. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Staleness Deviations (2/2) Keep vector of time stamps –VT k [i] = Local time of last write from S i If T(k) – VT k [i] > Limit, pull updates from S i (and update VT k [i])

12. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Bounding Ordering Deviations Check size of local queue of tentative updates –Tentative = Applied but not committed If too large, agree on ordering of tentative updates before proceeding… –Ensuring consistent ordering

13. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Ensuring Consistent Ordering Make a specific process responsible for specific data item –Primary-Based Protocols Enable multiple processes to carry out write operations –Replicated-Write Protocols

14. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Remote-Write Protocols Figure 7-20. The principle of a primary-backup protocol. Can also be done non-blocking (W5 before W2) W5

15. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Local-Write Protocols Figure 7-21. Primary-backup protocol in which the primary migrates to the process wanting to perform an update. Efficient for multiple writes from same client when non-blocking

16. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Replicated-Write: Active Replication Multicast updates to all replicas –Sequencer –Totally ordered multi-cast Inefficient compared to primary-backup if there are many more reads than write Efficient compared to primary-backup if there are many more writes than reads

17. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Replicated-Write: Quorum Process group of size N Define –write quorum size, N W At least N W processes need to agree to let us write –read quorum size, N R At least N R processes need to agree to let us read To make sure to read latest version –N W + N W > N –N W + N R > N No two processes can get permission to write At least one process will have latest version

18. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Replicated-Write: Quorum Figure 7-22. Three examples of the voting algorithm. (a) A correct choice of read and write set. (b) A choice that may lead to write-write conflicts. (c) A correct choice, known as ROWA (read one, write all). To make sure to read latest version N W + N W > N N W + N R > N To make sure to read latest version N W + N W > N N W + N R > N

19. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Client-Centric Consistency (1/4) Each client keeps track of one or both of: –Write set: identifiers of writes performed by the client itself –Read set: identifiers of the writes that are relevant to the read operations performed by the client –Identifiers of all writes which had taken place when the prior reads took place

20. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Client-Centric Consistency (2/4)

21. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Client-Centric Consistency (3/4) Monotonic-write consistency –On write: Client hands in write set to server Server may need to communicate with other servers to become up-to-date Client is given identifier of the new write Read-your-writes consistency –Read Client hands in write set to server Server may need to communicate with other servers to become up-to-date Server returns updated write set The other two are similar…

22. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Client-Centric Consistency (4/4) Bound size of write and read sets by grouping operations into sessions –E.g., clear sets when a client exits a specific application Each operation gets a Lamport timestamp –Server S i keeps vector WVC i WVC i [j]: timestamp of the latest write operation from S j that S i has processed –Client A keeps vector SVC SVC A [j]: timestamp of the latest write operation initiated at S j A does not use S i until WVC i   SVC A

23. Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5 Summary Various protocols for ensuring consistency Data-centric consistency –Primary-based protocols –Replicated-write Primary-based protocols most widely used due to simplicity –Exact selection of protocol type depends on rate at which request are processed and on message delay Client-centric consistency may be efficiently implemented using vectors of Lamport timestamps