1. Consistency and Replication Chapter 6 Presenter: Yang Jie RTMM Lab Kyung Hee University

2. Content 1.Definition of Consistency and Replication 2.Introduction Reasons for Replication & Problems of Replication 3.Object Replication 4.Data-centric models

3. Consistency and Replication Definition :  Replication:  Replication of data  Consistency:  Consistency of replicated data Reason for Consistency:  Keep replicas to be the same

4. Consistency and Replication Introduction:  Reasons for Replication:  Replication  Performance  Problem of Replication:  Consistency Problem Whenever a replica is updated, that replica becomes different from the others.  Synchronization Problem

5. Object Replication (1)  Purpose: managing data in distributed system  Object Replication:  Consider objects instead of data alone  Benefit of encapsulating and operating data

6. Object Replication (2) Organization of a distributed remote object shared by two different clients.

7. Object Replication (3) A remote object capable of handling concurrent invocations on its own. A remote object for which an object adapter is required to handle concurrent invocations

8. Object Replication (4) A distributed system for replication-aware distributed objects. A distributed system responsible for replica management

9. Consistency Models Introduction:  Consistency Model: A contract between processes and the data store.  Two kind of Models:  Data-centric Consistency Models  Guarantee for a number of processes  Simultaneously update  Sequential consistency  Client-centric Consistency Models  Guarantee for a single processes  Lack simultaneous updates

10. Data-centric Consistency Models The general organization of a logical data store, physically distributed and replicated across multiple processes.

11. Data-centric Consistency Models  Data-centric Consistency Models  Strict Consistency  Linearizability and Sequential Consistency  Causal Consistency  FIFO Consistency  Weak Consistency  Release Consistency  Entry Consistency

12. Strict Consistency Behavior of two processes, operating on the same data item. A strictly consistent store. A store that is not strictly consistent.

13. Linearizability and Sequential Consistency (1) A sequentially consistent data store. A data store that is not sequentially consistent.

14. Linearizability and Sequential Consistency (2) Three concurrently executing processes Process P1Process P2Process P3 x = 1; print ( y, z); y = 1; print (x, z); z = 1; print (x, y);

15. Linearizability and Sequential Consistency (3) Four valid execution sequences for the processes of the previous slide. The vertical axis is time. x = 1; print ((y, z); y = 1; print (x, z); z = 1; print (x, y); Prints: 001011 Signature: 001011 (a) x = 1; y = 1; print (x,z); print(y, z); z = 1; print (x, y); Prints: 101011 Signature: 101011 (b) y = 1; z = 1; print (x, y); print (x, z); x = 1; print (y, z); Prints: 010111 Signature: 110101 (c) y = 1; x = 1; z = 1; print (x, z); print (y, z); print (x, y); Prints: 111111 Signature: 111111 (d)

16. Causal Consistency (1) Necessary condition: Writes that are potentially casually related must be seen by all processes in the same order. Concurrent writes may be seen in a different order on different machines.

17. Causal Consistency (2) This sequence is allowed with a casually-consistent store, but not with sequentially or strictly consistent store.

18. Causal Consistency (3) a)A violation of a casually-consistent store. b)A correct sequence of events in a casually-consistent store.

19. FIFO Consistency (1) Necessary Condition: Writes done by a single process are seen by all other processes in the order in which they were issued, but writes from different processes may be seen in a different order by different processes.

20. FIFO Consistency (2) A valid sequence of events for FIFO consistency

21. FIFO Consistency (3) Statement execution as seen by the three processes from the previous slide. The statements in bold are the ones that generate the output shown. x = 1; print (y, z); y = 1; print(x, z); z = 1; print (x, y); Prints: 00 (a) x = 1; y = 1; print(x, z); print ( y, z); z = 1; print (x, y); Prints: 10 (b) y = 1; print (x, z); z = 1; print (x, y); x = 1; print (y, z); Prints: 01 (c)

22. FIFO Consistency (4) Two concurrent processes Process P1Process P2 x = 1; if (y == 0) kill (P2); y = 1; if (x == 0) kill (P1);

23. Weak Consistency (1) Properties: Accesses to synchronization variables associated with a data store are sequentially consistent No operation on a synchronization variable is allowed to be performed until all previous writes have been completed everywhere No read or write operation on data items are allowed to be performed until all previous operations to synchronization variables have been performed.

24. Weak Consistency (2) a)A valid sequence of events for weak consistency. b)An invalid sequence for weak consistency.

25. Release Consistency (1) Rules: Before a read or write operation on shared data is performed, all previous acquires done by the process must have completed successfully. Before a release is allowed to be performed, all previous reads and writes by the process must have completed Accesses to synchronization variables are FIFO consistent (sequential consistency is not required).

26. Release Consistency (2) A valid event sequence for release consistency.

27. Entry Consistency (1) Conditions: An acquire access of a synchronization variable is not allowed to perform with respect to a process until all updates to the guarded shared data have been performed with respect to that process. Before an exclusive mode access to a synchronization variable by a process is allowed to perform with respect to that process, no other process may hold the synchronization variable, not even in nonexclusive mode. After an exclusive mode access to a synchronization variable has been performed, any other process's next nonexclusive mode access to that synchronization variable may not be performed until it has performed with respect to that variable's owner.

28. Entry Consistency (2) A valid event sequence for entry consistency

29. Summary of Consistency Models (1) a)Consistency models not using synchronization operations. b)Models with synchronization operations. ConsistencyDescription StrictAbsolute time ordering of all shared accesses matters. Linearizability All processes must see all shared accesses in the same order. Accesses are furthermore ordered according to a (nonunique) global timestamp Sequential All processes see all shared accesses in the same order. Accesses are not ordered in time CausalAll processes see causally-related shared accesses in the same order. FIFO All processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order (a) ConsistencyDescription WeakShared data can be counted on to be consistent only after a synchronization is done ReleaseShared data are made consistent when a critical region is exited EntryShared data pertaining to a critical region are made consistent when a critical region is entered. (b)

30. Summary of Consistency Models (2) ConsistencyAdvantageDisadvantage StrictIdeal modelRely on absolute global time Linearizability and SequentialEasier and better than strict, time does not play a role. Changing the protocol to improve read performance makes write performance worse. CausalMake a distinction between events potentially causally related and those not. Require keeping track of which processes have seen which writes. FIFOEasy to implement.The results may be counterintuitive. Not all applications require seeing all writes,let alone seeing them in order. Weak(1)Enforces consistency on a group of operations,not on individual reads and writes. (2)Limit only the time when consistency holds,rather than limiting the form of consistency When a synchronization variable is accessed,the data store does not know whether this is being done because the process is either finished writing or is about to start reading data. ReleaseTell the difference between entering a critical region and leaving one. Does not associate shared data items with locks or barriers and at acquire time has to determine empirically which variables it needs. EntryMake sure a process can perform multiple reads and writes in a critical section without invoking any data transport. How to properly associating data with synchronization variables.

31. 6.3 Client-centric Consistency Models Presented by: Wu Xiaoling Date: 2004-11-23 Real-Time Multimedia Lab Kyung Hee University

32. Client-centric Consistency Models  Eventual Consistency  Monotonic-Read Consistency  Monotonic-Write Consistency  Read-Your-Writes Consistency  Writes-Follow-Reads Consistency

33. Client-Centric Consistency Models The consistency models in the previous section provide a system-wide consistent view on a data store. This section consider data stores characterized by the lack of simultaneous updates, or when such updates happen, they could be easily resolved. Client-centric consistency provides guarantees for a single client concerning the consistency of accesses to a data store by that client.

34. Client-centric Consistency Models Goal: avoid system-wide consistency: concentrating on what specific clients want, instead of what should be maintained by servers Examples: – DNS – WWW

35. Client-centric Consistency Models Environment – most operations: “ read ” – “ no ” simultaneous updates – a relatively high degree of inconsistency tolerated Wanted – eventual consistency – consistency seen by one single client

36. Client Centric Consistency In some cases if for a long time no update takes place all replicas gradually become consistent Eventual consistency

37. Eventual Consistency A mobile user accessing different replicas of a distributed database has problems with eventual consistency

38. Client-Centric Consistency Models Notations: – : the version of data x at local copy at time t – : the set of a series of write operations at that took place since initialization to obtain – : If operations in have also been performed at at a later time

39. Monotonic Reads If a process reads the value of x, any successive read operation on x by that process will always return the same or a more recent value. a)A monotonic-read consistent data store b)A data store that does not provide monotonic reads.

40. Monotonic Reads (Example) Consider a distributed email database. In such a database, each user’s mailbox may be distributed and replicated across multiple machines. Mail can be inserted in a mailbox at any location. Suppose a user reads his mail in San Francisco. When the user later flies to New York and opens his mailbox again, monotonic read consistency guarantees that the messages that were in the mail box in San Francisco will also be in the mailbox when it is opened in New York.

41. Monotonic Writes A write operation by a process on x is completed before any successive write operation on x by the same process. a)A monotonic-write consistent data store. b)A data store that does not provide monotonic-write consistency.

42. Monotonic Writes (Example) Consider a software library. In many cases, updating a software is done by replacing one or more functions, leading to a next version. With monotonic-write consistency, guarantees are given that if an update is performed on a copy of the library, all preceding updates will be performed first. The resulting library will then indeed become the most recent version and will include all updates that have led to previous versions of the library.

43. Read Your Writes The effect of a write operation by a process on data item x, will always be seen by a successive read operation on x by the same process. a)A data store that provides read-your-writes consistency. b)A data store that does not.

44. Read Your Writes (Example) Consider updating passwords. To enter a digital library on the Web, it is often necessary to have an account with an accompanying password. However, changing a password may take some time to come into effect, with the result that the library may be inaccessible to the user for a few minutes and it may take time to propagate passwords to the various servers that constitute the library. An implementation of read-your-writes consistency would solve this problem.

45. Writes Follow Reads A write operation by a process on x following a previous read on x by the same process, is guaranteed to take place on the same or a more recent value of x that was read. a)A writes-follow-reads consistent data store b)A data store that does not provide writes-follow-reads consistency

46. Writes Follow Reads(Example) Writes-follow-reads consistency can be used to guarantee that users of a network newsgroup see a posting of a reaction to an article only after they have seen the original article. Assume that a user first reads an article A and then reacts by posting a response B. By requiring Writes-follow-reads consistency, B will be written to any copy of the newsgroup only after A has been written as well. Note that users who only read articles need not require any specific client-centric consistency model. The writes-follow-reads consistency assures that reactions to articles are stored at a local copy only if the original is stored there as well.

47. Client centric model-Summary Monotonic read If a process reads the value of a data item x, any successive read operation on x by that process will always return that same value or a more recent value Monotonic write A write operation by a process on a data item x is completed before any successive write operation on x by the same process Read your writes The effect of a write operation by a process on a data item x will always be seen by a successive read operation on x by the same process Writes follow reads A write operation by a process on a data item x following a previous read operation on x by the same process, is garanteed to take place on the same or more recent values of x that was read

48. Xu Hui Real-time and Multimedia LAB Kyung Hee University, Korea Nov. 23,2004 Distribution and Consistency protocols

49. Distribution Protocols  Distribution Protocols  Ways of propagating, distributing updates to replicas  Support consistency model, but independent of it  Content  Replica Placement  Update Propagation  Epidemic Protocols

50. Replica Placement The logical organization of different kinds of copies of a data store into three concentric rings.

51. Permanent Replicas  Permanent Replicas  Initial set of replicas that constitutes a distributed data store  The number of it is small  Two Distribution Scheme  The first one File is replicated across a limited number of servers on a single local-area network Request is forwarded to one of the servers  The second one (Mirroring) File is copied to a limited number of servers, called mirror sites, which are geographically spread across the Internet. Clients simply choose one of the mirror sites from a list offered to them.

52. Server-Initiated Replicas  Server-Initiated Replicas  Created at the initiative of the data store  Exist to enhance performance  Work Scheme how to decide where and when replicas should be created or deleted  Each server keeps track of access counts per file, and where access requests come from.  When a server Q decides to reevaluate the placement of the files it stores, it checks the access count for each file.  If the total number of access requests for F at Q drops below the deletion threshold del (Q,F), it will delete F unless it is the last copy.

53. Server-Initiated Replicas Counting access requests from different clients.

54. Client-Initiated Replicas  Work Scheme  When a client wants access to some data, it connects to the nearest copy of the data store from where it fetches the data it wants to read,  When most operations involve only reading data, performance can be improved by letting the client store requested data in a nearby cache.  The next time that same data needs to be read, the client can simply fetch it from this local cache.  Client-Initiated Replicas  Created at the initiative of clients  Commonly known as client caches  Used only to improve access times to data

55. Update Propagation  Introduction  Generally initiated at a client  Subsequently forwarded to one of the copies  Three design issues  State Vs. Operations What is actually to be propagated  Pull Vs. Push Protocols Whether updates are pulled or pushed  Unicasting Vs. Multicasting Whether unicasting or multicasting should be used

56. State Vs. Operations Three possibilities of What is actually to be propagated  Propagate only a notification of an update  Whenever an operation on an invalidated copy is requested, that copy generally needs to be updated first  Use little network bandwidth  Work best when read-to-write ratio is relatively small  Transfer data from one copy to another  Useful when the read-to-write ratio is relatively high  Propagate the update operation to other copies  Tell each replica which update operation it should perform  Updates can often be propagated at minimal bandwidth costs

57. Pull versus Push Protocols A comparison between push-based and pull-based Protocols in the case of multiple client, single server systems. Whether updates are pulled or pushed  Push-based approach  Updates are propagated to other replicas without those replicas even asking for the updates  Pull-based approach  a server or client requests another server to send it any updates it has at that moment.

58. Unicasting Vs. Multicasting Whether unicasting or multicasting should be used  Introduction  With multicasting, the underlying network takes care of sending a message efficiently to multiple receivers.  In unicast communication, when a server that is part of the data store sends its update to N other servers, it does so by sending N separate messages, one to each server.  Compare  In many cases, it is cheaper to use available multicasting facilities.  Multicasting can often be efficiently combined with a push-based approach to propagating updates.

59. Epidemic Protocols  Introduction  Update propagation in eventual-consistent data stores is often implemented by a class of algorithms known as Epidemic Protocols.  Don’t solve any update conflicts  Only concern is propagating updates to all replicas in as few messages as possible.  Two Parts  Update Propagation Models 1.P only pushes its own updates to Q 2.P only pulls in new updates from Q 3.P and Q send updates to each other  Removing Data

60. Consistency Protocols  Consistency Protocols  Describe implementations of specific consistency models  Models: sequential consistency, weak consistency with synchronization variables, and atomic consistency  Content  Primary-Based Protocols Remote-Write Protocols Local-Write Protocols  Replicated-Write Protocols Active Replication Quorum-Based Protocols  Cache-Coherence Protocols Coherence detection strategy Coherence enforcement strategy

61. Primary-Based Protocols Primary-based remote-write protocol with a fixed server to which all read and write operations are forwarded. Remote-Write Protocols (1)

62. Primary-Based Protocols The principle of primary-backup protocol. Remote-Write Protocols (2)

63. Primary-Based Protocols Primary-based local-write protocol in which a single copy is migrated between processes. Local-Write Protocols (1)

64. Primary-Based Protocols Primary-backup protocol in which the primary migrates to the process wanting to perform an update. Local-Write Protocols (2)

65. Replicated-Write Protocols The problem of replicated invocations. Active Replication (1)

66. Replicated-Write Protocols a) Forwarding an invocation request from a replicated object. b) Returning a reply to a replicated object. Active Replication (2)

67. Replicated-Write Protocols Three examples of the voting algorithm: a) A correct choice of read and write set b) A choice that may lead to write-write conflicts c) A correct choice, known as ROWA (read one, write all) Quorum-Based Protocols