1. Distributed Systems: Consistency and Replication Ghada Ahmed, PhD. Assistant Prof., Computer Science Dept. Email: ghada@fcih.net Web: www.fcih.net/ghada/teaching/operating- systems-2 CS 342: OS-2 Fall 2015 1 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E

2. Agenda  Distributed System Overview  Consistency and Replication  Replication, Why?  Replication as Scaling Technique.  Consistency Definition.  Consistency Models. ◦Data-Centric Consistency Models. CS 342: OS-2 Fall 2015 2 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E

3. How does Google Search work? How does your Facebook Timeline stay updated around the clock? How does Amazon scan an ever- growing catalog of items to tell you that people who bought this item also bought socks? Is it magic? No, it’s distributed computing CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 3

4. Distributed System Overview  A distributed system is a collection of independent computers that appears to its users as a single coherent system.  There are four major reasons for building distributed systems: resource sharing, computation speedup, reliability, and communication.  Where are the challenges? 1.No machine has complete information about the system state. 2.Machines make decisions based only on local information. 3.Failure of one machine does not ruin the algorithm. 4.There is no implicit assumption that a global clock exists. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 4 users access remote resources in the same way they access local resources

5. Consistency and Replication CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 5

6. Replication, Why? Two primary reasons reliability performance CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 6  Continue working after one replica crashes by simply switching to one of the other replicas  Provide better protection against corrupted data.  Its important for scaling in numbers or with respect to size of geographical area. availability fault tolerance against data corruption &faulty operations A local cache decrease access time Price to be paid: consistency maintenance

7. Replication as Scaling Technique  Replication and caching for performance are widely applied as scaling techniques.  Price: keeping copies up to date may require more network bandwidth.  Do not install a local replica close to P  If N<<M, where ◦P access the local replica N times per second, and ◦The replica itself is updated M times per second. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 7 The access-to-update ratio is very low  many updated versions of the local replica will never be accessed by P

8. Consistency  Another problem with replication is that having multiple copies may lead to consistency problems.  Whenever a copy is modified, that copy becomes different from the rest.  Suppose that machines A and B open a file simultaneously  If A writes to the file, does B see the change?  If so, when?  What if A has a cached copy?  Usual answer is session semantics: changes are only pushed out when the file is closed.  What real examples do? CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 8

9. Google Apps  Ex: Google Docs-- shared documents.  If several people edit the same doc simultaneously, they will see each other’s changes.  In a word processor doc, users can see one another’s cursors and updates are shown at the level of individual characters as they are typed by any participant.  Users are left to resolve any conflicts that occur, but conflicts are generally avoided because users are continuously aware of each other’s activities.  In the case of a spreadsheet doc, users’ cursors and changes are displayed and updated at the granularity of single cells. If two users access the same cell simultaneously, the last update wins. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 9

10. How to keep the replicas consistent?  Tight consistency (synchronous replication):  when an update operation is performed on one copy, the update should be propagated to all copies before a subsequent operation takes place.  Update is performed at all copies as a single atomic transaction. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 10 Indivisibly Uninterruptable Difficult for large scale network -----take more time

11. How to keep the replicas consistent? (cont.)  Replicas need to reach agreement on when exactly an update is to be performed locally??  Ex: replicas may need to decide on a global ordering of operations using:  Lamport timestamps OR  A coordinator assign such an order.  Price: lot of communication time CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 11 what usually matters is not that all processes agree on exactly what time it is, but rather that they agree on the order in which events occur. The cure may be worse than the disease

12. Data-Centric Consistency Models  Terms to be used:  Consistency – discussed in the context of read and write operations on shared data.  Data store – distributed / shared memory, shared database, or shared file system.  Consistency model: a contract between processes and the data store. It says that if processes agree to obey certain rules, the store promises to work correctly. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 12 The general organization of a logical data store, physically distributed and replicated across multiple processes.

13. Continuous Consistency (1)  Applications specify their application-specific consistency semantics using ---- conits  A conit: is a physical or logical unit of consistency, defined by the application. Ex: blocks of seats on a flight.  For each conit, we quantify consistency continuously along a three independent axes:  Numerical deviation: deviation in numerical values between replicas / number of updates that have been applied to a given replica, but not yet seen by others.  Staleness deviation: relate to the last time a replica was updated.  Ordering deviation: deviation with respect to the ordering of update operations. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 13 Data-Centric Consistency Models Ex: in stock market prices, an application may specify that two copies should not deviate more than $0.02, which would be an absolute numerical deviation. A relative numerical deviation could be specified, stating that two copies should differ by no more than 0.5%. Weather reports typically stay reasonably accurate over some time, say a few hours. Main server may decide to propagate updates to the replicas only once in a while.

14. Continuous Consistency (2:Example) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 14 Data-Centric Consistency Models Consistency unit Logical time vector A=(last known update time at A, last known update time at B) Replica A received this operation from B and has been committed (shaded) Not seen by A Tentative update operations Num.dev.(unseen operation, abs(max ( committed local value – final remote value))) For A num.dev. (1, abs(max((2,0)-(2,5))))

15. Continuous Consistency (3)  Trade-off: if a conit represents a lot of data (complete database) then updates are aggregated for all the data in the conit. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 15 Data-Centric Consistency Models

16. Continuous Consistency (4)  On the other hand: making conit very small is not a good idea, as the total number of conits to be managed grows. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 16 Data-Centric Consistency Models

17. Continuous Consistency (5)  Developers must specify the consistency requirements for their applications.  Ex: AffectsConit(ConitQ, 1, 1); //define conit append message m to queue Q; DependsOnConit(ConitQ, 4, 0, 60); read message m from head of queue Q; CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 17 numerical deviation, ordering deviation, and staleness can be interpreted as that there should be at most 4 unseen update operations at other replicas, there should be no tentative local updates, and the local copy of Q should have been checked for staleness no more than 60 seconds ago. Data-Centric Consistency Models

18. Consistent Ordering of Operations  An important class of models comes from the field of concurrent programming.  Many physically separate software developers may work together on one white board design. need to share resources and access these resources simultaneously  Consistently ordering operations on shared, replicated data augment those of continuous consistency in the sense that when tentative updates at replicas need to be committed, replicas will need to reach agreement on a global ordering of those updates. they need to agree on a consistent ordering of those updates CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 18 Data-Centric Consistency Models

19. Consistent Ordering of Operations Sequential Consistency (1)  process P i writes the value a into a data item x, then a process P i reads the value b from a data item x (initially nil) respectively; W i (x)a and R i (x)b CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 19 Data-Centric Consistency Models Time It took some time to propagate the update Acceptable

20. Consistent Ordering of Operations Sequential Consistency (2)  A data store is “sequentially consistent,” when the result of any execution is the same as if the (R&W) operations by all processes on the data store  were executed in some sequential order and  the operations of each individual process appear in this sequence in the order specified by its program. All processes see the same interleaving of operations See all W but only its R CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 20 Data-Centric Consistency Models

21. Consistent Ordering of Operations Sequential Consistency (3) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 21 Data-Centric Consistency Models sequentially consistent data store not sequentially consistent Sequence not Time

22. Consistent Ordering of Operations Sequential Consistency (4) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 22 Data-Centric Consistency Models Consider three concurrently executing processes P1, P2 and P3 The output of P1, P2, and P3 in that order 6-bit string…64 patterns Is 000000 allowed??

23. Consistent Ordering of Operations Sequential Consistency (5)  The contract between the processes and the distributed shared data store is that the processes must accept all of these as valid results.  The processes must work correctly if any of valid sequences occurs.  A program that works for some of these results and not for others violates the contract with the data store and is incorrect. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 23 Data-Centric Consistency Models

24. Consistent Ordering of Operations Causal Consistency (1)  A causal consistency model obeys the following:  Writes that are potentially causally related ◦must be seen by all processes ◦in the same order.  Concurrent (i.e., causally unrelated) writes ◦may be seen in a different order ◦on different machine CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 24 Data-Centric Consistency Models

25. Consistent Ordering of Operations Causal Consistency (2) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 25 Data-Centric Consistency Models For example, W1(x)a and R2(x)a are causally related, but W2(x)b and W1(x)c are merely concurrent, which allows P3 and P4 to see them indifferent orders.

26. Consistent Ordering of Operations Causal Consistency (3) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 26 Data-Centric Consistency Models shows a violation of causal consistency, because b (written by W2(x)b) may be computed from a (read by R2 (x)a) and P3 and P4’s reading orders do not reflect that. does not violate causal consistency, because W1(x)a and W2(x)b are merely concurrent

27. Sequential and causal consistency are defined at the level read and write operations. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 27 Data-Centric Consistency Models

28. Grouping Operations (1)  Multiprocessors, which share a store, enforce mutual exclusion from coded critical sections (CS), instead of doing write and read operations.  Upon entering a CS, the processor is guaranteed that its local data store is up to date.  All reads and writes of its owned objects inside the CS are atomic.  No other processes can read or write those objects, till that processor leaves the CS. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 28 Data-Centric Consistency Models

29. Grouping Operations (2)  Necessary criteria for correct multiprocessor synchronization: 1.at an acquire, all remote changes to the guarded data must be made visible. 2.before updating a shared data item, a process must enter a critical section in exclusive mode to make sure that no other process is trying to update the shared data at the same time. 3.if a process wants to enter a critical region in nonexclusive mode, it must first check with the owner of the synchronization variable guarding the critical region to fetch the most recent copies of the guarded shared data. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 29 Data-Centric Consistency Models

30. Grouping Operations (3) CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 30 Data-Centric Consistency Models A valid event sequence for entry consistency

31. Client-Centric Consistency Models  Data-centric: assume concurrent processes may be simultaneously updating the data store.  Client-centric:  Most operations: “read”  Lack of simultaneous updates, and when happened it can be easily resolved. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 31 Client-Centric Consistency Models

32. Eventual Consistency  Assume: the only situation that needs to be handled:  Read-write conflicts: ◦One process wants to update a data item while another is concurrently attempting to read that item.  Eventual Consistency  In case of large-scale distributed and replicated DB: if no updates take place for a long time, all replicas gradually become consistent.  Requires only that updates are guaranteed to propagate to all replicas.  Works fine as long as clients always access the same replica CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 32 Client-Centric Consistency Models

33. Eventual Consistency: Mobile user example CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 33 Client-Centric Consistency Models Different replicas are accessed over a short period of time.

34. Client-Centric Consistency Models  Client-centric consistency provides guarantee for a single client concerning the consistency of access to a data store by that client.  No guarantees are given concerning access by different clients.  Client-centric consistency models:  Monotonic reads  Monotonic writes  Read-your-writes  Write-follows-reads CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 34 Client-Centric Consistency Models

35. Monotonic reads  A data store is said to provide Monotonic-read consistency If a process reads the value of a data item x, any successive read operation on x by that process will always return that same value or a more recent value. = If a process has seen a value of x at time t, it will never see an older version of x at a later time. Example: Each time you connect to a different email server, the server fetches all the updates form the server you previously visited CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 35 Client-Centric Consistency Models

36. Monotonic reads: Example CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 36 A monotonic-read consistent data store does not provide monotonic reads WS(x i ): write set = sequence of operations on x at local copy Li Client-Centric Consistency Models

37. Monotonic Writes  A data store is said to provide Monotonic-write consistency If a write operation by a process on a data item x is completed before any successive write operation on x by the same process. = The new write must wait for old ones to finish Example: Maintaining versions of replicated files in the correct order everywhere (propagate the previous version to the server where the newest version is installed) software updates CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 37 Client-Centric Consistency Models

38. Monotonic Writes: Example CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 38 Client-Centric Consistency Models A monotonic-write consistent data store. A data store that does not provide monotonic-write consistency. WS(x i ): write set = sequence of operations on x at local copy Li The write operations performed by a single process P

39. Read Your Writes  A data store is said to provide read-your-write consistency If the effect of a write operation by a process on data item x, will always be seen by a successive read operation on x by the same process. = A write operation is always completed before a successive read operation by the same process. Example: Updating your Web page and guaranteeing that your Web browser shows the newest version instead of its cached copy. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 39 Client-Centric Consistency Models

40. Read Your Writes: Example CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 40 Client-Centric Consistency Models WS(x i ): write set = sequence of operations on x at local copy Li A data store that provides read-your-writes consistency A data store that does not The effect of the previous write operation by P have not been propagated to L2

41. Writes Follow Reads  A data store is said to provide writes-follow-reads consistency If a write operation by a process on a data item x following a previous read operation on x by the same process, is guaranteed to take place on the same or a more recent value of x that was read. Example: users of a network newsgroup see a posting of a reaction to an article only after they have seen the original article CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 41 Client-Centric Consistency Models

42. Writes Follow Reads: Example CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 42 Client-Centric Consistency Models WS(x i ): write set = sequence of operations on x at local copy Li A writes-follow-reads consistent data store A data store that does not provide writes-follow-reads consistency

43. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 43 monotonic-read monotonic-write read-your-writes writes-follow-reads Client-Centric Consistency Models email server software updates Updating your Web page newsgroup

44. Replica Management  Regardless of which consistency model is chosen, we need to decide where, when and by whom copies of the data- store are to be placed.  Placement Problem:  Replica-server placement ◦To find the best location to place a server that can host a data store.  Content placement ◦To find the best server for placing content. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 44

45. Replica-server placement  To compute the best placement of replica servers:  Optimization problem: to select the best K locations out of N, where K<N. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 45 Replica Management Too few replica servers would be chosen Too many replica servers would be chosen Best Cell size: the average distance between 2 nodes and the number of required replicas

46. Content Replication and Placement CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 46 Replica Management

47. Permanent replicas  Statically configured.  Used as backup facility and used as the only replicas that are allowed to be changed to guarantee consistency.  Initial set of replicas that constitute a distributed data store. (small number)  Example: Distribution of a Web Site 1.Replication across a limited number of servers at a single location. 2.Mirroring: Replication across a limited number of servers which are geographically spread across the internet. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 47 Content Replication and Placement

48. Server-Initiated Replicas  Dynamically placing replicas.  Typically used by web hosting companies to geographically locate replicas close to where they are needed most.  Used for placing read-only copies close to clients.  When a server Q decides to evaluate the placement of the files it stores:  It checks the access count for each file. If the total number of access requests for F at Q drops below deletion threshold del (Q,F), it will delete F unless it is the last copy.  if for some server P, access count cntQ(P,F) exceeds more than half of the total requests for F at Q, server P is requested to take over the copy of F. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 48 Content Replication and Placement

49. Server-Initiated Replicas: Counting access requests from different clients CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 49 Content Replication and Placement Consistency: responsibility of the data store

50. Client-initiated replicas  Created as a result of client requests to improve access time to data.  Example: browser caches.  Works well assuming, of course, that the cached data does not go stale too soon.  Placement of client caches is relatively simple: a cache is normally placed on the same machine as its client, or otherwise on a machine shared by clients on the same local-area network. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 50 Content Replication and Placement Consistency: responsibility of client

51. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 51

52. Content Distribution Update Propagation: State vs. Operations What is to be propagated? 1.Propagate only a notification of an update. Invalidation protocol: other copies are informed that an update has taken place and that the data they contain are no longer valid. Less network bandwidth. useful if low read-to-write ratio 2.Transfer data from one copy to another. useful if high read-to-write ratio 3.Propagate the update operation to other copies (active replication) to tell each replica which update operation it should perform (and sending only the parameter values that those operations need). CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 52 Replica Management

53. Content Distribution Update Propagation: Pull vs Push Protocols Updates are pushed or pulled?  Push-based/Server-based Approach: sent “automatically” by server,  The client does not request the update.  Useful when a high degree of consistency is needed.  Often used between permanent and server-initiated replicas.  Pull-based/Client-based Approach: used by client caches (e.g., browsers), updates are requested by the client from the server. No request, no update! CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 53 Replica Management

54. Content Distribution Update Propagation: Pull vs Push Protocols (cont.)  Hybrid schemes are possible: e.g., “leases” – a promise from a server to push updates to a client for a period of time.  Once the lease expires, the client reverts to a pull-based approach (until another lease is issued). CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 54 Replica Management IssuePush-basedPull-based State on server.List of client replicas and caches.None. Messages sent.Update (and possibly fetch update later).Poll and update. Response time at client. Immediate (or fetch-update time).Fetch-update time.

55. Content Distribution Update Propagation: Unicasting vs Multicasting  Multicasting can often be efficiently combined with a push-based approach to propagating updates.  When the two are carefully integrated, a server that decides to push its updates to a number of other servers simply uses a single multicast group to send its updates.  With a pull-based approach, it is generally only a single client or server that requests its copy to be updated.  In that case, unicasting may be the most efficient solution. CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 55 Replica Management

56. Extra Reading list  Haifeng Yu and Amin Vahdat. 2002. Design and evaluation of a conit-based continuous consistency model for replicated services. ACM Trans. Comput. Syst. 20, 3 (August 2002), 239-282. DOI=http://dx.doi.org/10.1145/566340.566342  Szymaniak, M.; Pierre, G.; van Steen, M., "Latency-driven replica placement," in Applications and the Internet, 2005. Proceedings. The 2005 Symposium, pp.399-405, 31 Jan.-4 Feb. 2005 doi: 10.1109/SAINT.2005.37 CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 56

57. Next time Distributed File Systems CS 342: OS-2 Fall 2015 DISTRIBUTED SYSTEMS: PRINCIPLES AND PARADIGMS BY TANENBAUM, ANDREW S., VAN STEEN, MAARTEN 2E 57