1. Replication (1)

2. Topics r Why Replication? r System Model r Consistency Models – How do we reason about the consistency of the “global state”? m Data-centric consistency m Client-centric consistency r We will examine consistency protocols which describe an implementation of a specific consistency model. r Other Implementation Issues r Examples

3. Readings r Van Steen and Tanenbaum: 6.1, 6.2 and 6.3, 6.4 r Coulouris: 11,14

4. Why Replicate? r Replication refers to the maintenance of copies at multiple site r Reliability m If one replica is unavailable or crashes, use another m Avoid single points of failure r Performance m Placing copies of data close to the processes using them can improve performance through reduction of access time. m If there is only one copy, then the server could become overloaded.

5. Common Replication Examples r DNA naming service r Web browsers often locally store a copy of a previously fetched web page. m This is referred to as caching a web page. r Replication of a database r Replication of game state

6. System Model r A collection of replica managers (RMs) provide a service to clients m One replica manager per replica r The front-end for a client allows a client to see a service that gives it access to logical objects, which are in fact replicated at the RMs r Clients request operations: some are read-only operations and some are update operations

7. System Model FE Requests and replies C Replica C Service Clients Front ends managers RM FE RM Clients’ request are handled by front ends. A front end makes replication transparent.

8. Phases in Executing Request r Issue request m the FE either sends the request to a single RM that passes it on to the others or multicasts the request to all of the RMs r Coordination m The RMs decide whether to apply the request; and decide on its ordering relative to other requests (according to FIFO or total ordering) r Execution m The RMs execute the request r Agreement m RMs agree on the effect of the request, e.g., perform 'lazily' or immediately r Response m One or more RMs reply to FE.

9. Group Communication r Replication systems need to be able to add/remove RMs r A group membership service provides: m Interface for adding/removing members Create, destroy process groups, add/remove members A process can generally belong to several groups m Implements a failure detector This monitors members for failures (crashes/communication), and excludes them when unreachable m Notifies members of changes in membership m Expands group addresses Multicasts addressed to group identifiers Group expanded to a list of delivery addresses Coordinates delivery when membership is changing

10. Services provided for process groups Join Group address expansion Multicast communication Group send Fail Group membership management Leave Process group A process outside the group sends to the group without knowing the membership Membership service provides leave and join operations Failure detector notes failures and evicts failed processes from the group The group address is expanded Members are informed when processes join/leave

11. Group Communication r Replication systems need to be able to add/remove RMs r A group membership service provides: m Interface for adding/removing members Create, destroy process groups, add/remove members A process can generally belong to several groups m Implements a failure detector This monitors members for failures (crashes/communication), and excludes them when unreachable m Notifies members of changes in membership m Expands group addresses Multicasts addressed to group identifiers Group expanded to a list of delivery addresses Coordinates delivery when membership is changing

12. Developing Replication Systems r Consistency r Faults r Changes in Group Membership

13. A Replication Problem r Multiple copies may lead to consistency problems. r Whenever a copy is modified, that copy becomes different from the rest. r Modifications have to be carried out on all copies to ensure consistency. r The type of application has an impact on the consistency requirements needed and thus on the implementation.

14. Consistency Model r A consistency model describes the rules to be used in updating replicated data r The rules define the order of operations. r Rules used depend on the application r Consistency defined within the context of read and write operations on shared data is data-centric m Strict m Sequential m Causal m FIFO

15. Strict Consistency r Strict consistency is defined as follows: m Read is expected to return the value resulting from the most recent write operation m Assumes absolute global time m All writes are instantaneously visible to all r Suppose that process p i updates the value of x to 5 from 4 at time t 1 and multicasts this value to all replicas m Process p j reads the value of x at t 2 (t 2 > t 1 ). m Process p j should read x as 5 regardless of the size of the (t 2 -t 1 ) interval.

16. Strict Consistency r What if t 2 -t 1 = 1 nsec and the optical fiber between the host machines with the two processes is 3 meters. m The update message would have to travel at 10 times the speed of light m Not allowed by Einsten’s special theory of relativity. r Can’t have strict consistency

17. Sequential Consistency r Sequential Consistency: m The result of any execution is the same as if the (read and write) operations by all processes on the data were executed in some sequential order m Operations of each individual process appears in this sequence in the order specified by is programs. r We have seen this in the banking example m One implementation used Lamport’s clocks.

18. Causal Consistency r Causal Consistency: That if one update, U 1, causes another update, U 2, to occur then U 1 should be executed before U 2 at each copy. r Application: Bulletin board m Possible Implementation: Using vector clocks

19. FIFO Consistency r Writes done by a single process are seen by all other processes in the order in which they were issued r … but writes from different processes may be seen in a different order by different processes. r i.e., there are no guarantees about the order in which different processes see writes, except that two or more writes from a single source must arrive in order.

20. FIFO Consistency r Caches in web browsers m All updates are updated by page owner. m No conflict between two writes m Note: If a web page is updated twice in a very short period of time then it is possible that the browser doesn’t see the first update. r Implementation: m Each process adds the following to an update message: (process id, sequence number) m Each other process applies the update messages in the order received from a single process.

21. Implementation Options: Sequential Consistency r We saw how to use Lamport’s logical clocks for sequential consistency. r Another option is to have a centralized processor that is a sequencer.

22. Implementation Options: Sequential Consistency r We saw how to use Lamport’s logical clocks for sequential consistency. r Another option is to have a centralized processor that is a sequencer. r Each update request it sent to the sequencer which m Assigns the request a unique sequence number m Update request is forwarded to each replica m Operations are carried out in the order of their sequence number

23. Implementation Options: Sequential Consistency r The use of a sequencer also does not solve the scalability problem. m It may become a performance bottleneck. m What if it goes down? r A combination of Lamport timestamps and sequencers may be necessary. r The approach is summarized as follows: m Each process has a unique identifier, p i, and keeps a sent message counter c i. The process identifier and message counter uniquely identify a message. m Active processes (or a sequencer) keep an extra counter: t i. This is called the ticket number. A ticket is a triplet (p i, t i, (p j, c j )). m All other processes are passive

24. Implementation Options: Sequential Consistency r Approach Summary (cont) m Passive processes (non-sequencer) send their messages to their sequencer. m Lamport’s totally ordered multicast algorithm is used among the sequencers to determine the order of update operations. m When an operation is allowed, each sequencer sends the ticket to its associated passive processes. It is assumed that the passive process receives these tickets in the order sent.

25. Implementation Options: Sequential Consistency r Approach Summary (cont) m If a sequencer terminates abnormally, then one of the passive processes associated with it can become the new sequencer. m An election algorithm may be used to choose the new sequencer.

26. Implementation Options: Sequential Consistency r Let’s say that we have 6 processes: p 1,p 2,p 3,p 4,p 5,p 6 r Assume that p 1,p 2 are sequencers; p 3,p 4 are associated with p 1 and p 5,p 6 are associated with p 2 r Let’s say that p 3 sends a message which is identified by (p 3, 1). r p 1 generates a ticket as follows: (p 1, 1, (p 3, 1)) r The ticket number is generated using the Lamport clock algorithm. Ticket number

27. Implementation Options: Sequential Consistency r Let’s say that p 5 sends a message which is identified by (p 5, 1). r p 2 generates a ticket as follows: (p 2, 1, (p 5, 1)) r Which update gets done first? Basically, p 1,p 2 will apply Lamport’s algorithm for totally ordered multicast. r When an update operation is allowed to proceed, the sequencers send messages to their associated processes.

28. Data-Centric Consistency Models r The consistency models just discussed are called data-centric consistency models. r Assumptions: m Concurrently processes may be simultaneously updating m Updates need to be propagated quickly.