6.4 Data and File Replication Sreekanth Padidala

Why replicate Performance Reliability Resource sharing Network resource saving

Challenge Transparency Replication Concurrent control Failure recovery Serialization

Atomicity In database systems, atomicity is one of the ACID transaction properties. An atomic transaction is a series of database operations which either all occur, or all do not occur[1]. All or nothing

Atomicity In DFS (Distributed File System), replicated objects (data or file) should follow atomicity rules, i.e., all copies should be updated (synchronously or asynchronously) or none.

Goal One-copy serializability: The effect of transactions performed by clients on replicated objects should be the same as if they had been performed one at a time on a single set of objects.[2]

Architecture FSA , File service agent, client interface RM, replica manager, provide replication functions [3]

Architecture[3]

Options for Read/Write Read-one-primary: read from a primary RM (consistency) Read-one: read from any RM (concurrency) Read-quorum: read from a quorum of RMs (currency) WRITE Write-one-primary: write to one primary replica Primary RM propagates the updates to all other RMs Write-all: atomic updates to all RMs (subsequent writes must wait) Write-all-available: atomic updates to all available (non-faulty) RMs Failure recovery Write-quorum: atomic updates to a quorum of RMs Write-gossip: updates to any RM and are lazily propagate to others

Read operations [3] Read-one-primary, FSA only read from a primary RM, consistency Read-one, FSA may read from any RM, concurrency Read-quorum, FSA must read from a quorum of RMs to decide the currency of data

Write Operations[3] Write-one-primary, only write to primary RM, primary RM update all other RMs Write-all, update to all RMs Write-all- available, write to all functioning RMs. Faulty RM need to be synched before bring online.

Write Operations Write-quorum, update to a predefined quorum of RMs Write-gossip, update to any RM and lazily propagated to other RMs

Read one primary, write one primary Other RMs are backups of primary RM No concurrency Easy serialized Simple to implement Achieve one-copy serializability Primary RM is performance bottleneck

Read one, Write all Provides concurrency Concurrency control protocol needed to ensure consistency (serialization) Achieve one-copy serializability Difficult to implement (there will be failed TM to block any updates)

Read one, Write all available Variation of Read one, Write all May not guarantee one-copy serializability Issue of loss conflict in transactions

Read quorum, Write quorum Version number attached to replicated object Highest version numbered object is the latest object in read. Write operation advances version by 1 2*write-quorum > number of replicas Read-quorum+write_quorum > number of replicas Usually read-quorum is chosen to be smaller than write-quorum Voting by witnesses Weighted voting schemes.

Gossip Update Applicable for frequent read, less update situations Increased performance Typical read one, write gossip Use timestamp

Basic Gossip Update Used for overwrite Three operations, read, update, gossip arrive Read, if TSfsa<=TSrm, RM has recent data, return it, otherwise wait for gossip, or try other RM Update, if Tsfsa>TSrm, update. Update TSrm send gossip. Otherwise, process based on application, perform update or reject Gossip arrive, update RM if gossip carries new updates.

Causal Order Gossip Protocol[3] Used for read-modify In a fixed RM configuration Using vector timestamps Using buffer to keep the order

Windows Server 2003[4] Support DFS “State based, multi master” scheduled replication Use namespace for transparent file sharing Use Remote Differential Compression to propagate change only to save bandwidth

Continued[5] If replication detects a conflict, last update wins. No merge files, but copies are kept for reference.

Reference [1] Wikipedia; http://en.wikipedia.org/wiki/Atomicity [2] M. T. Harandi;J. Hou (modified: I. Gupta);"Transactions with Replication";http://www.crhc.uiuc.edu/~nhv/428/slides/repl-trans.ppt [3] Randy Chow,Theodore Johnson, “Distributed Operating Systems & Algorithms”, 1998 [4] "Overview of the Distributed File System Solution in Microsoft Windows Server 2003 R2";http://technet2.microsoft.com/WindowsServer/en/library/d3afe6ee-3083-4950-a093-8ab748651b761033.mspx?mfr=true [5] "Distributed File System Replication: Frequently Asked Questions";http://technet2.microsoft.com/WindowsServer/en/library/f9b98a0f-c1ae-4a9f-9724-80c679596e6b1033.mspx?mfr=true